Weekend Reading #301 — Three Body Capital

This is the three-hundredth-and-first weekly edition of our newsletter, Weekend Reading, sent out on Saturday 8th February 2025

To receive a copy each week directly into your inbox, sign up here.

*****

What we’re thinking.

We wrote last week about the two things really on our mind (and likely everyone else’s). Nvidia and the dollar. The past week has allowed everyone to further crystallize thinking on what all these developments with AI mean. Interesting to us has been the market reaction. Amazon, Google and Meta in particular went to all-time highs rather fast (pre-results) and Nvidia plunged. Does this mean that the market believes these companies are bound for dramatically lower capex? We don’t know yet but maybe. Google’s results gave some comfort on near term capex and stopped the Nvidia plunge for now, but time will surely tell on this one. These companies themselves are still trying to figure it all out. Either way as we said last week, Nvidia is sure as heck not the slam dunk it was a couple of weeks ago when the only discussion was on its valuation. Some will argue the adjustment has already happened in its stock price. We are less sure.

Last week we started thinking the dollar also may have reached some kind of near-term peak and begun to come down but then we all got the “non-tariffs”. The fascinating thing is that today the dollar is weaker than it was last week, tariffs or not. Further food for thought. And how can we let the week go by without opining on Trump, his tariffs, USAID and all that stuff. Our brief two pennies worth is that Trump is a right brained master dealing with left brained leaders and politicians. He is playing a different game. And if you are reading opinions, best make sure you are reading genuine original thinking, not those on the payroll of USAID.

What we are reading.

On Monday I sent myself this image to remember to put it into the newsletter.

On Wednesday (2 days later) I came across this one with the addition of OpenAI’s Deep Research.

This reminded me of a chart I came across nearly 10 years ago from Tim Urban on his Waitbutwhy website.

It caused me a great existential moment (which has continued). All those theories about what happens when the machines overtake us? AGI, ASI etc. Well, here we are. Now what?

Separately, I read Aaron Stupple’s new book, The Sovereign Child, over the past couple weeks after discovering it from Naval’s Twitter feed. His theory in a nutshell whether it applies to eating, bedtime, screens or anything else for that matter is let kids do whatever they want without any restrictions. The thinking goes that kids are smarter than we give them credit for and will figure out good habits from bad habits themselves. My personal take is that in some areas this could be a good idea but in most it’s an absolute disaster. There is a reason kids have parents! Most kids today are obscenely overscheduled and have absolutely zero freedom and this no doubt has negative effects so some suggestions on improving this are welcome. On the other hand, after the plethora of evidence in the past year or two, I think it is clear now that overexposure to social media-dominated smart phones and iPads are terrible for kids in terms of the effects on brain development, especially younger kids. To be honest there were many parts of this book I actually laughed out loud at as they were so bizarre. But what I would love to do is listen to a conversation between Jonathan Haidt and Stupple on these issues which affect every kid (and most importantly their parents!). That would be pure gold. DC

Do AI models have personalities? It turns out that they’re pretty much coded to please, according to this study that revealed that LLM models exhibit social desirability bias that “exceeds typical human standards”. And even more so when the LLM recognise that their personality is being tested, in which case they lean towards appearing more likeable, increasing traits like extraversion, conscientiousness, openness and agreeableness. Perhaps it’s the personalities and biases of the “trainers” that are being amplified in the generation of these models, and well-meaning as the original trainers may be, the impracticality of being “always positive” risks translating into users having an unrealistic expectation of what they are able to achieve with the help of an “AI assistant”. Sometimes, the answer is just “sorry, that’s not happening” (and not just to cheeky queries to a Chinese AI about censored subjects). To address that, alternative models have been developed in open-source land, such as Negative LLAMA, which – as its name suggests – tends to be a little bit less nice, and a little more down to earth. Life isn’t always rainbows and butterflies, and Negative LLAMA knows that. EL

What we are watching

A show I’ve recently found myself gripped to is the Apple TV original, Severance. Initially, I was quite hesitant as much of the content I've watched on Apple’s platform has been rather mediocre and uninspiring. However, this one turned out to be perhaps one of the strangest yet most brilliant shows I’ve watched in a long while (and trust me, it gets good once you’re even further in). The attention to detail is unmatched with hidden easter eggs in almost every scene. With the second season now releasing week by week, I’ve even found myself watching the post-episode breakdowns on YouTube where commentators discuss fan theories around the ongoing questions the show raises and unpick some of the scenes. The show is a sci-fi dystopian version of office life in a world where employees undergo surgical procedures in order to separate their work from their personal lives. HS

Unable to resist the recent Elon/Trump battle with the establishment USAID proponents last week I got sucked into watching what turned out to be a superb soliloquy from Senator John Kennedy of Louisiana (no relation to the famous family). He appears to be a proper character. Brilliant accent and so succinctly put together that I genuinely enjoyed it. Highly recommend for those into spectator sports. And he is spot on (and he knows it). DC

What we are listening to

My favourite historian, Simon Sebag Montefiore, was on Bari Weiss’s Honestly podcast this week talking about the end of history or lack thereof and where we are in this moment regarding the west, war, civilization and all such topics. A fantastic listen. DC

What we’re doing.

It’s one thing to read about people deploying private DeepSeek instances – it’s another to attempt to do it. I’d previously had a go at deploying a machine learning instance which wasn’t that fruitful. This time, I had the benefit of Claude. Asking one AI for instructions for deploying another AI model on AWS directly via Powershell and Linux Ubuntu was definitely exciting, and most definitely educational – the entrance to a rabbit hole down LLM land.

In theory, this should’ve been a walk in the park. Ask Claude for instructions on how to deploy DeepSeek R1 on AWS, with step by step instructions for selecting the right compute instance and all other system requirements based on the paper released by the DeepSeek team and a reference to its Github documentation. In reality, this took hours: Claude made errors on suggested command line executions, failed to suggest the correct technical requirements (insufficient RAM and storage), failed to include downloading required dependencies (including those generally required for models prior to DeepSeek)… but we got there in the end.

DeepSeek would've required buying too much AWS capacity for an experiment, but just for the experience, I settled on a deployment of a simple version of Facebook's OPT model (from 2022) with only 350m parameters that managed to write a short poem about a cat, which probably wouldn’t make it very far in a poetry competition…

(venv) ubuntu@ip-172-31-21-185:~$ python3 test_model3.py

Starting model test...

CUDA available: False

Initialising model...

Device set to use cpu

Model initialised successfully

Testing with prompt: Write a creative poem about a cat. Be descriptive and imaginative.

Poem:

Asking to truncate to max length but no maximum length is provided and the model has no predefined maximum length. Default to no truncation.

Model response:

Write a creative poem about a cat. Be descriptive and imaginative.

Poem:

I'm the Cat

What do you like about me?

I have a soft spot for cats

I'm a big fan of your cat

What do you do with your cat?

I have a cat

Learning no. 1: LLMs work without GPUs and without CUDA. They’re much slower on CPU, but if you’re not training a new model and trying to derive model weights, the huge GPU compute capacity is probably not necessary.

Learning no. 2: To prove the point, if you have a decent enough system, and don’t have the tendency to subject yourself to Command Line torture, download LM Studio and take your pick of LLMs to download and run locally. You might find that these work on a system with much less processing power than you’d imagine is needed.

Learning no. 3: It’s not so much AI per se, rather compression. Llama 3.3 (Meta’s latest model) with 70bn parameters is 37.1gb; Llama 3.2 with 1bn parameters is 1.32gb. Alibaba’s Qwen 2.5 model optimised for coding purposes is 8.5gb for 14bn parameters vs 18.6gb for 32bn parameters and 2gb for 3bn parameters. DeepSeek R1 distilled into Qwen’s 7bn model is a similar size to the original Qwen 2.5 7bn parameter model at 4.7gb. There are even tiny models that are below 500mb to play with, specialising in simpler tasks.

Notice the pattern? More parameters = better accuracy = bigger size = more data to tap in generating responses. The implication of this is important: every LLM model is a slightly different variation of how best to compress the training data (a snapshot of the entire internet, supposedly 64 zettabytes) into something below 70gb. That’s a compression ratio of 100m times – a really powerful ZIP file. Put differently, the “model” is a probabilistic formula for inferring from 70gb of “stuff” what the 64 zettabyte internet universe is, based on the given prompt. Does more training mean a better model? Not necessarily. More parameters might help, but the models start getting bigger and slower. Hence the quality of the prompt (i.e. the human using it) is paramount to determining the quality of the output, and more likely than not, incremental gains in model performance are (in my view) more likely to come from improvements in methodology (e.g. "Mixture of Experts", uncensoring models through "abliteration", optimising models for CPU use, blending different models etc) rather than more brute force training.

Whether this takes us to a superintelligence is probably still up for debate. But what’s NOT up for debate is this: anyone performing a role that is essentially a memory recall and regurgitate function with little/no critical thinking is now officially worth… £18/mth at most. And their competition doesn’t eat, sleep, need days off or complain, while literally knowing it all (almost). Things are going to get interesting.

As an aside, we put the same prompt for the Cat poem to Claude Opus (with 137bn parameters, almost 400x that of my original OPT experiment), which fortunately did a much better job. No surprise what having more parameters can do!

Here is a creative poem about a cat:

Feline Fascination

Silky fur of midnight black,
Emerald eyes that shimmer bright.
With padded paws and supple back,
She slinks through shadows of the night.

Observe her graceful form in motion,
A tigress stalking through the room.
Contented purrs, a soothing potion,
Her presence chases away gloom.

By sunny window she reclines,
A regal queen upon her throne.
Through dreamtime's twists and winding vines,
To lands enchanted she is flown.

Mischief dances in her gaze,
When flicking tail and crouching down.
What secret plots do her thoughts raise?
Is some vase destined to crash down?

She leaps with effortless precision,
Pouncing on a catnip mouse.
Defying gravity's firm decision,
The acrobat of any house.

Independent, coy and clever,
She'll bestow affection as she may.
Though vanishing for hours forever,
Near her human she will stay.

A sphinx of magic and illusion,
Both fierce and fragile, meek and wild.
With feline grace in soft profusion,
This purring riddle, nature's child.

Maybe time to add "poets" to the list of jobs under threat. EL