The computing demand of ML is endless

The NVDA stock drop on the DeepSeek news is a result of a fundamental misconception about semiconductor economics in the age of AI.

2025-01-27

Against all conventional wisdom and common sense, I give investing advice to my friends and family.

Fortunately, I've had a good track record so far:

  • I've been extremely bullish on tech my entire life--avoiding broad market index funds in favor of the magnificent 7.
  • I seek out information from people much smarter than myself, like the superforecasters of the rationalist community, which has helped me navigate events like the 2020 pandemic.
  • I spend way too much time online which has helped with events like the 2021 $GME fiasco.

My number one stock pick has been $NVDA since witnessing GPT-3 in 2020. Here's an excerpt from one of my very formal and professional memos from around that time:

"My core observation is this: ever since ImageNet 2012, AI has progressed at an exponential rate faster than Moore’s Law. More and more money is being pumped into it. It feels like each new discovery opens up a dozen new potential areas of research. Anyone who claims that the field is approaching a maturation point has no idea what they’re talking about. Therefore, my core prediction is that this growth is not nearing the end of its S-curve, or even the middle, but rather it is just getting started with initiating the next tech era. Given how much disruption AI has already created, it’s reasonable to expect more disruption than the dotcom boom and the smartphone/social media boom combined.

If this is true, then historically-speaking this is the period where long-term individual stock picks are incredibly risky because large established companies are often too slow at adapting and end up losing to small unrestrained teams of young hungry engineers. [...] On the other hand, companies are smarter now. The smartest of them know what’s coming and are snapping up ML researchers as fast as universities can pump them out. Also the barrier to entry is much higher now. Facebook was created in a dorm room but the next S-curve’s equivalent won’t be. The computing power needed for a competitive AI company is astronomical. 

So with all that being said, here are my picks, invest at your own risk.

1 NVDA

Pros

  • Jensen Huang. I’ve watched a lot of interviews with a lot of different tech CEOs and he is one of the few who I feel truly understands the implications of what is coming in the next decade
  • The computing demand of ML is endless
  • CUDA has a tight grip on the balls of the ML community. We can never leave Nvidia and they know that :’(
  • Gaming is growing fast
  • Ampere doubled the perf of Turing, and it’s looking like Lovelace will 2x Ampere. Idk how the fuck we’re still seeing hardware perf doubling every 2 years in a post-Moore’s Law world but somehow Nvidia is doing it

Cons

  • Hardware company so it doesn’t have that “untethered from reality” growth often seen on the software side of things (although they are branching into software with stuff like DLSS)

"

This has aged pretty well. We were indeed at the beginning of exponential growth for AI, and CUDA does indeed have a tight grip on the ML world (although this is grip is loosening).

Everyone knows all of this now, but there is still one part of my memo that is still controversial: "the computing demand of ML is endless"--so this essay will focus on that statement.

Today (2025-01-27), Nvidia stock plummeted 17% on the news that the Chinese AI lab DeepSeek released an open source model on par with the best closed source models that was trained with a budget of around $5 million--a small fraction of the rumored training costs for frontier models.

China is definitely catching up to the US. The DeepSeek technical report is really impressive--this company has some extremely talented engineers. It's funny that $NVDA is cratering when a large portion of the technical report is dedicated to their Nvidia-GPU-specific innovations.

My thesis is that the selloff of semiconductor-related stocks is based on a fundamental misunderstanding of the DeepSeek situation. If anything, Nvidia/TSMC/etc. should be valued far more given recent news. People are expecting efficiency gains of training/inferencing frontier AI will lead to decreased demand for chips, but it'll actually increase demand because costs will fall. This is related to Jevons paradox: increased efficiency in the use of a resource can paradoxically lead to an overall increase in its consumption. AI capability probably has no ceiling and AI use-cases are surely endless, so AI doesn't seem to be one of those things where society as a whole will say "we have enough of that, we don't need any more". Likewise, SOTA model inference being much cheaper means much more inference will occur as new uses for LLMs become economically viable.

The demand for compute will continue to explode because there are so many uses for it:

  • Serving users. Demand for inference compute will skyrocket as frontier models get larger and more parts of the economy are automated.
  • Training bigger foundation models. Exponential increases in compute required for linear gains in capability.
  • Chain-of-Thought RL. The DeepSeek paper proves that RL on the CoT works really really well and RL is very compute intensive so demand will explode. The reasoning models hunger for an infinite amount of compute. More compute means you can search through more of the CoT space. There's also a feedback loop that is just getting started: you take a base model and do RL on the CoT. You then distill that capability into the next base model and do RL on its CoT and so on. This process scales directly with compute with no end in sight.
  • Generating synthetic data (uses tons inference compute).
  • Doing novel research (more compute means your researchers can launch many different simultaneous experiments).
  • Automating research. The most forward thinking AI companies have their eyes on the endgame: the self-improving feedback loop of AI making better AI. The data for these artificial engineers/scientists will be "this training configuration leads to these results". To collect this data, you need to actually run these training jobs, which is incredibly compute intensive.
  • A SOTA open-source model means more companies will finetune their own models using DeepSeek as a base, which means more demand for Nvidia hardware.

Yes, pretraining scaling is slowing down, but AI labs will still be training larger and larger models as more data becomes available. Even if they're not releasing these larger models due to the cost of serving them for many users, they're still using them internally to generate synthetic data and distill those capabilities into small models.

The commoditization of intelligence is in full-swing. AI is becoming very cheap. This is bad for those who rely on foundation-model-as-a-service as their main revenue stream e.g. Openai, Anthropic. But the frontier labs have always known that they have no moat (see the Google memo "We have no moat, and neither does OpenAI"). This revenue is not why investors are pouring money into these companies. They are shooting for AGI and the technological singularity. The winner of that race will win capitalism itself.

The non-semiconductor stock selloff is kind of reasonable. The crazy valuation of AI labs is partially because of the possibility that they are the ones to create AGI, and since DeepSeek has just revealed itself as a very serious competitor in this moonshot, you would expect slightly lower valuations of these labs.

But the semiconductor stock selloff is foolish. All of Nvidia's main customers believe that AGI is within reach and will lead to complete economic transformation. They don't care that the revenue from serving AI models is withering away. They will still be buying up as many chips as they can until they reach their goal.

Who should be panicking? Meta. The entire value-proposition of Llama is being the best open source model, but DeepSeek is now doing that better than them. I wonder if DeepSeek's decision to open-source was ideological or strategical (forcing closed-source western AI labs to have thinner margins).

The widespread panic in the US stock market has surely caught the attention of the CCP and I bet they'll throw their full weight behind their domestic AI labs. DeepSeek's parent company (High-Flyer) will definitely see their surprise massive success in the west as a signal to pour way more resources into DeepSeek. We should expect them to be a significant player in this race.

So why the big Nvidia sell off? My guess is it's just shallow thinking: "less compute-intensive training/inference means fewer gpus" without thinking about the second-order effects of this. But here is my attempt at a reasonable bear case for Nvidia:

  • DeepSeek's success will cause China to go all-in on creating their own GPUs to get around export restrictions.
  • Nvidia's competitors will soon finally catch up to them (maybe Google will start selling TPUs or maybe ROCm will finally be close to as good as CUDA)
  • This news causes the less "AGI-pilled" people in DC to decide that the USA's multi-billion dollar cluster investments are not worth it.

Nevertheless, we are in a gold rush and Nvidia will still sell every shovel they can manufacture to the desperate companies that want to be the first to get to AGI.