This post summarizes the Y Combinator Lightcone episode The 10 Trillion Parameter AI Model With 300 IQ (published November 1, 2024). The views below are the hosts’ opinions and speculation, not announcements. The episode does not announce a 10 trillion parameter model.
OpenAI had just raised $6.6 billion, which the hosts call the largest venture round ever. The episode asks what a world with much larger models would look like.
Key Takeaways #
- In the episode, OpenAI’s CFO says the money goes to compute first, and that compute is not cheap.
- The hosts imagine a model with 10 trillion parameters, two orders of magnitude above today’s frontier models. It is a thought experiment.
- The hosts argue that more accurate models could let developers spend less time on prompts and more on user experience.
- Distillation of larger models into smaller ones may be how most people benefit from large models.
- Voice applications were among the fastest growing in the hosts’ YC batch.
The Impact of OpenAI’s Funding #
In the episode, OpenAI CFO Sarah Friar says of the raise: “it’s compute first and it’s not cheap,” talent comes second, and the rest is normal operating expenses. She also says the company is on a scaling law where orders of magnitude matter, so the next model is an order of magnitude bigger, which makes it capital intensive.
The 10 Trillion Parameter Model #
The hosts ask what happens with a 10 trillion parameter model. They say frontier labs do not publish parameter counts, and that models like GPT-4o and Anthropic’s are speculated to be in the “500s of billions.” That makes 10 trillion about two orders of magnitude more. For scale, Meta says it believes its 405 billion parameter Llama 3.1 is the world’s largest openly available foundation model.
One host compares the possible jump to GPT-3, which has about 175 billion parameters (the episode says “170ish”) and which he credits with starting a wave of new AI companies in 2022 to 2023. That comparison is their opinion. The episode also notes such a model might be slow and costly to run, so it may only be worth using for the hardest problems.
The Shift in Development Focus #
The hosts discuss a more optimistic scenario. Founders spend a lot of time getting prompts to work and outputs to be accurate. If models become more deterministic and accurate, they say, founders could spend that time on ordinary software work such as UI and customer experience. They also say this could mean more competition, and that the winners would be whoever builds the best user experience. These are their predictions.
Distillation: Making AI More Accessible #
Distillation means using a large model to train or improve a smaller, cheaper one. The hosts say there is some evidence that Meta’s Llama 405B was mostly useful for making its 70 billion parameter model much better. Meta’s Llama 3.1 announcement says it used the 405B model “to improve the post-training quality of our smaller models.” The Llama 3 paper describes the family. The hosts also say OpenAI has enabled distillation inside its own API, for example from o1 into GPT-4o mini.
The Rise of Voice Applications #
The hosts say voice applications were among the fastest-growing in the summer 2024 YC batch. They mention companies using voice AI for debt collection and logistics calls, and OpenAI’s realtime voice API, which they say costs about $9 per hour. They say latency and handling interruptions had been the problem before.
Incumbents and Adoption #
The hosts say many engineering teams at established companies, especially those founded four or more years ago, do not take these tools seriously. They wonder whether that is generational, and say people used to slower technology change may be cynical. This is their speculation.
The Future of AI Development #
The hosts close with a hypothetical: if models were smart enough to do original analysis across millions of scientific papers, they might speed up scientific discovery. They call this a bullish case, not a forecast.
