Compute & Cost
Scaling Laws
The empirical observation that explains the arms race: more data, more parameters, and more compute time yield better models along a surprisingly smooth curve, predictable enough to build billion-dollar investments on. Whether the curve holds or flattens is the most expensive open question in the industry; most recently, growth shifted from training to compute time while answering.
An example
For years a surprisingly reliable observation held: more computing power, more data, and more parameters yield better models to a predictable degree. This was not theory but a measured curve, and it allowed something rare in research: before a training run costing hundreds of millions of euros, you could say fairly precisely how good the result would be. The investment decisions of recent years rested on exactly this.
A closer look
Found empirically: error falls predictably with model size, data volume, and computing effort when these grow in the right proportion. That made investments plannable: you knew roughly in advance what more money would buy.
Why it matters
What is interesting is what has become of the curve. The gain from pure enlargement has become noticeably flatter, and the emphasis has shifted, toward better data quality and toward compute time during the request rather than during training. For companies this means above all one thing: the assumption that “the next model will automatically do everything better” no longer holds reliably. Progress now comes unevenly, sometimes in reasoning, sometimes in cost, sometimes in context length. Anyone planning plans better against today's capability.
Commonly misunderstood
The curves say something about prediction error, not about usefulness. And they flatten: the gain per doubling of effort gets smaller, which has shifted the emphasis to data quality and post-training.