Compute & Cost
Test-Time Compute
The most recent lever: not the model gets bigger, but its thinking time per question: more intermediate steps, several attempts, internal checks. Quality thus becomes a runtime decision with a price tag: the routine question gets seconds, the contract problem minutes. Both have to be budgeted.
An example
For a long time the rule was: a model's capability is fixed after training. Test-time compute breaks with that. The same model gets better when it is allowed to compute longer at request time: generate several solution paths, compare, discard, pick the best. On difficult tasks the difference is considerable. On easy ones it is zero, and the extra computing time is wasted.
A closer look
Instead of training a larger model, the existing one gets more computing time while answering: longer derivations, multiple attempts, selection among the results. For tasks with a verifiable outcome, the gain is considerable.
Why it matters
This is the most economically interesting shift of recent times, because it turns quality into something purchased per transaction rather than a property of the model. That makes it plannable: a transaction with a high cost of error may take more computing time than one that is cheap to correct. This tiering should be a domain decision, not a default setting. And it explains why flat per-request price quotes are becoming increasingly useless.
Commonly misunderstood
The price is felt immediately, because it is incurred per request. For simple tasks it is the most expensive route to an answer that would have been right at lower cost too.