Compute & Cost
Compute
The currency of the AI era: computing power for training and operation, measured in chip-hours and electricity bills. Access to compute decides who stays at the front, hence the billions for data centers and the export controls on chips. For companies, compute translates simply into: what does an answer cost us, and what is it worth?
An example
Compute splits into two line items that have nothing to do with each other in business terms. Training a large model is a one-time investment in the hundreds of millions, borne by the vendor. Operation, every single request, is a running expense, borne by you. For a company that buys models rather than building them, only the second item matters, and it grows with usage, not with the project.
A closer look
Compute is incurred twice: once in bulk during training, continuously during operation. Over the lifetime of a successful model, the operating share often exceeds training considerably, because it grows with every use.
Why it matters
The usual planning error: the pilot with fifty requests a day costs nothing and is therefore not budgeted. At fifty thousand requests the math looks different, and by then the architecture decision has already been made. Whoever works out a figure per transaction early and multiplies it by the expected volume recognizes in time whether a smaller model, caching, or pre-filtering is needed. This calculation takes an hour and decides whether the pilot can become operations.
Commonly misunderstood
Planning usually looks at training, because the number is impressive. For your own budget, operation is almost always the decisive item.