Compute & Cost
Inference
Training is learning, inference is working: every single answer of a finished model. While training is expensive once, inference adds up with every use, and if the product succeeds it becomes the dominant cost block. That is why operational optimization almost always revolves around inference: smaller models, shorter contexts, caching, batching.
An example
Training is setting the weights, inference is using them. Every request to a model is an inference run: the input is broken into tokens, computed through all the layers, and one token after another is produced. The model does not change in the process. It learns nothing from your request unless the vendor explicitly uses it for training. Two identical requests therefore cost twice.
A closer look
In inference, the model computes through the input once and generates the answer token by token. Nothing is stored or learned in the process. The same question in a new session meets the same starting state.
Why it matters
The distinction clears up a common misunderstanding: “the model learns our data” is false under normal use, since your inputs do not change the weights. What is true is something else and is found in the terms of use: whether the vendor may store inputs and use them for future training. That is a contract question, not a technical one, and in business customer contracts it can usually be opted out of. Ask about it explicitly, and get it in writing.
Commonly misunderstood
That is why a model has no memory across sessions. What looks like memory is a history that the application sends along every time.