Foundations & Models
Fine-Tuning
A finished model gets tutoring with your own examples: the company's tone of voice, technical vocabulary, a particular task format. That is orders of magnitude cheaper than training your own and often more effective than long prompts. But it only pays off once simpler means are exhausted — many problems that look like fine-tuning are really context problems.
An example
An insurer wants replies written in its own house tone: polite, brief, no subjunctive. Fine-tuning is the right tool for that — you show the model a few hundred sample replies, and it adopts the style. The same insurer also wants the model to know the current rates. Fine-tuning is the wrong tool for that: rates change quarterly, and retraining every time is expensive, slow and imprecise. They belong in the context, not in the model.
A closer look
In post-training, a finished model is adapted further with a few hundred to a few thousand examples. Economical methods such as LoRA change only a small additional matrix instead of all weights, which considerably reduces cost and memory requirements.
Why it matters
The rule of thumb that prevents most misinvestments: fine-tuning changes *how* a model answers. Retrieval changes *what* it answers about. Behavior, format and tone can be trained in; facts should be looked up. Anyone who trains knowledge into the model gets a model that goes out of date without noticing, whose sources cannot be traced and whose updating costs money every time. For every fine-tuning project, therefore, first check whether the task is really a retrieval task.
Commonly misunderstood
Fine-tuning teaches form and style, not new factual knowledge. Anyone who needs current or company-specific content is better served by retrieval.