Usage & Prompting
Retrieval-Augmented Generation (RAG)
Instead of letting the model answer from memory, material matching the question is searched for and passed along: search first, then answer. That turns the model from a knowledge store into a drafter, and responsibility for the facts moves to where it belongs: into the documents. The gain is threefold and decisive in operation. The answer is current because the document is current. It is verifiable because the passage can be supplied. And it is deletable, because you can remove a document but not knowledge that has been trained in. The price is that a model project becomes a document project: quality depends on how text sections are cut, on the quality of the search, and on upkeep of the collection, not on the model.
An example
The query “How long is the warranty on series 7?” runs in three steps. First: the stored documents are searched for matching passages. Second: the three best hits are placed in front of the question. Third: the model answers and names the passage. If the warranty changes tomorrow, you swap the document, and the next answer is correct without anything being trained.
A closer look
The process has three steps: translate the question into a vector, search for matching text passages, give both together to the model. The advantage is currency without retraining: the knowledge base can be swapped without touching the model.
Why it matters
For most enterprise cases RAG is the right way to build, because it delivers the three things fine-tuning does not: currency, verifiability, and deletability. But it shifts the project entirely into data management. Quality depends on how documents are cut up, on search quality, and on upkeep of the collection, not on the model. Hence the sober advice: whoever plans RAG is planning a document project with a language model on top. Whoever plans it as a model project fails at upkeep.
Commonly misunderstood
Quality is decided in the search, not in the model. If the right passage isn't found, a very good model answers very convincingly beside the point.