Foundations & Models
Large Language Model (LLM)
At its core, an LLM does one thing: it predicts the most likely next word fragment, practiced billions of times on text from the internet, from books, from code. A surprising amount emerges from this simple task — translating, summarizing, arguing. What does not emerge is a built-in notion of true and false. The model knows plausibility, not truth.
An example
An LLM does not answer a question. It continues a text. After “The capital of France is” comes “Paris,” because that continuation was overwhelmingly common in the training texts. After “The case law of the Federal Court of Justice on file no. VIII ZR 402/19 states” there also comes a fluent continuation — even if that file number never existed. The model does not distinguish the two cases, because in both it performs the same operation: generating the most probable continuation.
A closer look
An LLM estimates the probability of the next token at each position and samples from it. The capabilities that become visible in the process — translating, summarizing, programming — are not trained individually but are by-products of this one task on very large amounts of text.
Why it matters
Almost everything one needs to know in operation follows from this one property. An LLM is strong where the form of language is the task: rephrasing, summarizing, translating, structuring, producing drafts. It is unreliable where factual accuracy is the task and the facts are not in the context. The consequence is not a warning but a design principle: anyone who wants dependable answers gives the model the source instead of relying on its memory. That is exactly what grounding and retrieval-augmented generation do.
Commonly misunderstood
A language model has no database it looks things up in, and no separation between what it knows and what it plausibly fills in. That is why correct and incorrect answers sound the same.
Where this term does its work in the magazine.
1 article uses the term or takes up its principle.