Usage & Prompting
Temperature
A dial for how much risk to take when rolling the dice for the next token: low means predictable and repeatable, high means varied to far-fetched. For contracts and extraction you want it near zero, for brainstorming it can be higher. Anyone who needs reproducible results has to know this dial.
An example
At temperature 0, the model always picks the most probable next token; the same input yields largely the same output. At 1.0 it rolls the dice more: good for suggested phrasings, bad for extracting an invoice number. A team running a data extraction on the default setting was puzzled by sporadically differing results on identical documents. The cause was in one line of configuration that nobody had touched.
A closer look
The value scales the probability distribution before selection. At 0 the most probable token is always taken; at higher values rarer ones are, too. For extraction and classification it belongs low, for drafts it can be high.
Why it matters
The setting is trivial and is regularly overlooked anyway, because it has a default value meant for conversation, not for processing. Rule of thumb: anything that extracts, classifies, or checks a result runs at 0. Anything that should compose text can go higher. More important still: even at 0 the output is not guaranteed to be identical, because computation order varies on the hardware. Anyone who needs reproducibility logs the output instead of relying on its repeatability.
Commonly misunderstood
Even at temperature 0, responses are not guaranteed to be identical. Parallelization and changes on the provider's side cause deviations that should not be relied upon.