Training & Alignment
Constitutional AI
Instead of learning every desired behavior from individual ratings, the model is given principles — a constitution — and critiques and improves its own answers against them. This makes alignment readable: you can discuss the principles instead of a thousand individual judgments. Anthropic developed the approach for Claude; it does not replace human feedback but makes it scalable.
An example
Instead of having humans rate every answer, the model is given a written set of principles — for instance: no instructions for causing harm, no disparagement, name uncertainty when uncertain — and uses them to critique its own answers before any training happens on them. The advantage is readability: the rules exist as text and can be discussed, instead of being implicit in millions of individual judgments.
A closer look
Instead of collecting human feedback for every case, the model is given a set of written principles and critiques and revises its own answers against them. This scales better and makes the underlying rules legible.
Why it matters
For companies, the procedure is less interesting than the principle: explicit, written rules can be checked, implicit preferences cannot. This can be carried over to your own use without training anything yourself, by actually writing down the house rules for AI use instead of leaving them to the teams' gut feeling. Whoever cannot formulate what a system must not do cannot check it either. And the second layer remains: the principles of a purchased model were written by someone else, by their standards.
Commonly misunderstood
Written principles are not neutral. Whoever formulates them makes value decisions. The advantage of the procedure is that these decisions are at least readable.