Training & Alignment
Tests & Harnesses
The division of labor flips: the human defines what “right” means (test cases, acceptance criteria, thresholds), and the AI produces against it until it fits. The harness, the test rig, thus becomes the actual place of quality: whoever writes it leads. Bad criteria diligently produce the wrong thing; good ones make even mediocre models usable.
An example
The test checks a claim: for this input, this invoice number must come out. The frame around it is what makes checking practical: it holds two hundred cases ready, calls the system for each one, compares, counts, shows the deviations, and runs automatically with every change. Without the frame, it stays at three cases someone clicks through by hand when they remember to.
A closer look
A harness is the frame that runs a task automatically and checks the result: for code, test runs; for text, format checks or a second model with clear instructions. It turns an impression into a number.
Why it matters
With non-deterministic systems, the role of the test changes. A single assertion says little, because the same input may be answered differently next time. What counts is the rate across many cases and its change over time. That is why the frame matters more than the individual test, and why it is the infrastructure without which neither prompt work nor model switches nor red teaming deliver anything lasting. It should be the first thing built and is usually the last.
Commonly misunderstood
The frame determines what counts as right. That determination is a substantive decision and belongs to people, not to the automation.