Training & Alignment
Synthetic Data
When real data is scarce, expensive, or sensitive, you generate artificial data: model-generated examples, rare edge cases, anonymized twins of real records. This can fill gaps and ease privacy concerns. It carries a quiet danger: models that learn mostly from model outputs amplify those outputs' errors. The copy of the copy grows fainter.
An example
A hospital wants to train a model to detect a rare finding but has only 40 real cases. So it generates 4,000 artificial ones. The model then recognizes artificial cases superbly and real ones hardly better than before. It has learned the quirks of the generator, not those of the disease. The reverse case works well, though: replacing addresses, names, and account numbers in test data with invented ones, so developers can work with realistic structures without seeing real personal data.
A closer look
A strong model generates examples that serve to train another. This is especially useful where real data is rare, expensive, or protected, for instance for edge cases that hardly occur in operation but are costly when they do.
Why it matters
Synthetic data is excellent for supplying structure and deceptive when it is supposed to supply substance. The reason is simple: it can only contain what the generator knows, and the generator does not know exactly the rare cases for which you are generating it. The second, often overlooked point is legal: synthetic data generated from real personal data is not automatically anonymous. If individual records can be traced back, it remains personal data, and checking for that belongs in the procedure.
Commonly misunderstood
Training only on generated data over several generations narrows the distribution. The rare disappears first. An anchor in real data remains necessary.
Where this term does its work in the magazine.
1 article uses the term or takes up its principle.