Synthetic data is training data generated by machine intelligence rather than collected from the real world. Instead of paying humans to label examples or scraping the web, you use an existing model to produce new training samples — question-answer pairs, code snippets, classified documents — shaped for the downstream task.

The upside is real. Microsoft's Phi series is the proof of concept: "Textbooks Are All You Need" showed that a small model trained primarily on high-quality synthetic textbook content could match or beat models 10x its size on coding and reasoning benchmarks. The key wasn't volume — it was curation. GPT-4 generated structured educational content, and the dataset was aggressively filtered for quality. Size lost to signal.

The downside is model collapse. Train on synthetic data generated by a model trained on synthetic data, and each generation loses fidelity. The distribution narrows, rare patterns disappear, and you end up with a model that sounds fluent but has forgotten the edges of the real world. This isn't theoretical — it's been demonstrated across multiple model families.

When a vendor says their model was trained on "proprietary data," ask whether that data is synthetic. It's not inherently disqualifying — Phi proves the approach can work — but it demands scrutiny: which source model generated it, how was quality filtered, and what evals validate the result? Synthetic data without rigorous evaluation is just noise with a higher price tag.