Synthetic data

As high-quality human text runs short, labs increasingly generate training data with models: worked math solutions, code with tests, rewritten documents, simulated conversations. Done carefully — with filtering and verification — it's a powerful way to teach specific skills.

Done carelessly, models trained on their own outputs can degrade ("model collapse"), so verification is the key ingredient.