Model Collapse

Home - Archive - About - Publications - Contact

Model Collapse

Posted 5/29/2026

I’ve heard many people talk about model collapse recently, especially regarding Large Language Models. This is a phenomenon where training a machine learning model on the outputs of itself or other machine learning models can often degrade performance and magnify errors. In this post I want to talk a little about the phenomenon and its implications for the future of LLMs.

How does Model Collapse Work?

Let’s demonstrate the problem with one of the simplest machine learning models: linear regression. Here we have a “true function” (a cosine wave) which can only be observed noisily. This means given an X value we can observe the Y value of the cosine wave plus a small error term. We’ll gather thirty samples from random X points and try to fit a degree 3 polynomial, so fitting y = ax + bx^2 + cx^3 + d to the sampled points.

10<br>11<br>12<br>13<br>14<br>15<br>16<br>17<br>18<br>19<br>20<br>21<br>22<br>23<br>24<br>25<br>26<br>27<br>28<br>29<br>30<br>31<br>32<br>33<br>34<br>35<br>36<br>#!/usr/bin/env python3<br>import matplotlib.pyplot as plt<br>import numpy as np<br>from sklearn.linear_model import LinearRegression<br>from sklearn.preprocessing import PolynomialFeatures

# Wrap the function in a class that behaves like a sklearn predictor<br>class TrueFunc:<br>def true_fun(self, X):<br>return np.cos(1.5 * np.pi * X)<br>def predict(self, X):<br>X_relevant = X[:,1] # Use X, not X^0 or X^2<br>return self.true_fun(X_relevant)

np.random.seed(0)<br>n_samples = 30<br>degree = 3<br>tf = TrueFunc()

X = np.sort(np.random.rand(n_samples))<br>X_rotated = X.reshape(-1, 1) # Make this an Nx1 array instead of 1xN<br>expanded_features = PolynomialFeatures(degree).fit_transform(X_rotated)<br># Sample with noise<br>y = tf.predict(expanded_features) + np.random.randn(n_samples) * 0.1<br># Fit a curve to the samples<br>reg = LinearRegression().fit(expanded_features, y)

# Plot the resulting curves across [0..1] and the sampled points<br>X_test = np.linspace(0, 1, 100)<br>X_test_rotated = PolynomialFeatures(degree).fit_transform(X_test.reshape(-1,1))<br>plt.figure(figsize=(6,4), dpi=120)<br>plt.plot(X_test, tf.true_fun(X_test), label="True Function")<br>plt.plot(X_test, reg.predict(X_test_rotated), label="Model")<br>plt.scatter(X, y, edgecolor="b", s=20, label="Samples")<br>plt.legend(loc="best")<br>plt.show()

So far so good, we’ve fit a line that approximates the original function and it does pretty well! But what happens if we noisily sample from our fit line, and try to train a degree three linear regressor on that? And then train another model off of that one?

Let’s go ahead and fit five hundred linear regression curves, each to thirty points sampled noisily from its predecessor:

10<br>11<br>12<br>13<br>14<br>15<br>16<br>17<br>18<br>19<br>20<br>21<br>22<br>23<br>24<br>25<br>26<br>27<br>28<br>modelCount = 500<br>models = [tf]<br>orig_X = []<br>orig_Y = []<br>for i in range(modelCount):<br>model = models[-1]<br>X = np.sort(np.random.rand(n_samples))<br>X_rotated = X.reshape(-1, 1) # Make this an Nx1 array instead of 1xN<br>expanded_features = PolynomialFeatures(degree).fit_transform(X_rotated)<br># Sample with noise<br>y = model.predict(expanded_features) + np.random.randn(n_samples) * 0.1<br>if( i == 0 ):<br>orig_X = X<br>orig_Y = y<br>reg = LinearRegression().fit(expanded_features, y)<br>models.append(reg)

plt.figure(figsize=(6,4), dpi=120)<br>X_test = np.linspace(0, 1, 100)<br>X_test_rotated = PolynomialFeatures(degree).fit_transform(X_test.reshape(-1,1))<br>for i in [0, 100, 200, 500]:<br>model = models[i]<br>if( i == 0 ):<br>plt.plot(X_test, model.true_fun(X_test), label="True Function")<br>else:<br>plt.plot(X_test, model.predict(X_test_rotated), label="Model %d" % i)<br>plt.legend(loc="best")<br>plt.show()

As we can see, the error terms accumulate, so we drift further and further from the original function. At five hundred curves, the shape of the original function is almost entirely lost. You can’t improve linear regression by generating synthetic data points from your fit curve and then re-fitting to those.

All machine learning models, whether regressors, classifiers, or generative models, suffer from a similar challenge. The models have imperfect outputs, they predict a little too high or too low, they occasionally misclassify a point, they hallucinate1 some outputs. Therefore, training on those mistakes will magnify errors and degrade performance in nearly every domain.

There are some rare exceptions where training on synthetic data is warranted, such as imputing missing values, which I discussed in another machine learning post, or training a small and simple model to mimic the behavior of a larger and more complex model. However, even in these scenarios data scientists know that synthetic data degrades model performance, and they must carefully ensure they accomplish their goals without damaging the model to the point that it loses utility.

How Does this Apply to LLMs?

In the case of large language models our objective is to mimic human writing by ingesting an enormous volume of text and fitting to patterns in word co-occurrence. Therefore, we want to avoid training on text written by other...