Towards demystifying the creativity of diffusion models

Skip to main content

Google

Research

Search

Towards demystifying the creativity of diffusion models

July 15, 2026<br>Zhengdao Chen, Research Scientist, Google Research

We show that a diffusion model’s creativity (its ability to generate novel data, rather than just memorize its training set) is a mathematical consequence of neural networks learning a "smoothed" version of the score function, driving the model to interpolate between training data points along the hidden data manifold.

Quick links

Paper

Code

Share

Copy link

Diffusion models are currently one of the most powerful types of tools for generative tasks that require complex and local structures, such as image generation and molecular discovery. They’ve shown an exciting capability to generalize beyond their training data and, in this sense, exhibit “creativity”. For instance, after being trained with datasets of actual images, they can transform random noise samples into novel, high-quality images.<br>While this creative capability is impressive, it raises an intriguing question: where does it come from? Understanding the answer to this question is an important step towards demystifying the “black-box” nature of diffusion-based generative AI.<br>To that end, in "On the Interpolation Effect of Score Smoothing in Diffusion Models", presented at ICLR 2026, we build upon the prior research literature and dive into the mathematics of diffusion models to answer this question. We show that a model’s creativity isn’t a random fluke. Instead, it is a consequence of how neural network training naturally "smooths" the transformation from noise back to the data during the generation process.

Understanding denoising

Training a diffusion model begins with taking a set of real training data samples — like cat photos — and intentionally corrupting them with noise until they become completely unrecognizable. The model is then trained to reverse this corruption step-by-step so that it can reconstruct a realistic-looking image from pure noise, a process called denoising.<br>If the model learns to perform this denoising process perfectly based only on its training samples, it should produce carbon copies of them during deployment time as well (a behavior known as memorization). In this scenario, the model acts as a retrieval tool rather than as a creative engine capable of generating novel outputs.<br>In practice, however, diffusion models usually do more than just memorize; they generalize to generate new data samples.<br>To understand how diffusion models actually denoise data, imagine random noise as a cloud of gas particles scattered across a room, where a “force field” pulls each particle in a specific direction until they form a meaningful shape. In a diffusion model, the moving particles are individual data points undergoing denoising. The “force field” is the score function (SF), which is learned from the training data and dictates where the particles should flow at any given moment.<br>If the model relies on a score function learned perfectly from the training data, the force field will drive the particles into positions that exactly replicate the training data points (i.e., memorization).

The score function drives the denoising process which turns pure noise into meaningful data (e.g., images).

Diffusion model creativity: The 1-dimension example

We discovered that the creativity of diffusion models actually originates from the approximate nature of how neural networks typically learn: imperfect training due to regularization naturally leads to a slight blurring of the learned score function in a process called “score smoothing” (Scarvelis et al., TMLR 2025). This, in turn, causes the denoising process to generate data that interpolates (in other words, fall in the space between) the training points, thus creating new and plausible data samples.<br>Imagine a one-dimensional world with only two training data points: +1 and -1. At late stages of the denoising process, the “perfect” score function looks like the curvy gray line in the figure below, which has a steep change of sign halfway between the two points, meaning a rapid switch in the direction of pull near 0. In other words, the whole space is divided almost sharply into two, with particles on the left pulled towards -1 and particles on the right pulled towards +1. In the end, every particle converges to one of the two training data points, and thus, memorization occurs.

The “perfect” score function drives the denoising process to collapse onto the training data, thus leading to memorization (the background color and its opaqueness indicate the direction and strength of the pull: red for rightward, blue for leftward).

In practice, however, diffusion models don’t have access to the “perfect” score function but use an approximate version learned by a neural network. Because of the regularization effect of weight decay during training, neural networks...