Diffusion language models produce text by progressively refining a masked sequence, whereas GPT models generate one token after another. This technique, already known from image generators, gained attention in the community in 2025 with LLaDA, Mercury and Gemini Diffusion, and then in 2026 with DiffusionGemma from Google, notably for its inference speed.
Two ways to generate text
GPT models are autoregressive: they estimate the next token from the tokens already produced. During training, multiple positions can be processed in parallel, but at inference, each token must wait for the previous one, creating a serial dependency problem. A token here refers to a unit of text, often a word fragment, that the model manipulates.
A masked diffusion model, on the other hand, starts from a sequence entirely filled with , i.e., unknown positions, and gradually reveals the words. It can choose several at each step, not necessarily following a left-to-right order. For the phrase “The cat sat down”, an autoregressive model adds one word per turn, while the diffusion model fills multiple positions at once and refines the sequence until the complete phrase is obtained. Both approaches use transformer architectures, but their objectives and inference procedures differ: the former extend a sequence, the latter refine it.
How diffusion models work
For images, diffusion relies on two processes. The forward process degrades clean data by progressively adding noise, while the reverse process trains a neural network to undo this degradation, typically using a mean squared error loss. During sampling, this correction operates over many steps.
Text poses a difficulty, because tokens are discrete identifiers, not continuous values like pixels. Degradation is therefore done by replacing tokens with , each token being masked with a probability equal to a corruption level between 0 and 1. At inference, a sampler initializes the masked positions, predicts a distribution over the vocabulary for each unresolved position, and then decides which predictions to keep. This choice usually relies on a confidence measure. The process repeats until the entire sequence is filled. Some samplers can also re-mask already validated tokens, an operation called revision, which is not a property common to all.
These models also use bidirectional attention within the sequence being denoised, whereas GPT only sees previous positions. A masked word can thus rely on the context preceding and following it, which is useful for editing and structured generation. According to the author, diffusion can also win in real-world latency when GPUs efficiently process positions in parallel, but quality remains decisive: autoregressive models remain more performant on certain reasoning and code benchmarks. The results of DiffusionGemma 26B compared to Gemma 26B are in the model card, whose reference article on denoising probabilistic diffusion models is the origin of this approach.
The remaining open challenges
Inter-token dependencies are the first obstacle. If two positions are sampled in parallel, each may seem plausible in isolation while being inconsistent with the other, which may require more denoising steps or coupled sampling. The sampling budget poses another question: reducing the number of steps improves latency but risks degrading quality, while increasing passes can erase the speed advantage.
Variable-length responses are also not natural for a fixed-size masked grid, although block-based or more flexible approaches seek to address this. Finally, the key-value (KV) cache, which speeds up generation by storing already computed results, becomes harder to exploit, because bidirectional refinement can modify the representations of an entire block. Block decoding, which freezes completed blocks, helps circumvent this problem.
