Key facts
- Google released DiffusionGemma, a free, open-weight AI model.
- The model uses text diffusion to generate entire token blocks simultaneously.
- DiffusionGemma achieves over 1,000 tokens per second on an NVIDIA H100.
- This speed is four times faster than standard autoregressive language models.
- The model's output quality is noted to be lower than Gemma 4.
- Running the model efficiently locally requires a specific drafter module that is not yet widely available.
Google has launched DiffusionGemma, a novel open-weight AI model that generates text using a diffusion process, similar to how image generators create pictures. This approach allows the model to produce entire blocks of text simultaneously, achieving speeds of over 1,000 tokens per second on an NVIDIA H100, which is four times faster than traditional autoregressive models. The model is available for free under an Apache 2.0 license, with weights accessible on Hugging Face.
Unlike autoregressive models that generate tokens one by one, DiffusionGemma starts with random tokens and iteratively refines them in parallel. This bidirectional attention mechanism makes it particularly adept at tasks where the beginning of the output is constrained by the end, such as code infilling and structured generation. Google demonstrated its capability by fine-tuning a version to solve Sudoku puzzles, achieving an 80% success rate.
However, running DiffusionGemma efficiently presents challenges. It requires a specific "drafter" module for speculative decoding, which is not yet integrated into common local inference runtimes like mlx-lm or LM Studio. While Google's announcement claims speeds of 700+ tokens per second on an NVIDIA GeForce RTX 5090, it also notes that the output quality trails behind the standard Gemma 4 model. Furthermore, when tested on NVIDIA NIM, the model was preconfigured with an 8,192 token context window, insufficient for agentic frameworks like Hermes Agent that require a minimum of 64,000 tokens, despite the model's architectural limit of 256K tokens.
The primary target audience for DiffusionGemma is developers with high-end GPUs (NVIDIA RTX 4090 or 5090) looking to build real-time applications like inline editors and autocompleters. Researchers may also find its bidirectional generation capabilities useful for tasks involving complex dependencies, such as protein sequences or mathematical graphs. The open-weight nature and integration with tools like vLLM and Hugging Face Transformers are expected to broaden its accessibility as the toolchain catches up.