Google’s DiffusionGemma Boosts AI Text Generation on NVIDIA GPUs

Terrill Dicki
Jun 10, 2026 16:39

DiffusionGemma introduces parallel text generation, delivering up to 1,000 tokens/second on NVIDIA GPUs, reshaping AI efficiency for developers.

Google DeepMind’s latest open model, DiffusionGemma 26B A4B, is making waves in AI development circles with its innovative approach to text generation. Launched on June 10, 2026, the model is optimized for NVIDIA platforms, delivering unprecedented speeds of up to 1,000 tokens per second on a single NVIDIA H100 Tensor Core GPU. This leap in efficiency holds significant implications for real-time AI applications like chat assistants, autonomous agents, and other high-throughput workflows.

Unlike traditional autoregressive models that generate text token by token, DiffusionGemma uses a diffusion-based approach, producing multiple tokens in parallel. This method, inspired by diffusion models in image generation, allows the model to deliver faster and more fluid user experiences while significantly reducing serving costs. Developers can utilize the model for both text and image modalities, with support for a context length of up to 256,000 tokens.

Performance Across NVIDIA Platforms

DiffusionGemma’s optimization for NVIDIA GPUs ensures flexibility across a range of hardware setups. On the NVIDIA DGX Spark, a personal AI supercomputer, the model achieves up to 150 tokens per second, making it ideal for localized AI research and prototyping. Meanwhile, the deskside NVIDIA DGX Station offers up to 20 PFLOPS of compute power, supporting models with up to 1 trillion parameters for larger enterprise workloads.

For developers working in desktop environments, NVIDIA RTX and RTX PRO provide optimized performance for local inference, making them accessible options for creators and professionals looking to integrate AI workflows directly into their workstations.

Technical Highlights

Built on the Gemma 4 26B A4B architecture, DiffusionGemma features 25.2 billion total parameters, with 3.8 billion active during inference to balance speed and memory efficiency. The model supports BF16 and NVFP4 precision formats, catering to both high-performance and resource-constrained environments. Developers can access the model through platforms like Hugging Face, where BF16 and NVFP4 checkpoints are available for prototyping and deployment.

Enterprise Integration With NVIDIA NIM

For companies ready to scale, NVIDIA’s NIM (Neural Inference Microservices) simplifies the deployment process. NIM packages the model as a containerized microservice, enabling integration into cloud, on-premises, or hybrid environments. It also exposes an OpenAI-compatible API, allowing developers to seamlessly send inference requests without complex setup.

Enterprise teams can begin testing by downloading the container from NVIDIA’s platform, setting up the NIM server, and making inference requests via a standard API workflow. This ease of integration positions DiffusionGemma as a powerful tool for businesses seeking to enhance AI-driven customer interactions, automate workflows, or deploy agentic systems.

Strategic Implications

DiffusionGemma’s release reflects a broader industry shift toward non-autoregressive and diffusion-based AI models, a growing area of research that has been explored in academic frameworks like diffusion language modeling. By accelerating text generation and reducing latency, the model directly addresses key constraints in real-time AI applications, making it a potential game-changer for developers and enterprises alike.

For developers, the combination of NVIDIA’s hardware acceleration and DiffusionGemma’s parallel decoding offers a compelling solution to the twin challenges of responsiveness and cost efficiency. With free prototyping through NVIDIA’s developer program and access to GPU-accelerated endpoints, adoption hurdles are minimal.

As Google and NVIDIA continue to push the boundaries of AI innovation, tools like DiffusionGemma are likely to set new benchmarks for speed and scalability, reshaping how businesses and developers approach AI-driven applications.

Image source: Shutterstock