diffusiongemma-26B-A4B-it-NVFP4
DiffusionGemma-26B-A4B-IT-NVFP4 is a quantized, multimodal LLM from NVIDIA based on Google's Gemma 4 architecture. It processes text, image, and video inputs to generate text output, with 25.2B total parameters but only 3.8B active due to its Mixture-of-Experts design. The NVFP4 quantization reduces model size and GPU memory requirements for faster inference. It supports 256K token context, function calling, and 35+ languages. Licensed under Apache 2.0 with Gemma terms overlay. Optimized for NVIDIA GPUs (Hopper, Blackwell) and served via vLLM.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | nvidia |
| Parameters | 14.4B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 1.7M |
| Likes | 95 |
| Last updated | 2026-07-04 |
| Source | nvidia/diffusiongemma-26B-A4B-it-NVFP4 |
What diffusiongemma-26B-A4B-it-NVFP4 is
Transformer-based MoE architecture with bidirectional encoder-decoder attention and discrete diffusion sampling. Generates text in parallel 256-token blocks at >1,100 tokens/sec on H100 FP8. Input: text, RGB images (variable resolution via 70–1120 token budgets), video (up to 60 sec @ 1 fps). Output: text with native function calling and structured JSON. NVFP4 quantization reduces FP16→4-bit activations/weights. Inference via vLLM with TRITON_ATTN backend. Evaluation on GPQA Diamond, AIME 2025, GSM8K, HumanEval, MMLU shows <1% accuracy loss vs. BF16 baseline. Training data cutoff: January 2025; includes CSAM filtering and PII removal.
Run diffusiongemma-26B-A4B-it-NVFP4 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="nvidia/diffusiongemma-26B-A4B-it-NVFP4")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: 4-bit quantization reduces memory significantly from FP16. Approximate VRAM: 14–26 GB for inference on single H100/B100 GPU (quantized weights ~13 GB + activation buffers). Inference tested on NVIDIA Blackwell B100; compatible with Hopper H100. Requires CUDA-capable NVIDIA GPU; Linux preferred. Multi-GPU setups supported via vLLM for higher concurrency. Exact VRAM/throughput depends on batch size, context length, and image/video input token budgets.
Not stated in card. Post-training quantization (NVFP4) completed by NVIDIA ModelOpt. No mention of LoRA, QLoRA, or fine-tuning feasibility on quantized weights. Requires review of vLLM and ModelOpt documentation or direct testing. Base model (Gemma 4) fine-tuning methodology unknown; assume standard transformer SFT/DPO may apply, but on quantized weights feasibility is unclear.
When to avoid it — and what to weigh
- CPU-only deployments — Model is explicitly optimized for NVIDIA GPUs (Hopper, Blackwell). CPU inference is not a supported deployment path; expect severe latency penalties.
- Real-time safety-critical applications without additional testing — Card notes model may amplify toxic language and societal biases from training data. Requires iterative testing and validation per V-model methodology before production deployment in safety-sensitive domains.
- Low-latency, single-digit token generation — Diffusion sampling generates 256-token blocks in parallel, not ideal for latency-sensitive streaming use cases requiring immediate output of first tokens.
- Proprietary/closed-source model modifications without Apache 2.0 compliance — Apache 2.0 license requires attribution and disclosure; Gemma Terms of Use overlay adds restrictions. Closed-source derivative products require legal review.
License & commercial use
Apache License 2.0 (SPDX-License-Identifier: Apache-2.0) with two additional terms: (1) Google Gemma Terms of Use, (2) Google Gemma Prohibited Use Policy. Apache 2.0 is OSI-approved and permissive, requiring attribution and liability disclaimer. Gemma overlay restricts use to non-commercial research and educational purposes unless explicit commercial agreement signed with Google. NVIDIA model card states 'ready for commercial and non-commercial use', but this conflicts with Gemma TOU if Google's commercial license is not independently secured.
Requires review. The model card claims 'ready for commercial and non-commercial use', but is governed by Apache 2.0 + Google Gemma Terms of Use + Gemma Prohibited Use Policy. Apache 2.0 alone permits commercial use, but Gemma TOU restricts commercial deployment unless you have a separate commercial agreement with Google. NVIDIA disclaimer notes this is a third-party (Google) model quantized by NVIDIA. Before deploying commercially, verify: (1) Direct Apache 2.0 commercial eligibility is asserted only by NVIDIA quantization layer, not by Google's Gemma base model. (2) Google's commercial Gemma license status. Consult legal before production commercial use.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Needs review |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | High |
Card does not claim the model 'is secure'. Security notes: (1) Training data includes CSAM filtering and PII removal—mitigates some data-sourced risks but no guarantee of zero leakage. (2) Model trained on internet-scale data containing toxic language and societal biases; may amplify these when prompted. (3) No mention of prompt injection, jailbreak testing, or adversarial robustness. (4) Native function calling feature expands attack surface if downstream tools are called without sandboxing. (5) Gemma Prohibited Use Policy restricts illegal, deceptive, and harmful uses; enforcement is policy-dependent, not technical. Conduct threat modeling and testing before production deployment, especially for safety-sensitive applications.
Alternatives to consider
Llama 3.1 70B (Meta)
Open-weights, OSI-licensed (Llama Community License), larger context (128K), broader tool ecosystem. Quantized variants available. Slightly heavier footprint but established production deployments and stronger documentation.
Mixtral 8x22B (Mistral)
Similar MoE efficiency (8 experts, ~13B active). Apache 2.0 license without Google overlay. Smaller total parameters (141B vs. 25.2B) but finer control. Strong vLLM support.
Claude 3.5 Sonnet (Anthropic, API-only)
Proprietary, closed-source, hosted only. No quantization needed. Strong multimodal (image/video) support and reasoning. Trade-off: no self-hosting, vendor lock-in, higher per-token cost but production-ready safety.
Ship diffusiongemma-26B-A4B-it-NVFP4 with senior software developers
This model demands NVIDIA GPU infrastructure (Hopper H100 or Blackwell B100) and vLLM orchestration. Before production, verify: Gemma commercial licensing with Google, benchmark VRAM/latency on your hardware, and test for toxic output mitigation in your use case. Contact our AI engineering team to validate deployment architecture and compliance.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
diffusiongemma-26B-A4B-it-NVFP4 FAQ
Can I use this model for commercial applications?
What is the estimated VRAM requirement for inference?
How does inference speed compare to the full-precision baseline?
Is fine-tuning supported on the NVFP4 quantized weights?
Software development & web development with DEV.co
Need help beyond evaluating diffusiongemma-26B-A4B-it-NVFP4? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to Deploy DiffusionGemma? Assess Your Infrastructure.
This model demands NVIDIA GPU infrastructure (Hopper H100 or Blackwell B100) and vLLM orchestration. Before production, verify: Gemma commercial licensing with Google, benchmark VRAM/latency on your hardware, and test for toxic output mitigation in your use case. Contact our AI engineering team to validate deployment architecture and compliance.