Gemma-4-26B-A4B-NVFP4
Gemma-4-26B-A4B-NVFP4 is NVIDIA's quantized version of Google's Gemma 4 multimodal LLM. It uses a mixture-of-experts architecture with 3.8B active parameters out of 25.2B total, optimized for inference on NVIDIA GPUs via vLLM. The model supports text, images, and video inputs with a 256K token context window and handles reasoning, coding, summarization, and conversational tasks. It is released under Apache 2.0 license and marked as ready for commercial and non-commercial use. The NVFP4 quantization reduces model size while maintaining near-baseline performance on reasoning and coding benchmarks.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | nvidia |
| Parameters | 14.4B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 2.2M |
| Likes | 112 |
| Last updated | 2026-05-11 |
| Source | nvidia/Gemma-4-26B-A4B-NVFP4 |
What Gemma-4-26B-A4B-NVFP4 is
Gemma 4 26B A4B is a sparse mixture-of-experts transformer with 30 layers, 262K vocabulary, and 8 active experts out of 128 total. It employs hybrid attention (local sliding-window at 1024 tokens + global attention), proportional RoPE for long-context support, and unified KV in global layers. Vision encoder adds ~550M parameters for image/video processing. Quantized to NVFP4 (4-bit custom NVIDIA format) using Model Optimizer v0.43.0. Calibrated on cnn_dailymail and Nemotron datasets. Requires vLLM with tensor parallelism=1 (expert parallelism and Flashinfer support pending upstream fixes). Tested on NVIDIA B200 hardware. Training data cutoff January 2025, covering 140+ languages.
Run Gemma-4-26B-A4B-NVFP4 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="nvidia/Gemma-4-26B-A4B-NVFP4")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~14–16 GB VRAM for NVFP4 quantized inference (4-bit weights + activations + KV cache for ~256K context). Full-precision baseline (bf16) would require ~45–50 GB. Verified on NVIDIA B200; should support L40S, H100, L100, and newer Blackwell GPUs. vLLM serving typically batches efficiently but does not yet support multi-GPU expert parallelism. Single-GPU deployments recommended until vLLM EP and Flashinfer support is released. Linux OS required.
Not clearly stated in model card. Base Gemma 4 26B A4B IT is a sparse MoE model; fine-tuning a quantized NVFP4 model is uncommon in practice. Recommend: (1) fine-tune the unquantized baseline (google/gemma-4-26B-A4B-it) with LoRA/QLoRA on NVIDIA hardware, then quantize with ModelOptimizer afterward; (2) or apply LoRA adapters in inference without re-quantizing. Requires review of NVIDIA ModelOptimizer and vLLM LoRA support compatibility. Sparse MoE fine-tuning may require specialized tooling (e.g., handling expert routing updates).
When to avoid it — and what to weigh
- Deployment outside NVIDIA GPU ecosystems — Model is explicitly designed and optimized for NVIDIA hardware (CUDA, TensorRT, vLLM). CPU or non-NVIDIA accelerator deployment is unsupported and will see significant performance degradation. Consider CPU-optimized models (e.g., llama.cpp compatible) if heterogeneous infrastructure is required.
- Distributed inference requiring expert or tensor parallelism across multiple GPUs — Current vLLM support is limited to TP=1 with pending issues in expert parallelism and Flashinfer backend integration. Large-scale distributed serving requires waiting for upstream vLLM PRs and releases, or using non-quantized baseline with TensorRT-LLM.
- Production systems requiring deterministic or bias-controlled outputs — Model card explicitly states the base model was trained on toxic internet-crawled data and may amplify societal biases or produce toxic responses. Requires internal alignment testing, prompt engineering, or fine-tuning before use in sensitive domains (healthcare, legal, finance).
- Use cases requiring video as native input (not frame-by-frame sequences) — Video support is noted on E2B/E4B model variants, not this 26B A4B model. Video must be decomposed into frames (up to 60 seconds at 1 FPS). If native video understanding is critical, verify suitability or consider alternative architectures.
License & commercial use
Apache License 2.0 (OSI-approved permissive license). Covers both the quantized NVIDIA model artifact and the underlying Google Gemma 4 26B IT base model. Allows modification, distribution, and private use with minimal restrictions (attribution required). See https://ai.google.dev/gemma/apache_2 for full terms.
Apache 2.0 is a permissive OSI license that explicitly permits commercial use. Model card states 'This model is ready for commercial/non-commercial use.' No gating or commercial license required. However, developers must ensure: (1) compliance with any third-party data rights in training data (model trained on web-crawled content); (2) internal review for bias and safety in regulated domains; (3) proper input validation for images/video (personal data, IP, CSAM filtering per model card note). Recommend legal review if deploying in healthcare, financial services, or other high-risk sectors.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | High |
Model card does not claim or audit security posture. Considerations: (1) Base model trained on web-crawled data with filtering for CSAM and sensitive data, but comprehensive security audit not detailed; (2) NVFP4 quantization does not inherently introduce or mitigate vulnerabilities but reduces model size (smaller attack surface for exfiltration); (3) vLLM inference server requires standard hardening (auth, TLS, input validation); (4) Model supports function-calling and tool-use (--enable-auto-tool-choice flag) — ensure tool definitions are validated and sandboxed; (5) Multimodal inputs (images, video) require validation to prevent prompt injection or adversarial perturbations. No formal threat model or pen-test data provided. Recommend standard LLM security practices: input filtering, rate limiting, output filtering, audit logging.
Alternatives to consider
google/gemma-4-26B-A4B-it (unquantized baseline)
Full-precision original model; better for fine-tuning, research, or if VRAM availability allows. No quantization artifacts; wider tool/library support. Trade-off: ~3x larger and slower on consumer GPUs.
meta-llama/Llama-3.1-405B-Instruct (quantized variants)
Larger dense model (405B params) with strong reasoning/coding performance and broader industry adoption. Suitable if deployment budget allows multi-GPU clusters or if you need stronger inference support across vLLM, TGI, and llama.cpp. Trade-off: no official MoE sparsity; higher latency per token on single GPU.
mistralai/Mistral-Large-v2 (smaller quantized alternatives)
Smaller dense alternatives (e.g., 34B) with broader quantization framework support (GPTQ, AWQ, bfloat16 via Ollama). Better portability if NVIDIA-specific optimization is not a hard requirement. Trade-off: lower reasoning/coding performance than Gemma 4.
Ship Gemma-4-26B-A4B-NVFP4 with senior software developers
Verify NVIDIA GPU availability, set up vLLM with the provided command, and test on your target hardware before scaling. Consult our security and fine-tuning guides for regulated use cases.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Gemma-4-26B-A4B-NVFP4 FAQ
Can I use this model commercially without a license agreement?
What GPU do I need to run this model?
Can I fine-tune this quantized model?
What are the known limitations of the vLLM implementation?
Work with a software development agency
Need help beyond evaluating Gemma-4-26B-A4B-NVFP4? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to Deploy Gemma 4 in Production?
Verify NVIDIA GPU availability, set up vLLM with the provided command, and test on your target hardware before scaling. Consult our security and fine-tuning guides for regulated use cases.