Qwen3.5-397B-A17B-NVFP4
This is NVIDIA's quantized version of Alibaba's Qwen3.5-397B-A17B, a 397-billion-parameter mixture-of-experts language model compressed to FP4 precision for efficient inference. It accepts text, images, and video inputs, supports a 262K token context window, and is optimized for NVIDIA GPUs. The model is permissively licensed (Apache 2.0), ungatED, and ready for immediate commercial and non-commercial deployment. Performance benchmarks show minimal degradation compared to the FP8 baseline across multiple reasoning and coding tasks.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | nvidia |
| Parameters | Unknown |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 480.1k |
| Likes | 102 |
| Last updated | 2026-06-30 |
| Source | nvidia/Qwen3.5-397B-A17B-NVFP4 |
What Qwen3.5-397B-A17B-NVFP4 is
Qwen3.5-397B-A17B-NVFP4 is a post-training quantized MoE transformer with 397B total parameters but only 17B activated per token, reducing compute and memory footprint. Quantization was performed using NVIDIA Model Optimizer v0.42.0, applying FP4 precision to weights and activations of linear operators within transformer MoE blocks. The model supports multimodal input (text, RGB images, MP4/WebM video) and generates text output. Calibrated on CNN DailyMail and Nemotron-Post-Training-Dataset-v2. Tested on NVIDIA B200 hardware. Inference engines: SGLang (primary), vLLM. Context length: 262K tokens. Training dataset details (modality, collection, labeling) are undisclosed.
Run Qwen3.5-397B-A17B-NVFP4 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="nvidia/Qwen3.5-397B-A17B-NVFP4")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
**ESTIMATE — Verify with your infrastructure team.** With FP4 quantization and MoE (17B active per token), approximate GPU memory per instance: 80–150 GB for multi-GPU tensor parallelism (4–8 GPUs). Tested on NVIDIA B200; compatible with Blackwell. FP4 weights only; assume ~50–100 GB for model weights + activations + KV cache at typical batch sizes. Exact VRAM depends on batch size, max context length (262K), and tensor parallel configuration. Production serving typically requires 4–8 H100/B200 GPUs.
Not explicitly documented. The model is post-training quantized to FP4; full-parameter fine-tuning would require dequantization, incurring high memory overhead. LoRA/QLoRA fine-tuning feasibility is unknown and vendor documentation does not address it. Contact NVIDIA or test on a small pilot. Standard practice: fine-tune the base Qwen3.5-397B-A17B (unquantized) then quantize, rather than fine-tuning the FP4 version directly.
When to avoid it — and what to weigh
- CPU-Only or Non-NVIDIA Deployment — Model is optimized for NVIDIA GPUs (tested on B200, Blackwell architecture). CPU inference or non-NVIDIA accelerators are not documented or supported. Deployment outside GPU clusters incurs severe performance penalties.
- Real-Time, Single-Token Latency-Critical Applications — MoE models with sparse activation and 397B parameters require tensor parallelism across multiple GPUs. Single-GPU inference or strict <50ms latency SLAs are not feasible. Throughput optimized, not latency optimized.
- Datasets with Heavy Non-English or Specialized Domain Bias — Training data is undisclosed. CNN DailyMail calibration is English-language news. If your use case requires strong performance on low-resource languages, specialized vertical domains (e.g., biomedical NER), or non-text modalities, in-domain evaluation is required.
- Applications Requiring Guaranteed Absence of Toxic or Biased Output — Model card acknowledges the base model was trained on internet data containing toxic language and societal biases. No filtering or fine-tuning for safety is documented. Requires guardrails and human review for sensitive use cases.
License & commercial use
Apache License 2.0. This is a permissive, OSI-approved license allowing commercial and non-commercial use, modification, and distribution, provided copyright and license text are retained. Model card explicitly states: 'This model is ready for commercial/non-commercial use.'
Apache 2.0 is permissive and explicitly permits commercial use. Model card states 'ready for commercial/non-commercial use.' However: (1) This is NVIDIA's quantized derivative of Alibaba's Qwen3.5-397B-A17B. Verify that Alibaba's base model license is compatible with your use (Alibaba's Qwen models are typically Apache 2.0 or Qwen License; check Qwen/Qwen3.5-397B-A17B directly). (2) Model card recommends developers 'work with their internal model team to ensure this model meets requirements for the relevant industry and use case.' For regulated industries (healthcare, finance, automotive), internal compliance review is mandatory. (3) No SLA, warranty, or indemnification is stated. Use is at your risk.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model is unquantified for embedded adversarial robustness or formal verification. Considerations: (1) Quantization to FP4 may reduce robustness to adversarial perturbations vs. full-precision baseline; no formal analysis provided. (2) Model amplifies biases and toxic content from training data; prompt injection and jailbreaking risks are plausible and not mitigated by the model itself. (3) NVIDIA provides a vulnerability reporting channel (intigriti VDP link). (4) Inference engines (SGLang, vLLM) should be kept current to patch supply-chain or runtime vulnerabilities. (5) No formal security audit or threat model is documented. For regulated use, threat modeling and adversarial testing are required.
Alternatives to consider
Meta Llama 3.1 (70B or 405B)
Permissive Llama 2 License (Apache-compatible for commercial use under certain restrictions). Smaller (70B) for lower cost; larger (405B) for capability parity. Broader ecosystem (vLLM, TGI, Ollama, llama.cpp). No MoE sparsity but simpler quantization landscape.
Alibaba Qwen3.5-397B-A17B (unquantized FP8 or BF16)
Same base model, full precision or higher precision quantization. If VRAM and inference latency are not critical, better model quality. Avoid if cost and scale are primary drivers.
Anthropic Claude (via API)
Closed-source, hosted alternative for text-only tasks. No infrastructure overhead; strong on safety and alignment. Higher per-token cost; not self-hostable; no multimodal (video) input.
Ship Qwen3.5-397B-A17B-NVFP4 with senior software developers
Assess your GPU infrastructure, validate quantization performance on your workload, and configure tensor parallelism with SGLang or vLLM. Start with a small pilot on 4–8 GPUs. Contact us for deployment architecture review or fine-tuning strategy if your use case requires customization.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3.5-397B-A17B-NVFP4 FAQ
Can I use this model commercially in production?
How much GPU memory do I need?
Can I fine-tune this model?
Does this model support vision/video?
Custom software development services
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If Qwen3.5-397B-A17B-NVFP4 is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Ready to Deploy Qwen3.5-397B-A17B?
Assess your GPU infrastructure, validate quantization performance on your workload, and configure tensor parallelism with SGLang or vLLM. Start with a small pilot on 4–8 GPUs. Contact us for deployment architecture review or fine-tuning strategy if your use case requires customization.