Qwen3.5-122B-A10B-NVFP4
Qwen3.5-122B-A10B-NVFP4 is a quantized 122B parameter language model (with 10B active parameters via mixture-of-experts) derived from Alibaba's Qwen3.5-122B-A10B. It accepts text, image, and video inputs and outputs text. Quantized to NVFP4 format using NVIDIA Model Optimizer, it is optimized for inference on NVIDIA GPUs via SGLang. Licensed under Apache 2.0 and marked ready for commercial and non-commercial use. Last updated March 1, 2026.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | txn545 |
| Parameters | 64.4B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 120.8k |
| Likes | 24 |
| Last updated | 2026-03-01 |
| Source | txn545/Qwen3.5-122B-A10B-NVFP4 |
What Qwen3.5-122B-A10B-NVFP4 is
Transformer-based auto-regressive model using mixture-of-experts (MoE) architecture with 122B total parameters and 10B activated parameters. Supports multimodal input (text, RGB images, video in MP4/WebM), processes up to 262K token context. Quantized via post-training quantization to NVFP4 (weights and activations in linear transformer operators). Tested on NVIDIA B200 GPU. Requires SGLang runtime with specific PR merged. Calibrated on cnn_dailymail and Nemotron-Post-Training-Dataset-v2. Evaluated on GPQA benchmark.
Run Qwen3.5-122B-A10B-NVFP4 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="txn545/Qwen3.5-122B-A10B-NVFP4")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: With NVFP4 quantization (4-bit), approximate VRAM requirement is 60–80 GB for inference on a single GPU (rough calculation: 122B params × 4 bits ÷ 8 = ~61 GB base, plus runtime overhead). NVIDIA B200 or Blackwell microarchitecture recommended. Multi-GPU or distributed inference may be necessary for lower-VRAM setups. Exact VRAM and throughput figures are not provided in the card; verify with vendor or test.
No explicit fine-tuning guidance provided. Model is quantized post-training; standard LoRA/QLoRA approaches may be incompatible with NVFP4 quantization without additional research. Requires review with NVIDIA or the txn545 community for fine-tuning feasibility on quantized weights.
When to avoid it — and what to weigh
- CPU-only or non-NVIDIA GPU infrastructure — Model is designed and optimized for NVIDIA GPU-accelerated systems. Deployment on CPU or non-NVIDIA GPUs (AMD, Intel) is not supported per the documentation; performance will degrade significantly.
- You need guaranteed low latency on first token — No latency benchmarks are provided. With 122B parameters (even with 10B activated), time-to-first-token may not meet ultra-low-latency requirements without clear inference-time guarantees.
- Production systems without bias/toxicity tolerance — Model card explicitly warns that the base model may amplify toxic language and societal biases from internet training data. Requires downstream mitigation before use in sensitive domains (healthcare, legal, financial).
- Non-Linux operating systems — Preferred/tested OS is Linux. Windows and macOS support is not clearly stated; SGLang deployment on these platforms is Unknown.
License & commercial use
Licensed under Apache License 2.0 (permissive OSI-approved license). Full text available at the Apache license URL in the model card. No proprietary restrictions stated.
Model card explicitly states: 'This model is ready for commercial/non-commercial use.' Apache 2.0 is a permissive license that permits commercial use, modification, and distribution with attribution. No gating or additional commercial licensing fees are mentioned. However, verify compliance with the base model (Qwen3.5-122B-A10B) license terms, as this is a derivative work.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Moderate |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Good |
| Assessment confidence | Medium |
Model inherits security considerations from the base Qwen3.5-122B-A10B model. Card warns of toxic language and societal biases in training data. No security audits, adversarial robustness tests, or prompt-injection mitigations are documented. Requires downstream filtering/guardrails for production. SGLang runtime security posture is not described; review the SGLang project separately.
Alternatives to consider
Meta Llama 3.1 405B (gated, with commercial restrictions)
Larger open-weight model with stronger benchmarks, but gated and subject to Llama Community License (non-OSI); requires commercial licensing review.
Alibaba Qwen2.5-72B (unquantized, broader framework support)
Direct alternative from same vendor; unquantized offers more flexibility for fine-tuning and LoRA, but higher memory footprint and no NVFP4 optimization.
Mistral Large (closed weights, commercial API)
Enterprise-grade alternative if self-hosted quantized models prove insufficient; API-based eliminates infrastructure management at the cost of vendor lock-in.
Ship Qwen3.5-122B-A10B-NVFP4 with senior software developers
Evaluate this quantized model for your production LLM infrastructure. Download from HuggingFace and launch with SGLang on NVIDIA hardware. Test integration with your RAG or chatbot pipeline—no commercial licensing friction.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3.5-122B-A10B-NVFP4 FAQ
Can I use this commercially?
What GPU hardware do I need?
Can I fine-tune or use LoRA?
What is the exact context length?
Work with a software development agency
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If Qwen3.5-122B-A10B-NVFP4 is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Deploy Qwen3.5-122B-A10B-NVFP4 Today
Evaluate this quantized model for your production LLM infrastructure. Download from HuggingFace and launch with SGLang on NVIDIA hardware. Test integration with your RAG or chatbot pipeline—no commercial licensing friction.