Qwen3.6-27B-NVFP4
Qwen3.6-27B-NVFP4 is a 27-billion-parameter quantized language model from NVIDIA, derived from Alibaba's Qwen3.6-27B. It compresses the original model to 4-bit precision using NVIDIA's Model Optimizer, reducing memory footprint by approximately 2.5x while maintaining near-identical accuracy. The model handles text, image, and video inputs up to 262K context length and is optimized for inference on NVIDIA GPUs via vLLM.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | nvidia |
| Parameters | 18.2B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 538.7k |
| Likes | 315 |
| Last updated | 2026-06-30 |
| Source | nvidia/Qwen3.6-27B-NVFP4 |
What Qwen3.6-27B-NVFP4 is
A post-training quantized transformer with hybrid attention (Gated DeltaNet and Gated Attention) that uses NVFP4 (4-bit) precision for weights and activations in linear operators. Designed for vLLM inference on Hopper/Blackwell GPU microarchitectures. Calibrated on CNN-DailyMail and Nemotron-Post-Training-Dataset-v2. Evaluated on nine benchmarks including reasoning (MMLU Pro, GPQA, HLE), multimodal (MMMU Pro), coding (SciCode), and agentic tasks (τ²-Bench Telecom). Performance remains within 0.1–0.5 points of FP8 baseline across most benchmarks.
Run Qwen3.6-27B-NVFP4 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="nvidia/Qwen3.6-27B-NVFP4")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~6.5–7.5 GB VRAM for inference (27B params × 4 bits ÷ 8 + KV cache overhead). Requires NVIDIA Hopper or Blackwell GPU. Test hardware noted as GB300. Specific throughput (tokens/sec), batch size, and memory scaling are not provided; verify with vLLM profiling on target hardware.
Not documented. Model card does not mention LoRA, QLoRA, or parameter-efficient tuning compatibility. Quantization is post-training only. Contact NVIDIA or consult Model Optimizer documentation to assess fine-tuning feasibility on quantized weights.
When to avoid it — and what to weigh
- Extreme latency constraints or CPU-only deployment — Model is optimized for NVIDIA GPU inference via vLLM. No CPU or alternative runtime (TGI, llama.cpp) integration is documented. CPU inference will be prohibitively slow.
- Task requires unquantized model or full precision training — NVFP4 is a post-training quantization. Fine-tuning details and LoRA feasibility are not documented. Unquantized Qwen3.6-27B may be required if quantization loss is unacceptable for your domain.
- Toxic content or societal bias mitigation is mandatory — Model card explicitly notes the base model contains toxic language and societal biases from internet crawls. Model may amplify biases and return toxic responses, particularly with adversarial prompts. Domain-specific safety validation required.
- Non-Linux operating systems or non-Hopper/Blackwell hardware — Preferred OS is Linux; supported microarchitectures are Hopper and Blackwell only. Deployment on Windows, macOS, older NVIDIA GPUs, or AMD hardware is not documented.
License & commercial use
Apache 2.0. Permissive OSI-approved license permitting commercial and non-commercial use, modification, and distribution under Apache terms.
Model card states 'This model is ready for commercial or non-commercial use' and is governed by Apache 2.0. Apache 2.0 is a permissive OSI license that explicitly allows commercial use. No additional restrictions or proprietary licensing found in provided data. However, verify compliance with Alibaba's underlying Qwen3.6-27B license and any NVIDIA ModelOpt terms.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model card does not claim security properties. Practical considerations: (1) Quantization may introduce subtle numerical shifts; validate outputs in security-sensitive tasks. (2) vLLM runtime and CUDA stack are attack surface; keep driver and libraries updated. (3) Long context (262K tokens) exposes prompt-injection and context-leaking risks; sanitize user input. (4) Model inherits toxic language and bias from training data; use content filtering for user-facing deployments. No vulnerability disclosure process or security audit results provided.
Alternatives to consider
Qwen3.6-27B-FP8
Official baseline (Alibaba/HuggingFace). Higher precision (8-bit) trades ~2x memory savings for potential 0.1–0.5% accuracy gains on some benchmarks. Better for precision-critical tasks.
Qwen3.6-27B (unquantized)
Full 27B model at 16-bit precision. Larger footprint but allows fine-tuning and inference without quantization artifacts. Choose if model accuracy loss is unacceptable.
Llama 3.1 70B quantized (e.g., via TGI or llama.cpp)
Larger parameter count (70B vs 27B) with broader third-party runtime support (CPU-compatible quantization, multi-platform serving). Trade-off: higher compute cost, not NVIDIA-optimized.
Ship Qwen3.6-27B-NVFP4 with senior software developers
Qwen3.6-27B-NVFP4 combines 2.5x memory savings with near-baseline accuracy for high-throughput inference. Verify NVIDIA GPU compatibility, test vLLM integration, and validate quantization performance on your use case before production rollout.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3.6-27B-NVFP4 FAQ
Can I use this model commercially?
What GPU do I need?
Can I fine-tune this quantized model?
What is the context length?
Work with a software development agency
DEV.co helps companies turn open-source tools like Qwen3.6-27B-NVFP4 into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Ready to Deploy Quantized LLMs at Scale?
Qwen3.6-27B-NVFP4 combines 2.5x memory savings with near-baseline accuracy for high-throughput inference. Verify NVIDIA GPU compatibility, test vLLM integration, and validate quantization performance on your use case before production rollout.