Qwen3-Next-80B-A3B-Instruct-FP8
Qwen3-Next-80B-A3B-Instruct-FP8 is an 80-billion-parameter open-source language model from Alibaba's Qwen team, optimized for instruction-following tasks. It uses a hybrid attention architecture (combining Gated DeltaNet and Gated Attention) and sparse mixture-of-experts (MoE) to reduce computational cost while supporting up to 256K native context length (extensible to 1M). The FP8 quantization reduces memory footprint compared to the bfloat16 base model. It is licensed under Apache 2.0, not gated, and suitable for deployment on multi-GPU infrastructure.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | Qwen |
| Parameters | 81.3B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 402.2k |
| Likes | 90 |
| Last updated | 2025-09-22 |
| Source | Qwen/Qwen3-Next-80B-A3B-Instruct-FP8 |
What Qwen3-Next-80B-A3B-Instruct-FP8 is
The model combines 12 hybrid blocks of Gated DeltaNet and Gated Attention layers with a 512-expert MoE system (10 active experts per token, 1 shared). It activates only 3B parameters per token despite 80B total, reducing FLOPs per inference step. FP8 quantization uses fine-grained block-size-128 quantization; benchmarks cited are from the bfloat16 pre-quantization model. Training used 15 trillion tokens. Native context length is 262,144 tokens with reported extensibility to 1,010,000. Multi-token prediction (MTP) is integrated for inference acceleration. Deployment supported via sglang and vLLM with tensor parallelism.
Run Qwen3-Next-80B-A3B-Instruct-FP8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="Qwen/Qwen3-Next-80B-A3B-Instruct-FP8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
Estimated 120–160 GB GPU VRAM for FP8 quantization in full precision serving (4× A100 40GB or 2× H100 80GB recommended for 256K context with tensor parallelism). Actual requirements depend on batch size, context length, and framework overhead. Card notes default context 256K; smaller contexts (e.g., 32K) may fit on fewer GPUs. Training fine-tuning details not provided; inference optimized for tensor parallelism.
Card does not document LoRA, QLoRA, or full fine-tuning feasibility for FP8 variant. Base model card references suggest standard transformer fine-tuning possible, but FP8-quantized checkpoints may require post-quantization tuning or dequantization before adaptation. Recommend consulting Qwen team or base model documentation for fine-tuning best practices.
When to avoid it — and what to weigh
- Reasoning-Heavy Specialized Tasks Requiring Extended Thinking — Model card explicitly states FP8 version supports only instruct mode and does not generate <think></think> blocks. For chain-of-thought or long-form reasoning, the non-quantized base model or a thinking-capable variant may be required.
- Single-GPU or Low-VRAM Environments — 80B parameter model requires multi-GPU tensor parallelism (4 GPUs typical in documentation examples). Per-token activation of 3B does not eliminate full model weight storage; FP8 quantization helps but does not enable single-consumer-grade GPU deployment.
- Latency-Critical, Low-Throughput Scenarios — While MTP and sparse MoE improve throughput for batch processing, real-time ultra-low-latency applications may benefit from smaller models. Startup latency and per-token time depend on hardware and serving framework tuning.
- Strict Licensing Compliance with Non-Commercial-Only Models — Apache 2.0 permits commercial use, but any deployment should verify integration with proprietary dependencies (sglang/vLLM versions, infrastructure) for license compatibility.
License & commercial use
Apache License 2.0. Permissive open-source license allowing redistribution, modification, and commercial use with attribution and liability disclaimer. No proprietary or restricted-use clauses stated.
Apache 2.0 permits commercial deployment, commercial modification, and sale of services using this model. No gating, no paid tier restriction. However, verify that serving frameworks (sglang, vLLM) and any dependent infrastructure comply with your internal licensing policy. No warranty or support SLA from Qwen stated in model card; support likely community-driven or via Alibaba's commercial Qwen service offerings.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
No security audit, red-teaming results, or adversarial robustness data provided. Model card does not disclose known vulnerabilities or jailbreak resistance. As a large instruction-tuned model, standard LLM risks apply: prompt injection, misuse for content generation, and potential bias. Deployment should include input filtering, output monitoring, and rate limiting. FP8 quantization does not modify security posture. Recommend independent evaluation for sensitive use cases (healthcare, finance, legal).
Alternatives to consider
Qwen3-235B-A22B-Instruct-2507
Same Qwen family, larger dense model with stronger performance on some benchmarks (MMLU-Pro 83.0 vs. 80.6, GPQA 77.5 vs. 72.9), but higher compute cost and no Sparse MoE reduction. Choose if maximum capability and reasoning are critical.
Meta Llama 3.1 70B (or 405B)
Alternative open-source 70B+ model, but under Llama 2 Community License (non-OSI). Requires commercial review. May have different capability profile and deployment ecosystem; context length typically shorter unless extended.
DeepSeek-LLM or Mixture-of-Experts alternatives (e.g., Mixtral)
Competing sparse MoE designs; trade-offs in architecture, context window, and benchmark performance vary. Evaluate based on specific throughput, latency, and capability requirements.
Ship Qwen3-Next-80B-A3B-Instruct-FP8 with senior software developers
Evaluate Qwen3-Next-80B-A3B-Instruct-FP8 for your organization. Start with a small pilot on multi-GPU infrastructure using sglang or vLLM, test benchmark scores on your workloads, and verify licensing compliance with your legal and infrastructure teams.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3-Next-80B-A3B-Instruct-FP8 FAQ
Can I use this model commercially?
What GPU setup do I need?
Does the FP8 quantization hurt accuracy?
Can I fine-tune this model?
Software development & web development with DEV.co
Need help beyond evaluating Qwen3-Next-80B-A3B-Instruct-FP8? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to Deploy a High-Efficiency Open LLM?
Evaluate Qwen3-Next-80B-A3B-Instruct-FP8 for your organization. Start with a small pilot on multi-GPU infrastructure using sglang or vLLM, test benchmark scores on your workloads, and verify licensing compliance with your legal and infrastructure teams.