Qwen3-30B-A3B-FP8
Qwen3-30B-A3B-FP8 is a 30.5B-parameter mixture-of-experts (MoE) language model from Alibaba's Qwen team, with only 3.3B parameters active per token. It supports dual-mode operation: a thinking mode for complex reasoning (math, code, logic) and a non-thinking mode for fast general dialogue. The model is quantized to FP8 for reduced memory footprint, supports 32K native context (up to 131K with YaRN), and is trained on 100+ languages. It is available under Apache 2.0 and not gated.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | Qwen |
| Parameters | 30.5B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 450k |
| Likes | 84 |
| Last updated | 2025-07-26 |
| Source | Qwen/Qwen3-30B-A3B-FP8 |
What Qwen3-30B-A3B-FP8 is
FP8-quantized MoE causal language model with 48 transformer layers, 128 total experts (8 active per token), 32 query heads + 4 KV heads (GQA), and 29.9B non-embedding parameters. Native context window 32,768 tokens; supports extension to 131,072 via YaRN. Compatible with transformers>=4.51.0, vLLM>=0.8.5, SGLang>=0.4.6.post1, and local inference via Ollama, LMStudio, MLX-LM, llama.cpp, KTransformers. Requires fine-grained FP8 quantization support; distributed inference on transformers known to have issues with FP8 (may require CUDA_LAUNCH_BLOCKING=1).
Run Qwen3-30B-A3B-FP8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="Qwen/Qwen3-30B-A3B-FP8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: FP8 quantization (~31GB model size / 8 bits per param, plus overhead) suggests ~15–20GB GPU VRAM for inference on single GPU. MoE sparsity reduces per-token compute but not memory footprint. Exact requirements depend on batch size, context length, and serving framework. Multi-GPU setups or CPU+GPU offloading may be needed for thinking mode with long contexts. Verify with your inference framework's memory profiler.
Card does not explicitly document LoRA or QLoRA support for this quantized variant. Base model (Qwen3-30B-A3B) is post-trained; fine-tuning typically applied to unquantized BF16 checkpoint. FP8-quantized weights may require specialized quantization-aware fine-tuning or retraining of quantization parameters. Requires review of Qwen3 GitHub/documentation for official LoRA/QLoRA guidance. Recommend contacting Qwen team or consulting their GitHub issues for fine-tuning best practices on FP8 variant.
When to avoid it — and what to weigh
- Real-time ultra-low-latency applications requiring <100ms inference — Even with thinking mode disabled and FP8 quantization, 30B+ parameters will incur measurable latency. Thinking mode adds further overhead. Evaluate with your specific hardware; consider smaller 8B or 14B models if latency is critical.
- Limited context requirements but strict determinism needed — Thinking mode uses sampled decoding (Temperature=0.6, TopP=0.95) to avoid performance degradation and endless loops. If you require greedy/deterministic output, disable thinking mode, but you lose the reasoning advantage. Non-thinking mode still uses sampled decoding (Temperature=0.7).
- Edge deployment or CPU-only environments — Model size and FP8 quantization design target GPU inference. Quantization support for edge runtimes (llama.cpp, MLX) is available but not officially benchmarked. Local inference frameworks may lag behind official support.
- Distributed multi-GPU inference with transformers library — Known issue: fine-grained FP8 in transformers has compatibility issues under distributed setup. Requires workaround (CUDA_LAUNCH_BLOCKING=1) or use vLLM/SGLang instead.
License & commercial use
Apache 2.0 (OSI-approved, permissive open-source license). Allows commercial use, modification, and redistribution with minimal obligations: retain license and copyright notice, and disclose changes.
Apache 2.0 is a permissive OSI license with no restrictions on commercial use. You may use, modify, and redistribute Qwen3-30B-A3B-FP8 in proprietary products provided you: (1) include Apache 2.0 license text in distribution, (2) disclose material modifications, (3) state changes made to the code. No patent grant, liability limitations, or trademark use restrictions are provided beyond standard Apache 2.0 terms. This is clearer than many LLM licenses (e.g., Llama 2 Community License), but consult legal counsel for enterprise deployments to confirm compliance with your specific use case.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
No explicit security or safety features documented in card. Standard LLM considerations apply: (1) Model may generate harmful, biased, or false content; implement content filtering or human review in production. (2) Thinking mode outputs internal reasoning tokens; ensure logs and API endpoints do not expose sensitive user data or model internals inadvertently. (3) As an open model, weights are public; no confidentiality of model behavior. (4) FP8 quantization may introduce rounding artifacts; not suitable for cryptographic or security-critical applications. (5) Recommend security testing and prompt injection evaluation before production deployment, especially in agent-based workflows with tool integration.
Alternatives to consider
Qwen2.5-32B-Instruct or Qwen2.5-72B-Instruct
Qwen's previous-generation dense models. If you do not need dual thinking/non-thinking modes or MoE sparsity, Qwen2.5 may be simpler and have more mature deployment tooling. Trade-off: no reasoning mode, larger parameter count for same performance tier.
DeepSeek-R1 or similar open reasoning models
Alternative reasoning-focused open models. DeepSeek-R1 offers similar thinking/non-thinking capability. Evaluate on benchmarks (math, code, reasoning) specific to your use case. Note: DeepSeek uses different license (MIT or proprietary); verify commercial terms.
Mixtral 8x7B or similar open MoE models
If MoE efficiency is primary goal but reasoning is secondary, Mixtral or similar open MoE may be smaller and faster. Trade-off: no dual reasoning mode, less multilingual support, older training data. Suitable for cost-constrained scenarios.
Ship Qwen3-30B-A3B-FP8 with senior software developers
Start with vLLM or SGLang for production-grade inference, or experiment locally with Ollama or llama.cpp. Review hardware requirements, test thinking vs. non-thinking trade-offs with your workload, and ensure Apache 2.0 compliance for commercial use. Contact Devco for guidance on infrastructure, fine-tuning, or production deployment.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3-30B-A3B-FP8 FAQ
Can I use Qwen3-30B-A3B-FP8 in a commercial product without paying Qwen?
What GPU do I need to run this model?
How much slower is thinking mode than non-thinking mode?
Can I fine-tune the FP8 model directly?
Software development & web development with DEV.co
Need help beyond evaluating Qwen3-30B-A3B-FP8? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to deploy Qwen3-30B-A3B-FP8?
Start with vLLM or SGLang for production-grade inference, or experiment locally with Ollama or llama.cpp. Review hardware requirements, test thinking vs. non-thinking trade-offs with your workload, and ensure Apache 2.0 compliance for commercial use. Contact Devco for guidance on infrastructure, fine-tuning, or production deployment.