Qwen3-235B-A22B-Instruct-2507-FP8
Qwen3-235B-A22B-Instruct-2507-FP8 is a 235-billion-parameter mixture-of-experts (MoE) large language model from Alibaba's Qwen team. It activates only 22B parameters at inference time, reducing computational cost while maintaining strong performance across knowledge, reasoning, coding, and multilingual tasks. The model natively supports 262K token context and is distributed in FP8 quantized format for faster inference. It is available under the Apache 2.0 license with no gating restrictions.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | Qwen |
| Parameters | 235.1B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 221.4k |
| Likes | 148 |
| Last updated | 2025-09-17 |
| Source | Qwen/Qwen3-235B-A22B-Instruct-2507-FP8 |
What Qwen3-235B-A22B-Instruct-2507-FP8 is
A causal language model with 235B total parameters (234B non-embedding), 94 layers, 128 experts (8 activated per token), and 262K native context length. Quantized to FP8 with block size 128. Uses grouped query attention (64 Q heads, 4 KV heads). Supports non-thinking mode only (no internal reasoning blocks). Optimized for transformers ≥4.51.0, compatible with vLLM (≥0.8.5) and SGLang (≥0.4.6.post1) for distributed serving. Last modified September 2025.
Run Qwen3-235B-A22B-Instruct-2507-FP8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="Qwen/Qwen3-235B-A22B-Instruct-2507-FP8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: FP8 quantized 235B model likely requires 120–150 GB GPU VRAM for single-inference (FP8 ≈ 235B × 1 byte + overhead ≈ 235 GB nominal, reduced by quantization to ~120–150 GB). Distributed inference (tensor parallelism ≥4 GPUs, e.g., H100s) recommended. Documentation notes potential OOM; suggests reducing context to 32K if needed. Exact memory/throughput numbers not provided; requires profiling on target hardware.
No explicit fine-tuning or LoRA/QLoRA guidance in excerpt. Model card links to blog/GitHub for details. FP8 quantization may complicate gradient-based fine-tuning; verify compatibility with transformers Trainer or external frameworks (e.g., unsloth, bitsandbytes). Apache 2.0 permits derivative works, but implementation feasibility and performance regression unknown without empirical testing.
When to avoid it — and what to weigh
- Single-GPU or edge deployment — FP8 fine-grained quantization requires transformers ≥4.51.0 with known distributed inference issues; fine-grained fp8 in transformers has bugs requiring CUDA_LAUNCH_BLOCKING=1. 235B model demands multi-GPU setup; no evidence of mobile/edge optimization.
- Latency-critical real-time applications — Model card recommends 16K output tokens for typical queries; MoE dispatch overhead and context length of 262K imply non-trivial per-token latency. Prefer smaller models or ensemble methods for sub-100ms response targets.
- Internal reasoning transparency requirements — Model operates in non-thinking mode only; no chain-of-thought or intermediate reasoning visible to user. If interpretability or audit trails of reasoning steps are required, this architecture is unsuitable.
- Proprietary training or fine-tuning without review — Apache 2.0 permits commercial use, but no explicit mention of fine-tuning permissions, data privacy terms, or usage limits in provided excerpt. Model card links to blog and docs; verify terms before integrated training workflows.
License & commercial use
Apache License 2.0 (OSI-approved permissive open-source license). Permits commercial use, modification, and distribution with attribution and liability disclaimer. No explicit restrictions on model weights, derivative models, or commercial deployment stated in excerpt.
Apache 2.0 is a permissive OSI license that explicitly permits commercial use. However, the provided excerpt does not detail contractual restrictions, liability caps for inference-as-a-service, trademark use, or data/privacy obligations. Before deploying in production or as a commercial service, review the full license terms on the HuggingFace repository and Qwen documentation. No gating restrictions detected.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
No explicit security analysis, adversarial robustness data, or bias audits in excerpt. Model trained on data through unspecified date; potential hallucinations, outdated knowledge, and long-tail errors noted in agent benchmark variance. FP8 quantization may reduce explainability of weight distributions. Recommend: input validation, output guardrails, and testing against prompt injection before production deployment. No evidence of safety fine-tuning or red-teaming results disclosed.
Alternatives to consider
Deepseek-V3
Similarly scaled MoE model with comparable reasoning performance (AIME25 46.6 vs. 70.3 here, but stronger agent task variability). Alternative if provider diversification or different licensing terms required.
Claude Opus 4 (non-thinking)
Closed-source baseline with stronger MMLU-Pro (86.6 vs. 83.0) and some agentic tasks. Preferred if regulatory compliance, liability insurance, or vendor support are critical; sacrifices self-hosting and cost control.
Llama 3.1 (405B or smaller variants)
Meta's open-source alternative with similar capabilities at different scales. Meta's Llama license is permissive but requires review for commercial use. May offer better fine-tuning ecosystems and community support at trade-off to performance on reasoning benchmarks.
Ship Qwen3-235B-A22B-Instruct-2507-FP8 with senior software developers
Start with vLLM or SGLang for multi-GPU distributed inference. Review the full license and fine-tuning documentation on the HuggingFace repository and Qwen GitHub before production deployment. Contact your infrastructure team to provision 120–150 GB GPU VRAM and validate FP8 quantization compatibility with your serving framework.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3-235B-A22B-Instruct-2507-FP8 FAQ
Can I use this model for a commercial SaaS product?
What GPU hardware do I need to run this model?
Does this model support thinking/chain-of-thought output?
How does the 235B / 22B parameter split affect my deployment?
Software development & web development with DEV.co
From first prototype to production, DEV.co delivers software development services around tools like Qwen3-235B-A22B-Instruct-2507-FP8. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across open-source llms and beyond.
Ready to Deploy Qwen3-235B-A22B-Instruct-2507-FP8?
Start with vLLM or SGLang for multi-GPU distributed inference. Review the full license and fine-tuning documentation on the HuggingFace repository and Qwen GitHub before production deployment. Contact your infrastructure team to provision 120–150 GB GPU VRAM and validate FP8 quantization compatibility with your serving framework.