Qwen3-235B-A22B-Thinking-2507-FP8
Qwen3-235B-A22B-Thinking-2507-FP8 is an Apache-2.0-licensed, open-source 235B-parameter mixture-of-experts language model from Alibaba's Qwen team. It is FP8-quantized for deployment efficiency and emphasizes reasoning and extended thinking for complex problem-solving. The model supports a native 256K context length and can generate up to 81K tokens for challenging reasoning tasks. It is available without gating and has been benchmarked against leading closed-source models (GPT-4O, Claude, Gemini, DeepSeek-R1) on reasoning, coding, and knowledge tasks.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | Qwen |
| Parameters | 235.1B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 44.4k |
| Likes | 86 |
| Last updated | 2025-07-30 |
| Source | Qwen/Qwen3-235B-A22B-Thinking-2507-FP8 |
What Qwen3-235B-A22B-Thinking-2507-FP8 is
Qwen3-235B-A22B-Thinking-2507-FP8 is a causal language model with Mixture-of-Experts (MoE) architecture: 235B total parameters with 22B activated at inference; 94 layers; 128 experts (8 activated per forward pass); GQA with 64 Q-heads and 4 KV-heads. Quantized to FP8 (block size 128) for memory and inference efficiency. Context window: 262,144 tokens (native). Requires transformers ≥4.51.0 for proper tokenizer support. Compatible with vLLM (≥0.8.5), SGLang (≥0.4.6.post1), and local tools (Ollama, LMStudio, llama.cpp, KTransformers). Deployment frameworks must support the 'qwen3_moe' architecture. Model enforces internal reasoning (thinking) via default chat template; output includes explicit </think> tokens.
Run Qwen3-235B-A22B-Thinking-2507-FP8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="Qwen/Qwen3-235B-A22B-Thinking-2507-FP8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE (unverified; requires independent testing): FP8 quantization (~120–150 GB VRAM for full model on single GPU; tensor-parallel sharding recommended for 4+ GPUs). Model card recommends context length ≥131K for reasoning; OOM mitigation may require reducing context or using vLLM/SGLang with paged attention. Inference framework (vLLM/SGLang) essential for memory-efficient serving. Exact VRAM and throughput depend on quantization, context length, and inference framework tuning.
Not explicitly documented in the provided model card. Fine-tuning feasibility (LoRA/QLoRA) is Unknown. The model is positioned as a pre-trained and post-trained base model; no official fine-tuning guidelines, datasets, or target use cases are stated. Organizations considering fine-tuning should consult the GitHub repository (https://github.com/QwenLM/Qwen3) or Qwen documentation for supported methodologies.
When to avoid it — and what to weigh
- Latency-critical real-time applications — Model size (235B, 22B active) and generated output up to 81K tokens for complex reasoning create high latency. Not suitable for sub-second response requirements without sophisticated serving optimization.
- Single-token or streaming constrained environments — Thinking mode is mandatory and cannot be disabled. Output always includes internal reasoning tokens (</think>), increasing response length. Environments requiring fixed-size responses or minimal token generation overhead should evaluate alternatives.
- Limited VRAM or edge/mobile deployments — 235B parameters require substantial GPU memory even in FP8 (model card recommends context length ≥131K for reasoning; OOM issues reported at reduced context). FP8 quantization helps but full model unsuitable for edge devices or small inference clusters.
- Non-English-only use cases with minimal multilingual testing — While MultiIF and multilingual benchmarks are reported, testing outside English is less comprehensive than English. Organizations prioritizing non-English performance should validate on actual workloads.
License & commercial use
Apache License 2.0 (OSI-compliant permissive open-source license). Allows commercial use, modification, and distribution with attribution and no warranty. No artificial restrictions; model is ungated and downloadable without approval.
Apache-2.0 is an OSI-approved permissive license permitting commercial use, modification, and sublicensing. No additional restrictions stated in the model card. Commercial deployment is permitted under Apache-2.0 terms (attribution required; no liability accepted). Verify license compliance with legal counsel for production deployments. No proprietary usage restrictions or commercial licensing terms are implied.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
Typical LLM security considerations apply: (1) Model outputs should not be treated as ground truth without validation; (2) Long-context processing (256K tokens) and extended reasoning could enable prompt injection or exfiltration attacks if inputs are untrusted; (3) FP8 quantization does not introduce documented security risks but reduces precision and may alter adversarial robustness vs. BF16 variant; (4) Open-source model weights are publicly available, enabling offline analysis but also adversarial reverse-engineering; (5) Thinking-mode output includes internal reasoning, which may leak model behavior or confidential problem structure if logs are exposed. Recommended: validate inputs, isolate reasoning traces, and audit logs in production deployments.
Alternatives to consider
DeepSeek-R1 (open-source, similar thinking-based reasoning)
DeepSeek-R1 also emphasizes extended reasoning and is open-source, with comparable benchmarks on reasoning tasks. Smaller model sizes available (7B–671B range) may be more memory-efficient for some use cases. Requires independent evaluation of commercial-use license and community support.
OpenAI GPT-4O / GPT-4-Turbo (closed-source, highest reasoning benchmarks)
Proprietary models with higher AIME25 (92.7), HMMT25 (82.5), and broader real-world deployment support. No self-hosting or fine-tuning; API-based pricing. Preferred for enterprises avoiding open-source dependency risk but accept vendor lock-in and per-token costs.
Llama 3.1-405B (open-source, larger unified model, no enforced thinking)
Ship Qwen3-235B-A22B-Thinking-2507-FP8 with senior software developers
Qwen3-235B-A22B-Thinking-2507-FP8 is production-ready for enterprises building AI agents, research tools, and code-generation systems. Start with vLLM or SGLang for efficient inference, or consult our team to architect your reasoning LLM pipeline.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3-235B-A22B-Thinking-2507-FP8 FAQ
Can I use this model commercially without additional licensing?
What GPU memory do I need to run Qwen3-235B-A22B-Thinking-2507-FP8?
Is fine-tuning (LoRA, QLoRA) supported for this model?
How does the 'thinking' mode work, and can I disable it?
Work with a software development agency
DEV.co helps companies turn open-source tools like Qwen3-235B-A22B-Thinking-2507-FP8 into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Deploy Advanced Reasoning with Qwen3
Qwen3-235B-A22B-Thinking-2507-FP8 is production-ready for enterprises building AI agents, research tools, and code-generation systems. Start with vLLM or SGLang for efficient inference, or consult our team to architect your reasoning LLM pipeline.