Qwen3-Coder-30B-A3B-Instruct-AWQ
Qwen3-Coder-30B-A3B-Instruct is a 30.5B-parameter mixture-of-experts coding model with 3.3B active parameters, quantized to INT4 by stelterlab using llm-compressor. It supports 256K native context (extendable to 1M with Yarn), excels at agentic coding and tool calling, and is available under Apache 2.0. This is a community quantization of Qwen's official base model, marked experimental.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | stelterlab |
| Parameters | 30.8B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 42.2k |
| Likes | 6 |
| Last updated | 2025-08-02 |
| Source | stelterlab/Qwen3-Coder-30B-A3B-Instruct-AWQ |
What Qwen3-Coder-30B-A3B-Instruct-AWQ is
Mixture-of-experts causal language model (48 layers, 32 Q-heads, 4 KV-heads, 128 experts with 8 activated). Native 262K context length. Quantized to INT4 GEMM using llm-compressor; original weights from Qwen AI. Instruct-tuned with support for function calling and tool use. Does not generate thinking blocks. Requires transformers>=4.51.0 for qwen3_moe support. Marked experimental by quantizing party.
Run Qwen3-Coder-30B-A3B-Instruct-AWQ locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="stelterlab/Qwen3-Coder-30B-A3B-Instruct-AWQ")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE (verify against your infra): INT4 quantization likely requires 16–20GB VRAM for inference at native 256K context; BF16 full model requires ~60GB VRAM. Inference on single A100 (80GB) or dual A6000 (48GB each) plausible. OOM mitigation recommended (reduce context to 32K if needed, per model card). Requires transformers>=4.51.0 and cuda/compatible accelerator for llm-compressor quantization format.
Not documented in provided card. Feasibility of LoRA/QLoRA on quantized INT4 weights is Unknown—requires testing. Fine-tuning the original unquantized Qwen3-Coder base model is likely more straightforward; quantization may introduce complications. Consult llm-compressor and vLLM project docs for QLoRA compatibility on GEMM INT4 models.
When to avoid it — and what to weigh
- Production deployments without quantization validation — This is a community quantization marked experimental; Qwen has not published official INT4 benchmarks for this version. Requires in-house evaluation on your specific coding tasks before production use.
- Need for reasoning or step-by-step thinking — Model explicitly does not generate <think></think> blocks and operates in non-thinking mode. If your workflow relies on chain-of-thought reasoning, consider alternatives or instruct-only models with explicit reasoning support.
- Single-GPU deployment with <24GB VRAM — 30.5B parameters in BF16/FP32 requires ~60GB; even quantized INT4, estimate ~16–20GB VRAM for full context. Multi-GPU or quantization required for smaller single-GPU setups.
- Guaranteed SLA and enterprise support — This is a community quantization by stelterlab, not an official release. No guaranteed support, uptime SLA, or vendor backing. Qwen AI support may not cover this quantized variant.
License & commercial use
Apache 2.0 (OSI-approved permissive license). Model itself is unencumbered; no gating. Derivative quantization by stelterlab also released under Apache 2.0.
Apache 2.0 permits commercial use, modification, and distribution with minimal restrictions (attribution required). However, this is a community quantization (experimental) by stelterlab, not an official Qwen release. For production commercial deployments, confirm with Qwen AI that using third-party quantizations does not void any indirect support or indemnification they may offer on the base model. No SLA or commercial support stated for this specific variant.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | Medium |
Standard LLM attack surface (prompt injection, jailbreaking, model extraction via API). No security audit or adversarial robustness claims in card. Community quantization (stelterlab) introduces supply-chain risk: verify weight integrity and review llm-compressor source. Quantization may alter model behavior subtly; test on sensitive tasks before production. No formal vulnerability disclosure process stated.
Alternatives to consider
Qwen3-Coder-30B-A3B-Instruct (official unquantized)
Same base model; avoids experimental quantization; official Qwen support; trade-off is 3x VRAM (~60GB unquantized BF16). Use if serving capacity allows.
DeepSeek-Coder-33B-Instruct or smaller code LLMs
Established code-focused models with published quantization benchmarks; lower experimental risk. Smaller alternatives (7B–13B) fit more constrained hardware.
Claude API or GPT-4 (hosted)
If no on-premise requirement: managed service, no quantization concerns, guaranteed SLA, and proven agentic capabilities. Higher latency and cost per token.
Ship Qwen3-Coder-30B-A3B-Instruct-AWQ with senior software developers
This model requires 16–20GB VRAM (INT4) for optimal inference. Validate quantization quality on your internal coding tasks before production. Contact Devco to review integration with your vector DB, inference framework, and agentic pipeline.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3-Coder-30B-A3B-Instruct-AWQ FAQ
Can I use this quantized model commercially?
What hardware do I need to run this locally?
Is this the official Qwen3-Coder quantization?
Does it support reasoning or chain-of-thought prompting?
Custom software development services
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If Qwen3-Coder-30B-A3B-Instruct-AWQ is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Ready to Deploy? Assess Your Infrastructure.
This model requires 16–20GB VRAM (INT4) for optimal inference. Validate quantization quality on your internal coding tasks before production. Contact Devco to review integration with your vector DB, inference framework, and agentic pipeline.