Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF
A 35B parameter reasoning-focused LLM fine-tuned via supervised learning on Claude Opus chain-of-thought data. Distributed as GGUF quantizations for local inference via llama.cpp. Apache 2.0 licensed, non-gated. Shows +32.85 percentage-point improvement on MMLU-Pro versus base model.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | hesamation |
| Parameters | Unknown |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 40.6k |
| Likes | 285 |
| Last updated | 2026-04-19 |
| Source | hesamation/Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF |
What Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF is
Qwen3.6-35B-A3B base model fine-tuned using LoRA (rank 32, attention-only) on ~14,200 reasoning-distillation samples from three datasets (Claude Opus trajectories, curated Qwen examples, additional Claude reasoning). Trained for 2 epochs over 762 steps with 8192-token max sequence length and 32768 context window support. Merged and quantized to GGUF format (Q4_K_M, Q5_K_M, Q6_K, Q8_0 variants) for llama.cpp inference. Text-only fine-tuning; vision encoder inherited from base but not tuned.
Run Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="hesamation/Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE ONLY—verify for your setup: Q4_K_M ~14–16 GB VRAM (consumer GPU; low-end server); Q5_K_M ~18–20 GB; Q6_K ~22–24 GB; Q8_0 ~28–32 GB (closest to fp16 baseline). Base model 35B fp16 would require ~70 GB. Exact footprint depends on llama.cpp batch size, context length, and quantization implementation. CPU inference possible but slow.
Source model trained with LoRA (rank 32, alpha 32, attention-only) using Unsloth. Merged checkpoint available. Further LoRA tuning on merged model is feasible; re-quantizing after tuning requires llama.cpp toolchain. Low LoRA rank (32) suggests conservative budget; may be suitable for downstream LoRA on resource-limited setups. QLoRA compatibility unknown; not stated in card.
When to avoid it — and what to weigh
- Multimodal/vision requirements — Base Qwen3.6 includes vision encoder but this fine-tune is text-only. Image/video handling not improved by training; treat as base model capabilities only.
- Inference with latency-critical SLAs — 35B parameter model requires substantial compute. Q4_K_M quantization reduces quality; Q6–Q8 demand significant VRAM. Production latency depends heavily on quantization and hardware.
- Minimal documentation or training details needed — Card provides training hyperparameters and dataset sources but limited evaluation beyond one smoke-test MMLU-Pro run (70 samples with --limit 5). Quantization impact on downstream performance not characterized.
- Long-running production systems without community validation — Model is community-maintained by single developer. No official benchmarks on real-world tasks. Card explicitly welcomes independent community benchmarks; limited production reference deployments likely.
License & commercial use
Apache 2.0 (OSI-approved permissive license). No usage restrictions on commercial or proprietary application.
Apache 2.0 is an OSI-permissive license allowing commercial use, modification, and distribution with attribution and liability disclaimers. No commercial-use restrictions. However: (1) Base model (Qwen3.6-35B-A3B) and training data (Claude Opus, third-party reasoning datasets) may have separate license/TOS. Verify Qwen team and dataset owner licenses independently. (2) Community-maintained model; no SLA or commercial support offered. (3) Derivative works (GGUF quantizations) inherit Apache 2.0 but ensure upstream base-model license chain is compatible for your use case.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Unknown |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | Medium |
No explicit security analysis provided in card. Considerations: (1) Training data sourced from Claude Opus reasoning and community-curated datasets; potential for biased or harmful outputs inherited from training data. (2) Model card does not address jailbreak resilience, adversarial input handling, or content-filtering. (3) Local inference reduces data exfiltration risk vs. cloud APIs but operator is responsible for access control and model safety. (4) GGUF quantization is mechanically deterministic (no novel attack surface) but should be run in isolated/sandboxed environments if used on untrusted input.
Alternatives to consider
Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled (Jackrong)
Smaller (27B vs. 35B), same reasoning-distillation approach, likely lower VRAM footprint. Fewer downloads (40k for this model) suggests Jackrong version may have earlier validation; trade-off is parameter count.
Llama 3.1 (Meta) with LoRA fine-tuning on reasoning data
Larger official community (Meta-backed), extensive tooling (vLLM, TGI native support), clearer licensing. Requires own LoRA tuning if reasoning-specific behavior desired; more development overhead but better production readiness.
Mixtral 8x22B or similar MoE models
Base Qwen3.6 includes MoE (qwen3_moe tag noted). Mixtral offers comparable reasoning capability with sparser inference (faster latency) and wider ecosystem support. Larger context, more mature quantization ecosystem.
Ship Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF with senior software developers
Evaluate this model for cost-sensitive, on-premise reasoning workloads. Start with Q5_K_M quantization (~18–20 GB VRAM) and benchmark on your task before production rollout. Community-maintained; Devco can assist with integration, fine-tuning, and deployment validation.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF FAQ
Can I use this commercially?
What GGUF quantization should I choose?
How do I deploy this with llama.cpp?
What does the +32.85pp MMLU-Pro improvement mean?
Software development & web development with DEV.co
DEV.co helps companies turn open-source tools like Qwen3.6-35B-A3B-Claude-4.6-Opus-Reasoning-Distilled-GGUF into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Deploy Reasoning-Enhanced LLM Locally
Evaluate this model for cost-sensitive, on-premise reasoning workloads. Start with Q5_K_M quantization (~18–20 GB VRAM) and benchmark on your task before production rollout. Community-maintained; Devco can assist with integration, fine-tuning, and deployment validation.