Qwen3.6-14B-A3B-FableVibes-GGUF
Qwen3.6-14B-A3B-FableVibes-GGUF is a 14B-parameter mixture-of-experts language model quantized for CPU/GPU inference via llama.cpp. It was created by pruning a larger Qwen model, then fine-tuned on reasoning traces from Claude Fable 5 to restore reasoning capability in a compact footprint. The model produces internal reasoning tokens before responding, making it suitable for complex problem-solving on consumer hardware.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | tvall43 |
| Parameters | Unknown |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 54.9k |
| Likes | 69 |
| Last updated | 2026-06-15 |
| Source | tvall43/Qwen3.6-14B-A3B-FableVibes-GGUF |
What Qwen3.6-14B-A3B-FableVibes-GGUF is
14B active-parameter MoE model derived from Qwen3.6-35B via REAP pruning. Fine-tuned with QLoRA on ~4,600 Claude Fable 5 reasoning traces, supplemented by Claude Opus reasoning, Qwen tool-calling, and Evol-Instruct-Code datasets. Distributed in GGUF format across 8 quantization levels (F16 to Q2_K), supporting llama.cpp-compatible runtimes. Includes vision/mmproj support. Outputs Qwen's thinking format (extended token budget for CoT reasoning).
Run Qwen3.6-14B-A3B-FableVibes-GGUF locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="tvall43/Qwen3.6-14B-A3B-FableVibes-GGUF")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: Q4_K_M quantization ~8.4GB VRAM (recommended for 8–12GB systems); Q6_K ~11.3GB; F16 ~27GB for full precision reference. Thinking-token overhead increases peak memory during inference. Verify with your target hardware and prompt length. CPU inference via llama.cpp is supported but significantly slower than GPU.
Model was trained via QLoRA on reasoning traces; the underlying pruned base (tvall43/Qwen3.6-14B-A3B-FableVibes) may accept further LoRA passes. GGUF format is primarily for inference and not typically used for direct fine-tuning; re-quantizing after training would be required. Unknown whether the pruned+LoRA base is suitable for additional instruction tuning or domain-specific adaptation.
When to avoid it — and what to weigh
- Latency-critical real-time applications — Thinking token generation (hundreds to thousands of tokens) before response creates substantial first-token and total latency. Not suitable for conversational chatbots, live API endpoints, or sub-second response SLAs.
- Inference at scale without quantization — F16 format (~27GB) requires datacenter hardware. Quantization is essential for production; untested combinations of base-model pruning + LoRA + quantization may introduce degradation not documented in the model card.
- Instruction-following requiring guaranteed safety — No mention of safety alignment, adversarial robustness, or alignment testing in the model card. Not evaluated for sensitive domains (legal, medical, financial) or guardrail-dependent use cases.
- Production use without quality validation — Model card does not provide benchmarks, eval results, or comparison to baseline Qwen3.6-35B. Pruning + LoRA recovery approach is novel; actual capability recovery extent is unknown.
License & commercial use
Apache 2.0. Permissive OSI-approved license allowing redistribution, modification, and commercial use subject to license/notice preservation.
Apache 2.0 is a permissive open-source license. Commercial use is allowed provided you include a copy of the license and provide notice of modifications. No additional commercial restrictions or gating mentioned. However: (1) the base model (Qwen3.6-35B) license should be verified independently; (2) training data (Claude Fable 5 traces, etc.) may have terms of use—review data source licenses for commercial use of derived models.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Unknown |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | Medium |
No security evaluation documented. Considerations: (1) Model inherits any vulnerabilities from base Qwen3.6-35B; (2) Fine-tuning data sources unknown for safety implications; (3) Thinking tokens may expose internal reasoning patterns—review for sensitive use; (4) Quantization may affect robustness to adversarial inputs (unknown); (5) Deployed on user hardware—implement standard LLM guardrails (prompt injection filtering, output validation, rate limiting).
Alternatives to consider
Llama 2 13B / Mistral 7B (base quantized models)
Widely adopted, well-benchmarked open-source alternatives. Llama 2 has stronger community support; Mistral is more efficient. Trade-off: less explicit reasoning-trace distillation.
DeepSeek-Coder 6.7B or similar MoE-based models
MoE architecture with code/reasoning focus, smaller footprint. Provides comparison point for MoE pruning/quantization trade-offs.
Local Claude Opus via API (vs. self-hosted)
If latency is acceptable, API-based frontier reasoning avoids quantization trade-offs and deployment complexity. Higher cost but eliminates hardware burden and provides safety guarantees.
Ship Qwen3.6-14B-A3B-FableVibes-GGUF with senior software developers
Start with the Q4_K_M quantization (8.4GB) on your target hardware. Download from Hugging Face, test with llama.cpp or Ollama, and benchmark against your use case before committing to production. Consider evaluating against Llama 2 / Mistral baselines and reviewing the underlying base-model license.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Qwen3.6-14B-A3B-FableVibes-GGUF FAQ
Can I use this model commercially?
What hardware do I need to run this model?
Why does the model take so long to respond?
Is there a benchmark showing how well this model performs?
Work with a software development agency
From first prototype to production, DEV.co delivers software development services around tools like Qwen3.6-14B-A3B-FableVibes-GGUF. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across open-source llms and beyond.
Ready to evaluate this model for your reasoning workflow?
Start with the Q4_K_M quantization (8.4GB) on your target hardware. Download from Hugging Face, test with llama.cpp or Ollama, and benchmark against your use case before committing to production. Consider evaluating against Llama 2 / Mistral baselines and reviewing the underlying base-model license.