gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF
Gemma-4-12B v2 is a 12B parameter LLM quantized to GGUF format, specialized for coding and agentic tool-use tasks. It runs locally on ~4.5 GB VRAM/unified memory. The model achieves ~55% on tau2-bench telecom (technical troubleshooting) vs. ~15% for the base Gemma-4-12B-it, indicating 3.5× improvement on agentic workflows. Trade-off: general knowledge (MMLU-Pro) is slightly lower than base. Licensed Apache-2.0, ungated, 384k downloads.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | yuxinlu1 |
| Parameters | Unknown |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 384.4k |
| Likes | 1.1k |
| Last updated | 2026-06-19 |
| Source | yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF |
What gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF is
GGUF quantization of a fine-tuned Gemma-4-12B-it base, specialized post-training for coding, terminal, tool-use, and multi-step reasoning. Model card reports: (1) tau2-bench telecom: 55% vs. 15% base (20 tasks, Q8_0, greedy); (2) zero fabrication on ground-truth probe; (3) base model transfers to human 10× on same tasks; v2 persists in agent loop. General knowledge slightly traded for agentic focus. Context length, exact parameter count, and sampling defaults not stated. Training data rebuilt with Anthropic Claude Opus 4.8 after Fable 5 retirement. Developer indicates v3 in progress.
Run gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~4.5 GB VRAM/unified memory (model card stated; verify with your target quantization: Q3_K_M, Q4_K_M recommended). Requires GGUF-compatible runtime (llama.cpp, Ollama, vLLM with GGUF support). Model card notes Q2_K discontinued; smallest reliable is Q3_K_M. Exact parameter count and context length not stated in card; requires independent verification.
Unknown. Model card does not discuss LoRA, QLoRA, or fine-tuning accessibility. Assume starting point is the safetensors master (reportedly open-sourced by developer), but adapter/continuation guidance not provided. Recommend contacting developer or reviewing source repository for training code.
When to avoid it — and what to weigh
- General-knowledge or broad-domain tasks required — Model card reports MMLU-Pro scores slightly below base; specialized post-training trades broad knowledge for agentic depth. Use base Gemma-4-12B-it or a generalist if breadth is critical.
- Frontier-class accuracy needed — Model card estimates v3 may reach 60–70% on tau2-bench telecom; frontier models (Opus 4.8, mimo-v2.5-pro) exceed 90%. Do not expect parity with larger or more general models.
- Real-time, high-throughput production without optimization — GGUF is CPU/local-first; latency at scale will exceed optimized cloud endpoints. Verify throughput and batch-size constraints for your SLA.
- Unsupported tool formats or inference framework — Model uses Gemma 4's native tool format; model card warns of garbled output if not parsed correctly (e.g., missing `--jinja` in llama.cpp). Requires compatible client.
License & commercial use
Apache-2.0 (OSI-approved permissive license). Ungated. Base model is google/gemma-4-12B-it; post-training is original work by developer (yuxinlu1).
Apache-2.0 permits commercial use, modification, and distribution provided license text is included and liability is disclaimed. No restrictions on commercial deployment stated in card. However, verify that google/gemma-4-12B-it (base) itself permits commercial use under its own license terms; model card does not restate base license. Recommend confirming base model commercial eligibility before production deployment.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
Local execution eliminates cloud inference risks. Model card reports zero fabrication on ground-truth probe; tool-use tasks correctly read/inspect before acting, reducing hallucination-driven false commands. No security audit or adversarial robustness evaluation stated. GGUF format is inert (no remote code). Verify tool-use output validation and rate-limiting in your calling application. No known CVEs or exploit details provided in card.
Alternatives to consider
google/gemma-4-12B-it (base)
Broader general knowledge (higher MMLU-Pro); if agentic specialization is not required, base is simpler and less trade-off.
Anthropic Claude (API or self-hosted)
Frontier-class performance (90%+ on tau2-bench); preferred if accuracy and real-time API latency are constraints. Higher cost.
Qwen 3.6-27B (developer's upcoming fine-tune)
Larger footprint (~27B) but potentially higher capability on agentic tasks; recommended if VRAM/memory permits and accuracy is prioritized over deployment simplicity.
Ship gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF with senior software developers
Download Gemma-4-12B v2 GGUF and run locally with llama.cpp or Ollama. No API keys, no cloud dependency. Start building custom agentic workflows today.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF FAQ
Can I use this model commercially?
How much GPU/CPU do I need?
Why is my output garbled or full of zeros?
How does v2 compare to the base model?
Custom software development services
From first prototype to production, DEV.co delivers software development services around tools like gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across open-source llms and beyond.
Deploy a Private Coding Agent in Minutes
Download Gemma-4-12B v2 GGUF and run locally with llama.cpp or Ollama. No API keys, no cloud dependency. Start building custom agentic workflows today.