gemma-4-12B-coder-fable5-composer2.5-v1-GGUF
Gemma-4-12B-Coder is a fine-tuned 12B parameter coding model optimized for local inference. It combines real chain-of-thought reasoning from passing test cases with synthetic reasoning traces, and runs on 4.5–12 GB VRAM depending on quantization. The model is distributed as GGUF (quantized) weights for llama.cpp and similar runtimes. Licensed under Apache 2.0, it is intended for Python/algorithmic coding tasks and has reduced refusals compared to the base model.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | yuxinlu1 |
| Parameters | Unknown |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 675k |
| Likes | 2.6k |
| Last updated | 2026-06-19 |
| Source | yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF |
What gemma-4-12B-coder-fable5-composer2.5-v1-GGUF is
A distilled derivative of google/gemma-4-12B-it, fine-tuned on dual chain-of-thought data: (1) Composer 2.5 real CoT verified by execution, (2) Fable 5 synthetic CoT for failure-case recovery. Uses Gemma 4's native thinking channel. Context window is 256K (recently corrected upstream metadata bug). Distributed in GGUF quantizations (Q2_K through Q8_0) for local inference. No gating. Requires recent llama.cpp (gemma4_unified architecture support).
Run gemma-4-12B-coder-fable5-composer2.5-v1-GGUF locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
Minimum ~4.5 GB VRAM/unified memory (Q2_K quant, ~2–4K context window on 8 GB systems). Recommended: 12–16 GB for Q4_K_M with ~30–64K context. Context window is 256K max, but sustained high-context inference requires 24+ GB (Q4_K_M) or 32 GB (Q8_0 full quality). ESTIMATE: Verify exact context capacity in your target environment; card provides rough guidance but actual throughput depends on KV cache quantization, CPU/GPU architecture, and inference library settings.
Full-precision safetensors weights published separately (yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1). LoRA/QLoRA adaptation is feasible; card does not explicitly document LoRA compatibility but Gemma 4 derivatives typically support it. Start from safetensors, not GGUF. Train on verifiable (test-gated) data to preserve reasoning quality. Card notes training data is task-focused; domain-specific fine-tuning likely requires task-specific annotation.
When to avoid it — and what to weigh
- Production safety-critical systems without guardrails — Model explicitly lacks safety alignment ('not safety-aligned'). Training data is task-focused with no refusal patterns. Do not deploy to user-facing production without additional safety filtering, input validation, and output vetting.
- Multi-language or general-knowledge tasks — Specialized for Python/algorithmic coding. Factual recall and non-English capability are unknown and likely weak. Card notes 'general-knowledge facts/numbers should still be double-checked.' Use a broader-trained model for QA, translation, or general assistance.
- Real-time, ultra-low-latency requirements — GGUF inference on CPU or modest GPU is slower than cloud APIs or dedicated inference endpoints. First-token latency and throughput depend heavily on hardware; not suitable for sub-100ms SLA requirements.
- Scenarios requiring proof of provenance or auditability — Personal/hobby project ('shared as-is, no warranty'). Limited transparency on training data curation (Composer 2.5, Fable 5 datasets not independently verified). No published benchmarks for non-tau2 domains. Difficult to audit or reproduce.
License & commercial use
Apache License 2.0. Google released Gemma 4 (unlike Gemma 1/2/3) under Apache 2.0, and this fine-tune inherits that license. License text: https://ai.google.dev/gemma/apache_2. Full permissions to use, modify, redistribute, and prepare derivative works, provided Apache 2.0 terms are retained.
Apache 2.0 is a permissive OSI license that explicitly permits commercial use, modification, and redistribution, provided you retain copyright notices and license text. No restrictions on proprietary derivatives or commercial applications. However, verify Google's Gemma 4 commercial policy and any downstream dependencies (e.g., training data sources). This is a personal/hobby project ('shared as-is, no warranty'); assess your organization's risk tolerance for using unsupported derivatives in production.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
No inherent vulnerabilities disclosed. Model lacks safety alignment; reduced refusals mean it may produce harmful, biased, or nonsensical code without guardrails. GGUF inference is local/offline (reduces network attack surface) but client-side input/output validation is the user's responsibility. Verify code generated by the model before execution (especially in multi-user or untrusted environments). Training data provenance (Composer 2.5, Fable 5) is opaque; no third-party security audit published.
Alternatives to consider
Llama 2 / Llama 3.2 (7B/8B) + custom coding fine-tune
Open-source, well-documented, broader community support. Requires your own fine-tuning but provides full control over training data and safety alignment. Slightly smaller default models; fewer thinking traces.
DeepSeek-Coder-6.7B / CodeLlama-7B
Purpose-built coding models with stronger non-Python language support and published benchmarks. More mature community fine-tuning examples. Trade-off: less chain-of-thought reasoning exposed.
Proprietary APIs (OpenAI Codex, Claude, Vertex AI Code Assist)
Broader capability, enterprise safety alignment, managed scaling. Higher latency/cost and data residency concerns; no offline option. Use if local deployment is infeasible or multi-language support is critical.
Ship gemma-4-12B-coder-fable5-composer2.5-v1-GGUF with senior software developers
Download a GGUF quant (Q4_K_M recommended), spin up llama-server, and start generating verifiable Python solutions offline. No API key, no cloud latency, full control. Review the full model card for context window estimates and sampling guidance.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
gemma-4-12B-coder-fable5-composer2.5-v1-GGUF FAQ
Can I use this model in a commercial product without paying royalties?
I have 8 GB of VRAM. What context window can I expect?
Do I need to use 'thinking mode'?
Is this safe to deploy to users without additional safeguards?
Custom software development services
Adopting gemma-4-12B-coder-fable5-composer2.5-v1-GGUF is usually one piece of a larger software development effort. As a software development agency, DEV.co provides software development services and web development expertise — pairing senior software developers and web developers with your team to design, build, and operate open-source llms software in production.
Ready to Deploy a Local Coding Assistant?
Download a GGUF quant (Q4_K_M recommended), spin up llama-server, and start generating verifiable Python solutions offline. No API key, no cloud latency, full control. Review the full model card for context window estimates and sampling guidance.