granite-4.0-micro
Granite-4.0-Micro is a 3.4B parameter instruction-tuned language model from IBM released in October 2025. It supports 12 languages and is designed for enterprise applications including RAG, code tasks, and tool-calling. Licensed under Apache 2.0, it can be self-hosted and is not gated. The model is compact enough for edge deployment while maintaining instruction-following and function-calling capabilities.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | ibm-granite |
| Parameters | 3.4B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 33.6k |
| Likes | 274 |
| Last updated | 2025-11-03 |
| Source | ibm-granite/granite-4.0-micro |
What granite-4.0-micro is
A 3.4B dense transformer model fine-tuned from Granite-4.0-Micro-Base using supervised fine-tuning, reinforcement learning alignment, and model merging. Trained on open-source instruction datasets and synthetic data. Features structured chat formatting, tool-calling support (OpenAI function schema compatible), and Fill-In-the-Middle code completion. Context length not specified in documentation. Compatible with standard transformers-based deployment (CUDA/CPU supported via provided examples).
Run granite-4.0-micro locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="ibm-granite/granite-4.0-micro")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
Estimated 13–14 GB GPU VRAM for bfloat16 inference (3.4B params × 4 bytes/param + KV cache + overhead). Quantization (int8, int4) likely reduces to 7–8 GB. CPU inference possible but slow. Exact precision and context length untested by us; verify against your workload.
Model card indicates it was fine-tuned using SFT, RL alignment, and model merging. LoRA/QLoRA fine-tuning feasible on 24 GB+ VRAM GPUs. No explicit mention of LoRA adapters or example fine-tuning scripts provided; you will need standard transformers/PEFT libraries. Recommended for adaptation to domain-specific tasks or custom instruction styles.
When to avoid it — and what to weigh
- High-latency-sensitive, real-time applications — 3B parameters will have longer first-token-latency and lower throughput than larger models. Inference speed untested; requires benchmarking against your hardware.
- Domain-specific expert performance required — MMLU scores (65.98%) and MMLU-Pro (44.5%) are moderate for instruction models. If your use case demands >75% accuracy on specialized knowledge, evaluate against task-specific baselines.
- Long-context retrieval tasks without evaluation — Context length is not documented. If application requires >8K token windows, contact IBM or test empirically before committing.
- Strict output determinism or format guarantees — Tool-calling output format is JSON-based but generation-based; no hard guarantees on JSON validity under all conditions. Requires output validation in production.
License & commercial use
Apache License 2.0. Permissive open-source license allowing commercial use, modification, and distribution. Model is not gated. Training data includes permissive-licensed open-source datasets and IBM's internal synthetic data.
Apache 2.0 is a permissive OSI-approved license. Commercial use, deployment in closed products, and hosted services are permitted under Apache 2.0 terms (attribution and license notice required). No special enterprise agreement or paid licensing required. Model is not gated or metered. Verify compliance with any internal IP policies, but license itself does not restrict commercial deployment.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model is instruction-tuned with alignment via RL; default system prompt added to encourage 'professional, accurate, and safe' responses. However, no red-teaming results, adversarial robustness metrics, or known vulnerability disclosures are provided. Self-hosted deployment eliminates cloud-provider data exposure but requires your own infrastructure security. Tool-calling outputs (JSON) must be validated before execution; no built-in output sanitization claimed.
Alternatives to consider
Mistral-7B / Mistral Small
Larger (7B) dense model with stronger instruction-following and broader benchmark availability. Apache 2.0 licensed. Trade-off: ~2.3x more VRAM and latency.
Phi-4 or Phi-3
Microsoft's small instruct models (3B–14B range). MIT licensed, optimized for edge. Trade-off: less multilingual support and different training philosophy.
Llama-3.2-1B / Llama-3.2-3B
Meta's compact instruct models with strong benchmark performance. Llama 3.2 license (commercial-friendly). Trade-off: requires separate license review; Llama community larger.
Ship granite-4.0-micro with senior software developers
Start with a proof-of-concept: deploy to your infrastructure using the provided code snippets, benchmark inference latency and accuracy on your domain data, and validate tool-calling JSON output. Contact IBM Granite team for undocumented details (context length, fine-tuning best practices). For managed deployment, explore Devco's private-LLM or RAG service.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
granite-4.0-micro FAQ
Can I use this model commercially without paying IBM?
What GPU do I need to run this?
Is the context length sufficient for my RAG use case?
How do I fine-tune this for my domain?
Custom software development services
Need help beyond evaluating granite-4.0-micro? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to evaluate Granite-4.0-Micro for your use case?
Start with a proof-of-concept: deploy to your infrastructure using the provided code snippets, benchmark inference latency and accuracy on your domain data, and validate tool-calling JSON output. Contact IBM Granite team for undocumented details (context length, fine-tuning best practices). For managed deployment, explore Devco's private-LLM or RAG service.