granite-4.0-h-small
Granite-4.0-H-Small is a 32-billion-parameter instruction-tuned language model from IBM, released October 2025. It supports 12 languages, handles general NLP tasks (summarization, classification, QA, RAG, code), and includes tool-calling for function integration. Apache 2.0 licensed, ungated, and designed for enterprise applications. No context window length disclosed.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | ibm-granite |
| Parameters | 32.2B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 330.1k |
| Likes | 308 |
| Last updated | 2025-11-03 |
| Source | ibm-granite/granite-4.0-h-small |
What granite-4.0-h-small is
32.2B parameter dense transformer, instruction-tuned via supervised finetuning, reinforcement learning alignment, and model merging on permissive open-source + synthetic datasets. Supports structured chat format with system prompts, tool-calling via OpenAI function schema, and Fill-In-the-Middle code completion. Published 2025-11-03 on Hugging Face. Context length unknown. Trained on English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese.
Run granite-4.0-h-small locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="ibm-granite/granite-4.0-h-small")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: 32B dense model in bfloat16 requires ~64 GB GPU VRAM (e.g., A100 80GB, H100, or dual A100 40GB). FP8 quantization reduces to ~32 GB. CPU-only inference possible but slow. Exact requirements depend on batch size and serving framework; test on target hardware.
Model card confirms open-source instruction datasets and synthetic data were used. Fine-tuning via LoRA/QLoRA is feasible and documented in the transformers library. Model card does not detail LoRA rank recommendations or fine-tuning hyperparameters; refer to IBM's GitHub repository and huggingface training docs. Permissive license (Apache 2.0) allows commercial fine-tuning.
When to avoid it — and what to weigh
- Extremely Long Context Requirements — Context window length is not disclosed. If you require very long sequences (>8K tokens), verify context length or test empirically before committing.
- Real-Time Inference with Tight Latency Budget — 32B model requires significant compute. For sub-100ms latency, consider smaller models or quantized variants; evaluate on your hardware first.
- Specialized Domains Without Fine-Tuning Validation — Model is general-purpose instruction-tuned. Domain-specific tasks (medical, legal, scientific) may require fine-tuning and validation on representative data.
- License-Restricted Deployment Scenarios — While Apache 2.0 is permissive, verify compliance with internal IP policies, export controls, and any downstream commercial licensing requirements in your jurisdiction.
License & commercial use
Apache License 2.0. Permissive open-source license allowing use, modification, and distribution. No field-of-use or commercial restrictions.
Apache 2.0 is an OSI-approved permissive license. Commercial use, modification, and redistribution are allowed provided the license and copyright notice are retained. No patent protection guarantees. Verify internal IP and compliance policies, and confirm no downstream restrictions from your deployment platform (e.g., cloud provider terms).
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model card updated to include a default system prompt guiding 'professional, accurate, and safe' responses. No explicit red-teaming, adversarial robustness, or data leakage evaluation disclosed. Model is deployed via Hugging Face (gated=false) and Azure endpoints; verify endpoint security posture with your platform. Input sanitization and output validation are your responsibility. No formal security audit details published.
Alternatives to consider
Llama 2 / Llama 3 (Meta)
Similar parameter range, permissive license (though non-OSI for Llama 2 in some regions). Llama 3 offers comparable or better performance on many benchmarks and larger community. Trade-off: less recent IBM enterprise focus.
Mistral 7B/8x7B (Mistral AI)
Smaller, faster inference, Apache 2.0 licensed. Suitable if you can tolerate lower capability for cost/speed gains. MoE variant available for efficiency.
Comparable 32B variant, multilingual (Chinese-optimized), Apache 2.0 licensed. Good alternative if multilingual performance or cost in specific regions is a priority.
Ship granite-4.0-h-small with senior software developers
Start with a self-hosted or cloud deployment to evaluate tool-calling and multilingual capabilities. Our team can help you plan GPU requirements, fine-tuning strategy, and production architecture.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
granite-4.0-h-small FAQ
Can I use Granite-4.0-H-Small commercially?
What is the exact context window length?
How much GPU memory do I need to run this model?
Can I fine-tune this model for my domain?
Software developers & web developers for hire
DEV.co helps companies turn open-source tools like granite-4.0-h-small into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Ready to Deploy Granite-4.0-H-Small?
Start with a self-hosted or cloud deployment to evaluate tool-calling and multilingual capabilities. Our team can help you plan GPU requirements, fine-tuning strategy, and production architecture.