granite-docling-258M
Granite Docling 258M is a lightweight multimodal model (258M parameters) designed to convert document images into structured text and markdown. Built by IBM Research, it combines a vision encoder (SigLIP2) with a small language model (Granite 165M) and integrates directly into the Docling document processing pipeline. It handles PDFs, images, and URLs, extracting text, tables, formulas, and layout information with support for English and experimental support for Japanese, Arabic, and Chinese.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | ibm-granite |
| Parameters | 258M |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | image-text-to-text |
| Gated on HuggingFace | No |
| Downloads | 70k |
| Likes | 1.2k |
| Last updated | 2025-09-23 |
| Source | ibm-granite/granite-docling-258M |
What granite-docling-258M is
A Vision-Language Model (VLM) based on Idefics3 architecture with custom modifications: vision encoder replaced with siglip2-base-patch16-512 and language backbone with Granite 165M. Operates as an image-text-to-text model outputting DocTags (structured markup) compatible with docling-core for conversion to markdown, HTML, and other formats. Supports inference via transformers, vLLM, ONNX, and MLX-VLM frameworks. Trained on SynthCodeNet, SynthFormulaNet, SynthChartNet, and DoclingMatrix datasets. Released September 17, 2025.
Run granite-docling-258M locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="ibm-granite/granite-docling-258M")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~1–2 GB VRAM for inference in bfloat16 (typical GPU deployment). CPU inference feasible with MLX (Apple Silicon) or quantized ONNX. Batch inference via vLLM benefits from 8–24 GB VRAM for throughput. No quantization details in card; assume full precision baseline; ONNX/MLX variants may reduce footprint. Verify actual memory on your hardware before production.
Not clearly stated in card. Model is presented as a production inference tool integrated into Docling. No mention of LoRA, QLoRA, or domain-specific fine-tuning. Possible but requires custom training setup and reintegration with Docling pipeline. Not recommended unless addressing significant domain shift; prefer prompt engineering or RAG-based augmentation first.
When to avoid it — and what to weigh
- Real-Time Ultra-Low-Latency Requirements — Even at 258M parameters, inference latency for full documents is non-trivial. Not suitable for sub-100ms response SLAs. Batch processing is the intended use pattern.
- Non-Document Image Understanding — Model is specialized for document layout and structure. Performance on general scene understanding, face recognition, or non-textual imagery is not documented and likely poor.
- Proprietary/Sensitive Data without Infrastructure Control — If you require air-gapped or on-premises inference with strict data residency, model must be self-hosted. HuggingFace or cloud API endpoints may violate compliance.
- Multilingual Documents at Production Scale — Japanese, Arabic, and Chinese support are marked experimental. Expect degradation on non-English documents; test thoroughly before production deployment.
License & commercial use
Apache License 2.0 (permissive OSI-approved license). Allows commercial use, modification, and distribution provided the license and copyright notice are retained.
Apache 2.0 is a permissive open-source license that explicitly permits commercial use without royalty or special permission. You may use Granite Docling 258M in proprietary products, SaaS platforms, and internal enterprise systems. Ensure you: (1) include a copy of the Apache 2.0 license, (2) document material changes if modified, and (3) retain copyright and attribution notices. No liability assumed by IBM Research under the license. Verify compliance with your legal team if integrating into regulated systems (healthcare, finance).
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model processes document images and text; consider data sensitivity. Card does not describe input validation, adversarial robustness, or attack surface. Deployment over untrusted networks should use HTTPS/TLS. Self-hosting is recommended for regulated data (PHI, PII). Model outputs are deterministic (greedy decoding in examples) but could be exploited if user-controlled prompts are allowed; apply input sanitization. No security audit or disclosure policy mentioned; no known CVE data in card.
Alternatives to consider
Docling SmolDocling-256M (ds4sd/SmolDocling-256M-preview)
Slightly smaller predecessor with similar capability; may have different training data or stability profile. Check if available and compare inference cost/quality.
Llama 3.2 Vision (Meta)
Larger (11B) general-purpose VLM; better at open-ended vision tasks but overkill and slower for document-specific extraction. Higher cost, more flexibility.
Qwen VL-Chat or Claude 3.5 Vision (API)
Proprietary closed-source alternatives offering higher accuracy and broader understanding. Higher cost per request but no infrastructure burden; suit low-volume or specialized queries.
Ship granite-docling-258M with senior software developers
Start with the quick-start CLI or Python SDK. For production batch processing, set up vLLM and monitor inference latency on your document corpus. Ensure your infrastructure meets the 1–2 GB baseline VRAM and choose self-hosting for regulated data. Review the Docling project documentation and test on your document types before scaling.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
granite-docling-258M FAQ
Can I use Granite Docling 258M in a commercial product or SaaS service?
What is the minimum VRAM required to run inference?
Does this model support languages other than English?
Can I fine-tune this model on my own documents?
Work with a software development agency
DEV.co helps companies turn open-source tools like granite-docling-258M into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Ready to Deploy Granite Docling?
Start with the quick-start CLI or Python SDK. For production batch processing, set up vLLM and monitor inference latency on your document corpus. Ensure your infrastructure meets the 1–2 GB baseline VRAM and choose self-hosting for regulated data. Review the Docling project documentation and test on your document types before scaling.