dots.ocr
dots.ocr is a 3B-parameter vision-language model designed for multilingual document parsing, combining layout detection and text recognition in a single model. It handles text, tables, formulas, and reading order across English, Chinese, and other languages. The model is MIT-licensed, ungated, and reports state-of-the-art performance on OmniDocBench benchmarks.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | rednote-hilab |
| Parameters | 3B |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | image-text-to-text |
| Gated on HuggingFace | No |
| Downloads | 359k |
| Likes | 1.3k |
| Last updated | 2025-10-31 |
| Source | rednote-hilab/dots.ocr |
What dots.ocr is
dots.ocr is a unified vision-language model (3.04B parameters) built on a 1.7B LLM foundation with visual encoding capabilities. It performs image-to-text document understanding via prompt engineering—users specify layout categories, formatting rules (Markdown, LaTeX, HTML), and reading order constraints in the prompt. The model outputs structured JSON with bounding boxes, element categories, and extracted content. It supports flash-attention-2 for efficient inference and requires trust_remote_code=True for the Qwen VL processor.
Run dots.ocr locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="rednote-hilab/dots.ocr")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~6–8 GB VRAM (bfloat16 inference on single GPU); flash-attention-2 recommended for efficiency. CPU-only inference feasible but slow (minutes per document). Quantization (INT4) can reduce to ~2–3 GB. Original model uses CUDA; requires compatible GPU for practical throughput.
Model card does not document fine-tuning approach (LoRA, QLoRA, full parameters). Requires trust_remote_code=True, suggesting custom layers. Fine-tuning likely requires full training code from rednote-hilab GitHub. Feasibility for domain adaptation (e.g., medical forms) is Unknown without access to training recipes.
When to avoid it — and what to weigh
- Real-time, sub-100ms latency requirements — Model card shows max_new_tokens=24000, suggesting generation can be lengthy. No inference benchmarks provided; 3B+ parameters typically require multi-second latency on CPU or modest GPUs.
- Proprietary or closed-source document formats requiring custom parsing — Model is general-purpose layout parser. Highly specialized formats (legacy EDI, binary protocols, vendor-specific layouts) may require fine-tuning or custom preprocessing not documented.
- Offline or air-gapped environments with strict model size budgets — 3B parameters + vision encoder = ~6–8 GB loaded in bfloat16. Deployments to resource-constrained or embedded systems will require quantization (INT4/INT8) and offline verification.
- Guaranteed security/compliance for regulated data (PII, PHI, legal documents) — No mention of data retention, audit logging, or model behavior under adversarial inputs. MIT license offers no indemnification. Production use in HIPAA/SOC-2 contexts requires risk assessment and redaction pipelines.
License & commercial use
MIT License. Permissive OSI-approved license allowing commercial use, modification, and distribution with attribution and no warranty.
MIT license permits commercial deployment and integration. However: (1) model is research-stage (released July 2025); (2) no SLA, production support, or security audit documented; (3) custom_code requirement may complicate vendor whitelisting; (4) use with proprietary/regulated data requires independent security review and compliance sign-off. Suitable for commercial use at your organization's risk tolerance; not recommended for high-liability use cases (medical, legal, financial) without governance review.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | High |
No explicit security audit or threat model documented. Considerations: (1) trust_remote_code=True executes arbitrary Python during model load—vet source and use in isolated environments; (2) model may memorize training data (common in LLMs); (3) adversarial inputs (corrupted PDFs, injection attacks) behavior Unknown; (4) no guardrails against generating offensive/biased output; (5) deployed via Hugging Face CDN—use verified checksums and pinned versions in production. For sensitive data (PII, financial), implement sandboxing, input validation, and output redaction layers.
Alternatives to consider
Nougat (Meta)
PDF-to-Markdown specialist; excels at academic papers and formula recognition. Smaller footprint (~340M). Lower table TEDS on OmniDocBench (39.9 EN) vs. dots.ocr; no multilingual optimization.
GOT-OCR (Gettysburg)
Compact expert VLM; strong overall scores (0.287 EN edit) and open-source. Narrower scope (no explicit table/formula parsing). Fewer multilingual claims; 6B foundation may be larger than dots.ocr for some use cases.
MinerU (OpenDataLab)
Specialized pipeline tool; smallest edit distance (0.150 EN) on OmniDocBench. Modular design allows task-specific tuning. Trade-off: multi-model complexity vs. dots.ocr's unified architecture; higher latency.
Ship dots.ocr with senior software developers
Test dots.ocr on your document samples via the live demo, then plan a Devco proof-of-concept. Assess fine-tuning needs, compliance requirements, and hardware costs with our team.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
dots.ocr FAQ
Can I use dots.ocr in a commercial product?
What GPU/hardware do I need to run this model?
How does dots.ocr handle languages other than English and Chinese?
Can I fine-tune this model for my specific document type?
Custom software development services
DEV.co helps companies turn open-source tools like dots.ocr into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Ready to integrate document parsing into your pipeline?
Test dots.ocr on your document samples via the live demo, then plan a Devco proof-of-concept. Assess fine-tuning needs, compliance requirements, and hardware costs with our team.