sarvam-105b
Sarvam-105B is a 106B-parameter open-source Mixture-of-Experts (MoE) model with 10.3B active parameters per token, designed for complex reasoning, coding, and mathematics. It is optimized for 22 Indian languages and English, licensed under Apache 2.0, and demonstrates competitive performance against major closed-source models on reasoning and agentic benchmarks. The model uses a custom MLA-style attention architecture and requires significant GPU resources for inference.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | sarvamai |
| Parameters | 106B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 57k |
| Likes | 279 |
| Last updated | 2026-03-10 |
| Source | sarvamai/sarvam-105b |
What sarvam-105b is
Sarvam-105B employs a Mixture-of-Experts architecture with 128 experts, top-8 routing, and one shared expert. It uses MLA-style attention with decoupled QK head dimensions (q_head_dim=192, v_head_dim=128), head_dim=576, hidden_size=4096, intermediate_size=16384, and moe_intermediate_size=2048. Context length extrapolation via YaRN scaling (factor 40) supports 128K context. The model card lists evaluation at 65K context and requires trust_remote_code=True during inference. Supports bfloat16 and likely FP8/quantization variants.
Run sarvam-105b locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="sarvamai/sarvam-105b")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~212–425 GB of VRAM for full precision (bfloat16 recommended). For 8× tensor parallel on H100 80GB GPUs: ~26–53 GB per GPU. For inference, examples suggest tp_size=4–8 and dtype=bfloat16. Quantization (e.g., int4, int8) could reduce footprint but is not documented. Requires multi-GPU setup; single-GPU inference not practical.
Model card does not document LoRA, QLoRA, or supervised fine-tuning procedures. Custom attention architecture (MLA) and MoE routing may require framework-specific adapters. Sarvam-30B is offered as a smaller alternative if fine-tuning on consumer hardware is required. Feasibility requires review of Sarvam or Hugging Face community examples.
When to avoid it — and what to weigh
- Latency-Critical Consumer Applications — 106B parameters with MoE routing logic and 10.3B active parameters per token impose non-trivial latency. vLLM support requires source build or hot-patch; native support is pending. Not suitable for real-time chat on single-GPU consumer hardware.
- Edge Deployment or Constrained Environments — Model size and compute requirements (recommend tensor_parallel_size=4–8 from examples) make it impractical for edge, mobile, or on-device inference. Consider Sarvam-30B for smaller footprint, or quantized variants if available.
- Limited Context Window Needs Without Long-Context Features — While 128K context is supported, model is optimized for agentic and reasoning tasks. If your use case requires only short-context conversational AI without reasoning complexity, smaller models may offer better cost-efficiency.
- Proprietary Software Integration Without trust_remote_code Review — Model requires trust_remote_code=True during loading. Requires code audit before deployment in security-sensitive environments or production systems with strict dependency controls.
License & commercial use
Apache License 2.0 (apache-2.0). Permissive OSI-approved license. Source code and model weights available without gating (gated=false).
Apache 2.0 is a permissive license permitting commercial use, modification, and distribution. No commercial restrictions are imposed by the license itself. However, verify: (1) no additional terms or usage policies on Hugging Face model page; (2) attribution requirements in your deployment documentation; (3) liability disclaimers (Apache 2.0 includes broad disclaimers). No evidence of commercial support, SLA, or warranty from Sarvam AI in model card. For production deployment, contact Sarvam AI directly regarding enterprise support, security audits, or model-specific indemnification.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model requires trust_remote_code=True for loading. Audit required before production deployment to verify custom architecture implementation. No explicit security, safety filtering, or adversarial robustness documentation provided. MoE routing and custom attention layers introduce additional code surface; review Sarvam's security practices. No known CVEs or red-team reports in model card. Benchmark evals do not include adversarial or safety metrics. For sensitive applications, conduct independent safety and security review.
Alternatives to consider
Sarvam-30B
Smaller variant (30B parameters) from same developer. Recommended if inference latency or GPU memory is constrained; trade-off in capability for faster deployment and lower overhead.
Qwen3-Next-80B-A3B-Thinking
Comparable 80B model with reasoning capabilities. Direct competitor on AIME, HMMT, and SWE Bench tasks. Requires separate evaluation of license, serving infrastructure, and multilingual coverage.
GLM-4.5-Air
Closed-source alternative with weaker agentic performance (BrowseComp 21.3% vs Sarvam 49.5%) but potentially faster inference or lower total-cost-of-ownership. Evaluation: licensing, data residency, commercial terms.
Ship sarvam-105b with senior software developers
Sarvam-105B combines strong reasoning and agentic performance with comprehensive Indian language support. Self-host on multi-GPU infrastructure using vLLM, SGLang, or Hugging Face transformers. Apache 2.0 license enables commercial deployment. Estimate 4–8 H100 80GB GPUs; contact Sarvam AI for enterprise support and long-context benchmarks.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
sarvam-105b FAQ
Can I use Sarvam-105B commercially without additional licensing?
What GPU hardware do I need to run Sarvam-105B?
Does the model support fine-tuning?
What context length does Sarvam-105B support?
Custom software development services
From first prototype to production, DEV.co delivers software development services around tools like sarvam-105b. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across open-source llms and beyond.
Deploy Sarvam-105B for Your Agentic or Multilingual Use Case
Sarvam-105B combines strong reasoning and agentic performance with comprehensive Indian language support. Self-host on multi-GPU infrastructure using vLLM, SGLang, or Hugging Face transformers. Apache 2.0 license enables commercial deployment. Estimate 4–8 H100 80GB GPUs; contact Sarvam AI for enterprise support and long-context benchmarks.