SmolLM2-135M-Instruct
SmolLM2-135M-Instruct is a 135-million-parameter instruction-tuned language model from HuggingFace designed for on-device and edge deployment. It handles text generation, summarization, rewriting, and basic function calling with an Apache 2.0 license. The model was trained on 2 trillion tokens and fine-tuned with supervised learning and preference optimization. It is lightweight, free to use, and suitable for resource-constrained environments.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | HuggingFaceTB |
| Parameters | 135M |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 2.4M |
| Likes | 363 |
| Last updated | 2025-09-22 |
| Source | HuggingFaceTB/SmolLM2-135M-Instruct |
What SmolLM2-135M-Instruct is
Transformer decoder architecture (bfloat16 precision) with 134.5M parameters trained on 2T tokens via nanotron framework. Instruction version created through SFT on curated datasets and DPO using UltraFeedback. Supports transformers, ONNX, safetensors, and transformers.js frameworks. No gating; ungated access. Context length not disclosed. Evaluated on zero-shot and few-shot benchmarks (IFEval, MT-Bench, HellaSwag, ARC, MMLU, GSM8K, BBH).
Run SmolLM2-135M-Instruct locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="HuggingFaceTB/SmolLM2-135M-Instruct")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~270 MB model weights (bfloat16, 135M params ≈ 270 MB). Inference on CPU feasible; GPU acceleration recommended for throughput. Single GPU (e.g., 2–4 GB VRAM) sufficient for batch size 1–4. Exact VRAM/latency varies by framework and batch size; verify with your target hardware.
Card provides SFT code via alignment-handbook GitHub repo and references SFT dataset (smol-smoltalk). LoRA/QLoRA feasible given small parameter count; full fine-tuning practical on modest GPUs. DPO example shown in original training. Adaptation cost low; suitable for custom domain tuning with minimal resources.
When to avoid it — and what to weigh
- Complex Reasoning or Math — GSM8K scores (1.4% at 5-shot) indicate poor mathematical reasoning. Avoid for tasks requiring advanced logic, symbolic reasoning, or calculation.
- Multilingual or Non-English Content — Card explicitly states the model 'primarily understand[s] and generate[s] content in English.' Not suitable for non-English languages or multilingual applications.
- Factual Accuracy Requirements — Card warns that 'generated content may not always be factually accurate.' Avoid use cases where hallucination or factual errors have high cost (medical, legal, financial advice without verification).
- High-Precision Function Calling — Function calling support is minimal at 135M (documented mainly for 1.7B). Avoid for applications requiring robust structured output or API integration.
License & commercial use
Apache 2.0 license. Permissive OSI-approved license allowing modification, distribution, and private/commercial use with attribution and liability disclaimers.
Apache 2.0 is a permissive OSI license that explicitly permits commercial use, modification, and redistribution. No gating or proprietary restrictions noted. Attribution required per license terms. Suitable for commercial applications (SaaS, products, internal tools) without additional licensing.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
Card does not disclose security audit, adversarial robustness testing, or content filtering. As with all LLMs, outputs may contain biases or harmful content present in training data (FineWeb-Edu, DCLM, The Stack, curated datasets). No claims of safety features. Recommend: input validation, output monitoring, and threat model assessment for sensitive deployments. On-device deployment reduces network exposure.
Alternatives to consider
TinyLLaMA-1.1B
Slightly larger (1.1B) alternative; similar footprint, broader capability. More established community. Compare on your benchmarks.
Phi-2 (2.7B)
Microsoft model; larger capacity (~2x parameters) with strong reasoning. Higher compute cost; better for tasks requiring accuracy over speed.
Mistral-7B (or quantized variants)
Larger, stronger baseline if edge constraints allow 3–4 GB VRAM. Better instruction following and reasoning; trade-off is deployment scope.
Ship SmolLM2-135M-Instruct with senior software developers
SmolLM2-135M-Instruct offers permissive licensing and minimal hardware footprint for on-device or cost-optimized inference. Start with transformers or transformers.js today. For production deployment, fine-tuning, or integration strategy, contact our AI engineering team.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
SmolLM2-135M-Instruct FAQ
Can I use SmolLM2-135M-Instruct in a commercial product?
What GPU or hardware do I need to run this model?
Is this model factually reliable for mission-critical applications?
Can I fine-tune this model for my domain?
Work with a software development agency
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If SmolLM2-135M-Instruct is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Ready to Deploy a Lightweight LLM?
SmolLM2-135M-Instruct offers permissive licensing and minimal hardware footprint for on-device or cost-optimized inference. Start with transformers or transformers.js today. For production deployment, fine-tuning, or integration strategy, contact our AI engineering team.