SmolLM2-135M
SmolLM2-135M is a 135-million-parameter language model from HuggingFace designed for on-device deployment. It trades raw capability for extreme portability (~724 MB memory footprint), making it practical for mobile, edge, and resource-constrained environments. The model supports text generation, instruction-following, and basic reasoning tasks with improved performance over its predecessor. It is released under Apache 2.0, permitting commercial use without restriction.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | HuggingFaceTB |
| Parameters | 135M |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 1.5M |
| Likes | 212 |
| Last updated | 2025-02-06 |
| Source | HuggingFaceTB/SmolLM2-135M |
What SmolLM2-135M is
SmolLM2-135M is a Transformer decoder (llama-based architecture) pretrained on 2 trillion tokens across FineWeb-Edu, DCLM, The Stack, and curated proprietary datasets. The instruct variant undergoes supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) using UltraFeedback. Inference requires ~724 MB at bfloat16 precision; CPU/GPU/multi-GPU deployment is supported via HuggingFace Transformers. Context length is not documented. The model shows incremental gains over SmolLM1 on standard evals (HellaSwag, ARC, MMLU, CommonSenseQA, IFEval). Training used 64 H100 GPUs with the nanotron framework.
Run SmolLM2-135M locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="HuggingFaceTB/SmolLM2-135M")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~724 MB at bfloat16 precision (per card); full precision (fp32) ~2.9 GB. CPU inference is feasible but slow; GPU recommended for production. Multi-GPU scaling supported via Accelerate. No high-end hardware required; integrates with consumer GPUs (RTX 3060+, M-series, mobile SoCs). Requires review of actual latency/throughput for your target device.
LoRA/QLoRA feasibility is Unknown from the card. SFT dataset (smol-smoltalk) and finetuning code (alignment-handbook) are published, suggesting LoRA is likely viable given the small parameter count. Recommend testing quantized LoRA (4-bit) for on-device fine-tuning. Full fine-tuning is memory-accessible on modest GPUs (16 GB+).
When to avoid it — and what to weigh
- Complex reasoning or multi-step tasks — GSM8K (math) and BBH (big-bench hard) scores are very low (1.4% and 28.2% respectively). Not suitable for complex problem-solving or code generation at production quality.
- Multilingual or non-English use cases — Model explicitly understands primarily English. Generates poor quality in other languages.
- Fact-critical applications — Card states models may not be factually accurate and should not be trusted as definitive sources. Avoid mission-critical retrieval, medical diagnosis, or legal advice without extensive grounding and evaluation.
- Long-context or document-heavy tasks — Context length is not documented. Model architecture is lightweight; assume short context windows. Unsuitable for long-document summarization or few-shot learning with large exemplars.
License & commercial use
Apache License 2.0. Permissive open-source license approved by OSI. Allows commercial use, modification, and distribution with minimal restrictions (attribution and license copy required). No proprietary or restricted terms documented.
Commercial use is explicitly permitted under Apache 2.0. No gating, no commercial license required, no usage restrictions. Fully free for internal product development, SaaS, and enterprise deployment. Verify compliance with end-user data privacy and model outputs (e.g., ensure generated content does not infringe third-party IP). No vendor lock-in.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
No security posture, vulnerability disclosure, or adversarial robustness data provided. Standard LLM risks apply: model may reproduce training-data biases, generate plausible misinformation, or be exploited for prompt injection. Evaluate for your use case before deployment. Self-hosted deployment avoids cloud infrastructure risks. Input validation and output filtering recommended for user-facing applications.
Alternatives to consider
TinyLlama-1.1B
Similar size, broader community, stronger documented performance on reasoning tasks. Trade-off: slightly larger memory footprint.
Phi-2 / Phi-3 (3.8B–3.8B)
Microsoft's compact models; stronger instruction-following and reasoning (MMLU 42–47% vs 29% SmolLM2). Trade-off: larger, still under MIT/research licensing; Phi-2 has commercial use restrictions.
MobileBERT / DistilBERT (encoder-only)
Lighter-weight alternatives for classification, token-level tasks (NER, sentiment). Not suitable for text generation. Trade-off: encoder-only architecture limits generative use cases.
Ship SmolLM2-135M with senior software developers
SmolLM2-135M is production-ready for lightweight, on-device AI. Download the model from HuggingFace Hub, test on your hardware, and benchmark throughput/latency. For instruction-following tasks, use the -Instruct variant. Consider LoRA fine-tuning for domain adaptation. Start with a proof-of-concept on a single edge device before scaling.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
SmolLM2-135M FAQ
Can I use SmolLM2-135M commercially without licensing fees or restrictions?
What GPU/CPU do I need to run this model?
Is this model suitable for production chatbots or customer-facing AI?
What is the context length of SmolLM2-135M?
Software developers & web developers for hire
DEV.co helps companies turn open-source tools like SmolLM2-135M into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Evaluate SmolLM2-135M for Your Use Case
SmolLM2-135M is production-ready for lightweight, on-device AI. Download the model from HuggingFace Hub, test on your hardware, and benchmark throughput/latency. For instruction-following tasks, use the -Instruct variant. Consider LoRA fine-tuning for domain adaptation. Start with a proof-of-concept on a single edge device before scaling.