SmolLM-135M
SmolLM-135M is a lightweight text-generation model with 135M parameters developed by HuggingFace, designed for CPU and edge deployment. It trains on high-quality educational content (Cosmopedia v2, Python-Edu, FineWeb-Edu) and demonstrates competitive performance relative to its size. The model is permissively licensed (Apache 2.0) and offers multiple precision/quantization options, from full fp32 (12.6 GB) down to int4 (110 MB).
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | HuggingFaceTB |
| Parameters | 135M |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 176.6k |
| Likes | 263 |
| Last updated | 2024-08-01 |
| Source | HuggingFaceTB/SmolLM-135M |
What SmolLM-135M is
SmolLM-135M is a causal language model built on a Llama-style architecture, pretrained on 600B tokens across 600k steps using bfloat16 precision on 64 H100 GPUs. The model uses the cosmo2 tokenizer and is distributed in multiple formats (safetensors, ONNX compatible). No context-length limit is specified in the card. Performance differences are noted between the transformers-converted checkpoint and the original nanotron version, currently under investigation by the team.
Run SmolLM-135M locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="HuggingFaceTB/SmolLM-135M")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: Full precision (fp32) ~12.6 GB VRAM; bfloat16 ~269 MB VRAM; int8 ~163 MB VRAM; int4 ~110 MB VRAM. CPU inference is feasible but slower. GPU recommended for latency-sensitive use. Requires transformers library and compatible PyTorch install; bitsandbytes for int8/int4 quantization. For multi-GPU, accelerate library recommended.
Model card does not explicitly address LoRA, QLoRA, or fine-tuning methodologies. As a standard transformer checkpoint, LoRA fine-tuning is plausible using standard tooling (peft, LLaMA-Factory), but no curated recipes or benchmarks are provided. Full fine-tuning on a single GPU is feasible given model size. Requires experimental validation of downstream task performance given the noted discrepancy between transformers and original nanotron checkpoints.
When to avoid it — and what to weigh
- Production fact-critical or safety-sensitive applications — The model card explicitly states generated content 'may not always be factually accurate, logically consistent, or free from biases.' Avoid high-stakes contexts (medical, legal, financial advice) without additional validation layers.
- Multilingual or non-English use cases — The model 'primarily understand[s] and generate[s] content in English' per the limitations section. Non-English language requirements should use multilingual models or language-specific alternatives.
- Very long-context or document-level reasoning tasks — Context length is not specified in the model card. For tasks requiring deep document understanding or multi-turn retrieval-augmented generation with large knowledge bases, clarify max sequence length before deployment.
- Ultra-low latency or real-time streaming requirements — While small, 135M parameters on CPU may not meet strict sub-100ms inference SLAs. Quantization and batching strategies should be validated against your specific latency targets.
License & commercial use
Licensed under Apache 2.0 (apache-2.0), a permissive OSI-approved open-source license. Permits commercial use, modification, and distribution provided license and copyright notice are retained. No copyleft obligations.
Apache 2.0 is a permissive OSI license that explicitly allows commercial use, redistribution, and integration into proprietary products, provided the license and copyright notice are retained. No additional commercial agreement required. However, model card disclaimers (factual accuracy, bias) should inform your risk assessment for production deployments.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Moderate |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
No explicit security audit or red-team results provided in model card. Standard LLM risks apply: potential for generating biased, harmful, or misleading content; input injection vectors in conversational contexts; resource exhaustion via unbounded generation. Users should implement output validation, rate limiting, and content filtering appropriate to use case. Verify that self-hosted deployments isolate inference endpoints from untrusted networks.
Alternatives to consider
TinyLlama-1.1B
10x model size, improved performance on benchmarks, similar permissive license; trade-off is increased memory footprint and latency. Consider if accuracy is prioritized over edge deployment constraints.
Phi-2 (2.7B, Microsoft)
Similar educational data diet; larger and stronger on reasoning; MIT license (permissive). Higher compute cost; better for conversational and math tasks if inference budget permits.
MobileLLM (Apple, 355M and 1.3B)
Purpose-built for mobile and edge; comparable parameter ranges; permissive license. Limited public pre-release details; less community adoption and tooling maturity than SmolLM.
Ship SmolLM-135M with senior software developers
Start with our self-hosted LLM and custom app services to integrate SmolLM-135M into your infrastructure. Verify context length, fine-tune on your data, and validate factual accuracy for your use case.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
SmolLM-135M FAQ
Can I use SmolLM-135M commercially?
What is the maximum context length?
How does inference latency compare on CPU vs. GPU?
What is the performance discrepancy between transformers and nanotron checkpoints?
Custom software development services
From first prototype to production, DEV.co delivers software development services around tools like SmolLM-135M. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across open-source llms and beyond.
Ready to deploy SmolLM-135M?
Start with our self-hosted LLM and custom app services to integrate SmolLM-135M into your infrastructure. Verify context length, fine-tune on your data, and validate factual accuracy for your use case.