SmolLM2-360M-Instruct
SmolLM2-360M-Instruct is a 361M-parameter open-source language model designed for on-device deployment. It is trained on 4 trillion tokens and fine-tuned for instruction-following tasks including text generation, summarization, rewriting, and function calling. The model is permissively licensed under Apache 2.0, requires no authentication, and can run on modest hardware (CPU or single GPU).
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | HuggingFaceTB |
| Parameters | 362M |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 278.3k |
| Likes | 202 |
| Last updated | 2025-09-22 |
| Source | HuggingFaceTB/SmolLM2-360M-Instruct |
What SmolLM2-360M-Instruct is
Transformer decoder architecture, bfloat16 precision, trained with nanotron framework on 64 H100 GPUs. Post-training includes supervised fine-tuning (SFT) on public and curated datasets, followed by Direct Preference Optimization (DPO) using UltraFeedback. Supports chat templates and is compatible with Hugging Face Transformers, Transformers.js, and text-generation-inference (TGI). Tokenizer and model weights available in safetensors format.
Run SmolLM2-360M-Instruct locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="HuggingFaceTB/SmolLM2-360M-Instruct")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
Estimate: ~1.4 GB VRAM (bfloat16, 361M params). On CPU: feasible for batch size 1–2. GPU: single consumer GPU (RTX 3080 / A10) sufficient. No current batching, throughput, or quantization (int8/int4) performance data provided; verify with your load.
Model card references supervised fine-tuning (SFT) and DPO recipes in the alignment handbook. LoRA/QLoRA is plausible for the 361M size but not explicitly documented. Estimate ~8–16 GB VRAM for QLoRA fine-tuning with modest batch sizes. Code availability via GitHub repository (huggingface/smollm) but hyperparameter tuning guidance requires review.
When to avoid it — and what to weigh
- High-Accuracy Reasoning or Math — GSM8K (5-shot) scores 7.43 vs. Qwen2.5-0.5B at 26.8. Model card explicitly notes generated content may lack logical consistency; not suitable for high-stakes reasoning tasks.
- Multilingual Requirements — Model card states it 'primarily understand[s] and generate[s] content in English.' Non-English workloads require confirmation or alternative selection.
- Fact-Critical Applications — Model card warns content 'may not always be factually accurate.' Unsuitable for knowledge bases, regulatory compliance, or financial/medical guidance without retrieval augmentation and human review.
- Long-Context Tasks — Context length is Unknown. No data on performance beyond typical training window; avoid if >4K token sequences are required without verification.
License & commercial use
Apache 2.0 license. Permissive OSI license allowing commercial use, modification, and distribution under attribution and liability disclaimers. Model is ungated (no authentication required). License restrictions: none stated beyond standard Apache terms (e.g., trademark use, liability).
Apache 2.0 is a permissive open-source license permitting commercial deployment without prior approval. However, model card warnings on factual inaccuracy and hallucination apply: commercial users must validate outputs, especially for customer-facing or high-stakes use. Verify compliance with any downstream data licensing (training data sourced from FineWeb-Edu, DCLM, The Stack, and curated internal datasets; curated datasets not yet released per card). No support SLA documented.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
Standard LLM considerations: model may reproduce training-data biases and generate unsafe or misleading content. No adversarial robustness, prompt injection mitigation, or formal security audit documented. For production use, implement output validation, rate limiting, and input sanitization. Safetensors format mitigates some deserialization risks vs. pickle. No known CVEs or security incidents noted in data provided.
Alternatives to consider
Qwen2.5-0.5B-Instruct
Comparable 500M parameter scale; stronger on GSM8K (26.8 vs. 7.43) and MT-Bench (4.16 vs. 3.66). Slightly weaker on instruction-following (IFEval 31.6 vs. 41.0). Evaluate if math reasoning is critical.
SmolLM2-1.7B-Instruct
Same family, 5× larger; stronger performance across most benchmarks and adds function-calling support. Trade-off: higher VRAM (~5–6 GB) and latency. Choose if accuracy budget allows.
Phi-3-mini (3.8B) or Llama 2 7B
Larger models (~3–7B) with better reasoning and longer context. Consider if on-device constraint is relaxed and edge deployment is not critical; these have stronger commercial backing and documentation.
Ship SmolLM2-360M-Instruct with senior software developers
Download the model from Hugging Face, benchmark it on your hardware, and integrate it into your application using Transformers, TGI, or Transformers.js. Review the model card for limitations on factual accuracy before production use.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
SmolLM2-360M-Instruct FAQ
Can I use this model commercially?
What hardware do I need to run this?
How long is the context window?
Is fine-tuning supported?
Software development & web development with DEV.co
Adopting SmolLM2-360M-Instruct is usually one piece of a larger software development effort. As a software development agency, DEV.co provides software development services and web development expertise — pairing senior software developers and web developers with your team to design, build, and operate open-source llms software in production.
Ready to Deploy SmolLM2 Locally?
Download the model from Hugging Face, benchmark it on your hardware, and integrate it into your application using Transformers, TGI, or Transformers.js. Review the model card for limitations on factual accuracy before production use.