Zamba2-1.2B-instruct
Zamba2-1.2B-Instruct is a 1.2B parameter instruction-tuned LLM combining Mamba2 state-space layers with shared transformer attention blocks. It is licensed under Apache 2.0, ungated, and designed for low-latency inference on modest hardware. The model was fine-tuned on public instruction and chat datasets, and benchmarks show competitive performance against larger models (e.g., outperforming Gemma2-2B on MT-Bench despite being half the size). Deployment requires careful setup of Mamba2 kernel dependencies.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | Zyphra |
| Parameters | 1.2B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 407.9k |
| Likes | 30 |
| Last updated | 2025-02-07 |
| Source | Zyphra/Zamba2-1.2B-instruct |
What Zamba2-1.2B-instruct is
Zamba2-1.2B-Instruct is a hybrid SSM-transformer architecture with 1.2B parameters. It consists of a backbone of Mamba2 layers interleaved with shared attention layers; LoRA projections are applied to transformer blocks for parameter efficiency. Training involved two-stage fine-tuning: SFT on ultrachat_200k and Infinity-Instruct, followed by DPO on ultrafeedback_binarized, orca_dpo_pairs, and OpenHermesPreferences. The model card notes this is a temporary HuggingFace implementation; a standalone PyTorch version exists in the Zyphra GitHub repository. Context length is not specified. Optimized Mamba2 kernels are strongly recommended for inference performance; running without them incurs significant latency and memory penalties.
Run Zamba2-1.2B-instruct locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="Zyphra/Zamba2-1.2B-instruct")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
Estimate: ~2.5–3 GB VRAM (bfloat16 precision) on GPU with Mamba2 kernels installed. Without optimized kernels, memory usage and latency increase substantially (exact overhead unknown). CPU inference is not recommended. Requires CUDA-capable GPU and PyTorch with custom kernel compilation.
The base model uses LoRA projections on transformer blocks, indicating LoRA fine-tuning is feasible. The model was itself fine-tuned via SFT and DPO, and the HuggingFace transformers library is supported (after source install). QLoRA is likely compatible for memory-constrained scenarios, but Mamba2 kernel compatibility with quantization frameworks requires verification. Standard LoRA should be lower-risk.
When to avoid it — and what to weigh
- Long-context or document-heavy workloads — Context length is not disclosed; the model may not be suitable for summarization, long-document Q&A, or tasks requiring extended input sequences.
- Production deployment without SSM kernel expertise — The model card explicitly notes that running without optimized Mamba2 kernels results in significantly higher latency and memory usage. Setup requires `mamba-ssm` compiled from source and close PyTorch version management, increasing operational risk.
- Tasks requiring domain-specific or specialized reasoning — No domain-specific training or reasoning benchmarks (MATH, coding, scientific) are reported. Performance on specialized tasks is unknown.
- Strict reproducibility or production stability requirements — The model card explicitly states this is a temporary HuggingFace implementation and may not be fully compatible with all frameworks. A standalone PyTorch version exists but divergence risk remains.
License & commercial use
Licensed under Apache License 2.0, a permissive OSI-approved license.
Apache 2.0 permits commercial use, modification, and distribution. No restrictions on use case are stated in the license. However, the model card explicitly notes this is a temporary HuggingFace implementation and may not be fully compatible with all frameworks. For production commercial deployment, verify compatibility with your intended serving infrastructure and consider the standalone PyTorch version from Zyphra/Zamba2 for stability. Attribution is required under Apache 2.0.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Good |
| Assessment confidence | High |
Model is ungated and code is public. No security audit, adversarial robustness evaluation, or known vulnerability disclosures are mentioned. Inference via untrusted input carries standard LLM risks (prompt injection, hallucination). The model was trained on public instruction datasets; check dataset licenses and composition for potential bias concerns. Mamba2 kernel compilation from source introduces supply-chain considerations; verify source integrity before deployment.
Alternatives to consider
Gemma2-2B-Instruct
2.7B parameters, mature transformer architecture with broad framework support, comparable MT-Bench score (51.69 vs. 59.53). More widely deployed; trade-off: slightly larger, potentially higher latency.
StableLM-1.6B-Chat
1.6B parameters, traditional transformer, strong compatibility. Lower MT-Bench (49.87) but simpler deployment without custom kernel requirements.
H2O-Danube-1.8B-Chat
1.8B parameters, transformer-based, designed for edge deployment. Slightly larger than Zamba2 but avoids SSM kernel complexity; trade-off: lower instruction-following performance (MT-Bench 49.78).
Ship Zamba2-1.2B-instruct with senior software developers
Evaluate kernel setup requirements, verify context length for your use case, and test inference latency on your target hardware. Consider whether Mamba2 kernel compilation aligns with your DevOps capability. Review the standalone Zyphra/Zamba2 repository for production stability details.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Zamba2-1.2B-instruct FAQ
Can I use this model commercially?
What GPU VRAM do I need?
Do I need to compile Mamba2 kernels, or can I use it out-of-the-box?
What is the maximum context length?
Custom software development services
DEV.co helps companies turn open-source tools like Zamba2-1.2B-instruct into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Ready to Deploy Zamba2-1.2B-Instruct?
Evaluate kernel setup requirements, verify context length for your use case, and test inference latency on your target hardware. Consider whether Mamba2 kernel compilation aligns with your DevOps capability. Review the standalone Zyphra/Zamba2 repository for production stability details.