mzansilm-125m
MzansiLM is a 125M-parameter decoder-only language model trained on MzansiText, a multilingual corpus covering all eleven official South African languages (Afrikaans, English, Sepedi, Sotho, Swati, Setswana, Tsonga, Venda, Xhosa, Zulu, and Ndebele). It is a research baseline model designed for pretraining, fine-tuning, and evaluation on South African language tasks, not a production chat model.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | anrilombard |
| Parameters | 125M |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 176.5k |
| Likes | 25 |
| Last updated | 2026-05-19 |
| Source | anrilombard/mzansilm-125m |
What mzansilm-125m is
Decoder-only LlamaForCausalLM architecture with 125M parameters, 30 layers, 512 hidden size, 9 attention heads (3 KV heads), 2048 context length, and flash_attention_2 training. Custom BPE tokenizer with 65536 vocabulary. Requires Transformers 4.52.4+ (Transformers 5.x has validation incompatibility with the explicit head_dim=56 configuration). Available in SafeTensors format.
Run mzansilm-125m locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="anrilombard/mzansilm-125m")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~500 MB model weights (fp32) to ~250 MB (fp16/bfloat16). Inference on CPU feasible for small batch sizes; GPU (e.g. NVIDIA T4, RTX 3090) recommended for batch processing. Fine-tuning LoRA on 8 GB VRAM plausible; full fine-tuning ~16–24 GB. Verify precision support and actual peak memory empirically.
No explicit LoRA/QLoRA guidance in card. Standard Transformers fine-tuning supported; LoRA feasible given 125M parameter count (smaller adapters). QLoRA quantization plausible but not mentioned. Full fine-tuning on MzansiText or task-specific datasets is primary intended use case per model card.
When to avoid it — and what to weigh
- Production Chat or Instruction-Following — This is a base language model, not instruction-tuned. It produces raw text continuations, not structured conversations or task-specific responses.
- Non-South African Language Requirements — Model is optimized for South African languages. Performance on other languages, even English or Afrikaans outside the MzansiText training distribution, is not characterized.
- Transformers 5.x Compatibility Required — Current version incompatible with Transformers 5.x due to head_dim validation. If your stack mandates Transformers 5, requires manual patching or waiting for upstream fix.
- High-Throughput Commercial Serving — No evidence of production serving benchmarks, optimization for latency-critical inference, or vLLM/TGI tuning. Research-grade only.
License & commercial use
Apache License 2.0 (OSI-approved permissive license). Allows commercial use, modification, and distribution with attribution and no-liability clause.
Apache 2.0 is a permissive OSI license. Commercial use is permitted provided you include a copy of the license and acknowledge modifications. No gating, no closed-source restrictions. However, intended use is research; no SLA, support, or production guarantees from the developer. Commercial deployment is legally permitted but operationally unsupported.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Unknown |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | Medium |
Standard considerations for open-source transformers: no attestation of input validation, adversarial robustness, or data poisoning defense. Training data (MzansiText corpus) sourced from public/web data; composition and filtering practices not detailed. Recommend security review of downstream fine-tuning data and inference context if handling sensitive South African language content.
Alternatives to consider
Meta Llama 2 / Llama 3 (multilingual variants)
Larger, well-documented, broader language coverage. However, Meta's Llama Community License restricts some commercial use and Llama 2 is now older. Larger memory footprint.
Google mT5 or mBERT
Multilingual encoder-only models with broader pre-training. Better for classification/understanding tasks on South African languages, but not generative text completion.
OpenAI GPT (via API) or other closed-source LLMs
Production-grade, instruction-tuned, supported. No local deployment, higher cost, closed data practices. Not applicable if self-hosted requirement.
Ship mzansilm-125m with senior software developers
Explore self-hosted LLM deployment, custom fine-tuning, or RAG integration for South African language applications. Contact our team to discuss infrastructure, performance tuning, and production readiness.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
mzansilm-125m FAQ
Can I use MzansiLM commercially?
What are the VRAM requirements for inference?
Why does the model fail to load in Transformers 5?
Is MzansiLM instruction-tuned or chat-optimized?
Software developers & web developers for hire
Need help beyond evaluating mzansilm-125m? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to Deploy MzansiLM?
Explore self-hosted LLM deployment, custom fine-tuning, or RAG integration for South African language applications. Contact our team to discuss infrastructure, performance tuning, and production readiness.