xglm-564M
XGLM-564M is a 564-million-parameter multilingual open-source language model from Meta, trained on balanced data across 30 languages including English, Chinese, Russian, and many others. It is designed for zero-shot and few-shot text generation tasks across multiple languages. The model is available under the MIT license with no access restrictions, making it suitable for both research and commercial applications. It is not a cutting-edge model by current standards—last updated in January 2023—but remains a practical choice for resource-constrained multilingual deployments.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | |
| Parameters | Unknown |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 105.6k |
| Likes | 54 |
| Last updated | 2023-01-24 |
| Source | facebook/xglm-564M |
What xglm-564M is
XGLM-564M is an autoregressive transformer-based language model with 564M parameters trained on 500 billion sub-tokens across 30 languages with intentional upsampling for low-resource languages. Supported frameworks include PyTorch, TensorFlow, and JAX via HuggingFace Transformers. The model was introduced in the paper 'Few-shot Learning with Multilingual Language Models' (Lin et al., 2021). Training data was weighted toward English (49%), with Russian (9%) and Chinese (8%) as secondary languages; low-resource languages (e.g., Quechua, Haitian Creole) comprise <0.01% each. Context length and exact precision are not stated in the provided data.
Run xglm-564M locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="facebook/xglm-564M")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
Estimated requirements (verify before deployment): fp32 inference ~2.3 GB VRAM; fp16 ~1.2 GB VRAM; int8 quantization ~600 MB VRAM. Suitable for single-GPU inference on modest hardware (e.g., RTX 3060, M1/M2 MacBooks). Batch inference and fine-tuning will require proportionally more memory. Exact precision and context length not specified in provided data.
Standard transformer architecture supports LoRA and QLoRA fine-tuning via HuggingFace Transformers. No explicit fine-tuning guidance provided in model card excerpt. LoRA is recommended to minimize memory overhead for adaptation to specific languages or domains. Fine-tuning on low-resource languages may require careful hyperparameter tuning given their sparse representation in the base training set.
When to avoid it — and what to weigh
- High-Quality English-Only Tasks — Larger, more recent English-focused models (e.g., GPT-2-medium or above) will typically outperform XGLM-564M on English benchmarks. Last training update was January 2023; no knowledge of post-2022 events.
- Complex Reasoning or Instruction-Following — XGLM-564M is a base autoregressive model, not instruction-tuned. It lacks explicit alignment for chatbot-style interaction or complex multi-step reasoning. Consider instruction-tuned alternatives for production chat applications.
- Real-Time, Ultra-Low Latency Inference — While small, generation speed depends on hardware and serving setup. If sub-100ms latency is critical, quantized or distilled alternatives may be preferable.
- Proprietary or Sensitive Domain Data Requiring Gating — Model is freely available and ungated. If you need to restrict model access for regulatory reasons, this public release may not fit compliance requirements.
License & commercial use
Licensed under the MIT License, a permissive open-source license that permits modification, distribution, and commercial use without restriction, subject to preservation of the license header and copyright notice.
Commercial use is permitted under the MIT License. No gating, no special agreements, no proprietary restrictions stated. You may build commercial products with XGLM-564M provided you include the MIT license notice. No warranty or liability guarantees are provided; use at your own risk per standard MIT terms.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Stale |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Good |
| Assessment confidence | Medium |
Open-source, publicly auditable code via HuggingFace and fairseq repositories. No security audit summary provided. As an autoregressive model trained on broad internet data, it may produce unintended or harmful outputs; standard content filtering recommended in production. Model does not appear to have undergone adversarial robustness testing or bias mitigation beyond training data balancing. Review fairseq and Transformers code for supply-chain integrity; no cryptographic signatures mentioned.
Alternatives to consider
mT5 (Google)
Multilingual encoder-decoder, 13 language variants (250M–13B). Encoder-decoder architecture may be preferable for tasks requiring structured output (summarization, translation). More recent (2021 vs. 2022), wider adoption.
GPT-2 + multilingual fine-tuning
English-only base, but smaller and more mature. Use if you only need English generation and want the absolute smallest model footprint or lowest latency.
Bloom-560M (BigScience)
Similar parameter count (560M), trained on 46 languages with explicit multilingual design. Released 2022, more recent training cutoff and larger language coverage than XGLM-564M.
Ship xglm-564M with senior software developers
Start with our private-LLM or custom-apps services to integrate this model into your multilingual product. We'll help you assess hardware, fine-tuning, and compliance requirements. Contact our team for a technical review.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
xglm-564M FAQ
Can I use XGLM-564M in a commercial product without paying fees or obtaining a license?
What hardware do I need to run this model?
Is this model suitable for production use in 2024?
Does XGLM-564M support chat or instruction-following?
Custom software development services
Adopting xglm-564M is usually one piece of a larger software development effort. As a software development agency, DEV.co provides software development services and web development expertise — pairing senior software developers and web developers with your team to design, build, and operate open-source llms software in production.
Ready to Deploy XGLM-564M?
Start with our private-LLM or custom-apps services to integrate this model into your multilingual product. We'll help you assess hardware, fine-tuning, and compliance requirements. Contact our team for a technical review.