GLM-5-FP8
GLM-5-FP8 is a 753B-parameter open-source LLM from zai-org, released April 2026. It uses sparse mixture-of-experts architecture with FP8 quantization to reduce deployment cost. The model is designed for complex reasoning, coding, agentic tasks, and tool use. MIT-licensed, ungated, with 2M+ downloads. Supports multiple serving frameworks (vLLM, SGLang, KTransformers, Transformers, xLLM). Context length is not specified in the card.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | zai-org |
| Parameters | 753.9B |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 2M |
| Likes | 181 |
| Last updated | 2026-04-05 |
| Source | zai-org/GLM-5-FP8 |
What GLM-5-FP8 is
GLM-5-FP8 scales to 744B total parameters (40B active) trained on 28.5T tokens. Incorporates DeepSeek Sparse Attention (DSA) to reduce memory footprint. FP8 quantization applied. Post-training uses 'slime', an asynchronous RL infrastructure. Supports tool calling (glm47 parser) and reasoning modes (glm45 parser). Multi-modal capacity not mentioned in card. Benchmarks show competitive performance on AIME, IMO, SWE-bench, agentic tasks (Terminal-Bench, CyberGym, BrowseComp), and reasoning (HLE). Multilingual: English and Chinese.
Run GLM-5-FP8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="zai-org/GLM-5-FP8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: FP8 quantization with 744B parameters ≈ 372GB raw model weight (744B × 0.5 bytes/param FP8). Effective working memory (KV cache, activations) for typical serving: ~600–800 GB total GPU VRAM for batch size 1–4. vLLM recipe specifies 8× GPUs (e.g., 8× H100 80GB ≈ 640GB) at 85% utilization. Activation memory and context length not disclosed; 131k token context window noted in evals likely requires > 1TB aggregate memory. Exact specifications require testing.
Card does not discuss LoRA, QLoRA, or instruction-tuning capability. Post-training used proprietary 'slime' RL infrastructure; reproducibility and fine-tuning API unknown. Standard Transformers library support (v0.5.4+) suggests PyTorch/HuggingFace compatibility, but fine-tuning guides are not referenced. Requires review of GitHub repo or model card for supervised fine-tuning and adapter method feasibility.
When to avoid it — and what to weigh
- Real-time, Ultra-low-latency Requirements — 744B parameters (40B active) still requires significant GPU resources. Inference latency likely measured in hundreds of ms even with speculative decoding. Unsuitable for sub-100ms SLAs or high-frequency inference.
- Resource-Constrained or Edge Deployments — FP8 + MoE quantization reduces footprint but still demands 8+ high-end GPUs per the vLLM recipe. Not feasible on CPU, mobile, or constrained edge devices without further quantization (INT4/INT2).
- Multimodal Input (Vision, Audio) is Required — Card describes text-generation pipeline only. No mention of image, video, or audio modalities. Use closed-source alternatives (GPT-5, Claude, Gemini) if multimodal input is mandatory.
- Guaranteed Factual Accuracy or Hallucination-Free Output — Standard LLM behavior; no guardrails or retrieval-augmented generation mentioned. Requires integration with external knowledge bases or fact-checking pipelines for factual-critical applications.
License & commercial use
MIT License. Permissive OSI-approved license permitting commercial use, modification, and redistribution with attribution. No restrictions on deployment model (closed-source, SaaS, on-premise, etc.).
MIT is a permissive OSI license that explicitly allows commercial use. No gating, no academic-only clause, no clause requiring derived works to be open-source. Commercial applications, proprietary integrations, and SaaS deployment are permitted. No support guarantees, SLAs, or commercial licensing provided by zai-org. Commercial users should review the LICENSE file on the GitHub repo and conduct their own compliance review if bundling or modifying.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
Standard LLM security considerations apply: prompt injection, jailbreaking, token leakage, and model extraction are possible. No safety fine-tuning, content filtering, or guardrails mentioned in the card. Inference on untrusted inputs requires containerization and rate-limiting. FP8 quantization does not inherently improve or degrade security. Access to sparse attention implementation (DSA) is open-source, enabling security researchers to audit but also potential adversaries to optimize attacks. Sensitive data should not be stored in long context windows without encryption.
Alternatives to consider
DeepSeek-V3.2
Similar dense-to-sparse scale. Open-source. Comparable on HLE (25.1 vs 30.5), SWE-bench (73.1 vs 77.8), and agentic tasks (BrowseComp 67.6 vs 75.9). May offer different trade-offs in latency or context handling; review benchmarks for your use case.
Claude Opus 4.5 (Proprietary)
Closed-source, commercial API. Stronger on some benchmarks (HLE-with-Tools 43.4*, SWE-bench Verified 80.9, CyberGym 50.6). If maximum reasoning and tool-use performance is critical and cost/latency are secondary, consider Claude. No self-hosting or IP control.
Llama 3.3 70B or 405B (Open-source)
Smaller parameter count, likely lower inference cost and latency. Llama license (LLAMA2 variant) is permissive for most commercial uses. Benchmarks on reasoning and agentic tasks typically lag GLM-5; use if VRAM/cost is the limiting factor and acceptable performance floor is lower.
Ship GLM-5-FP8 with senior software developers
GLM-5-FP8 is production-ready for software engineering automation, long-context reasoning, and multi-step agentic workflows. MIT-licensed and self-hostable. Review hardware requirements, benchmark performance for your domain, and test deployment on your target GPU cluster. Contact our AI engineering team to evaluate fit and design a custom fine-tuning or evaluation pipeline.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
GLM-5-FP8 FAQ
Can I use GLM-5-FP8 commercially?
What GPU hardware do I need to serve GLM-5-FP8?
What is the context length of GLM-5-FP8?
Does GLM-5-FP8 support vision or multimodal input?
Software developers & web developers for hire
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If GLM-5-FP8 is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Deploy GLM-5-FP8 for Complex Reasoning and Agentic AI
GLM-5-FP8 is production-ready for software engineering automation, long-context reasoning, and multi-step agentic workflows. MIT-licensed and self-hostable. Review hardware requirements, benchmark performance for your domain, and test deployment on your target GPU cluster. Contact our AI engineering team to evaluate fit and design a custom fine-tuning or evaluation pipeline.