GLM-4.5-Air-FP8
GLM-4.5-Air-FP8 is a 106B-parameter mixture-of-experts (MoE) model with 12B active parameters, quantized to FP8 precision for reduced memory footprint. It supports reasoning, code generation, tool use, and conversational tasks in English and Chinese. MIT-licensed and ungated, it is available for download and commercial use. Inference requires 2× H100 or 1× H200 GPUs for full-featured operation; fine-tuning is supported via LoRA on 4× H100 or equivalent.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | zai-org |
| Parameters | 110.5B |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 37.9k |
| Likes | 80 |
| Last updated | 2025-08-12 |
| Source | zai-org/GLM-4.5-Air-FP8 |
What GLM-4.5-Air-FP8 is
GLM-4.5-Air-FP8 is a hybrid reasoning MoE model released by zai-org under MIT license. Architecture includes MTP (multi-token prediction) layers, speculative decoding support, and dual-mode reasoning (thinking and non-thinking). Trained on 128K context length (stated in system requirements table). FP8 quantization reduces weight and cache size compared to BF16 baseline. Model code is integrated in HuggingFace Transformers, vLLM, and SGLang. Last updated 2025-08-12.
Run GLM-4.5-Air-FP8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="zai-org/GLM-4.5-Air-FP8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
Minimum inference: 2× H100 or 1× H200 GPU with ≥1TB server memory, batch size ≤8, FP8 support native to hardware. Full 128K context: 4× H100 or 2× H200. Fine-tuning LoRA: 4× H100 with 96GB VRAM per GPU (H20 example given). Estimated VRAM per GPU (FP8, batch size 1): ~50–70 GB for inference; higher for context handling.
LoRA fine-tuning is supported via Llama Factory and Swift frameworks. Llama Factory requires 4× H100 with batch size 1 per GPU. Swift offers more flexible scaling (H20 96GB examples: 4 GPUs for LoRA, 32+ for SFT/RL). Full fine-tuning (SFT/RL) requires significantly more GPUs (32–128× H20). QLoRA is not explicitly mentioned; assume LoRA is primary lightweight option.
When to avoid it — and what to weigh
- Extreme latency sensitivity — FP8 quantization and MoE routing add computational overhead. Recommended batch size ≤8; real-time, single-token-per-request use cases may see higher latency than smaller, non-quantized models. Check benchmarks in technical report before committing.
- Limited GPU infrastructure — Minimum 2× H100 or 1× H200 required for full-featured inference; 4× H100 needed for fine-tuning. Not suitable for edge devices, mobile, or single-GPU development environments. Requires 1T+ server memory.
- Strict output determinism — Mixture-of-experts models may exhibit variance in routing decisions, affecting reproducibility. If exact-match token sequences are required, evaluate variation under your use case before production deployment.
- Proprietary dependency concerns — Model weights and reasoning/tool parsers depend on HuggingFace Transformers, vLLM, and SGLang updates. Any breaking changes in those libraries could affect inference. Monitor upstream compatibility regularly.
License & commercial use
MIT license. Explicitly permits commercial use and secondary development according to model card.
MIT is a permissive OSI-approved license that explicitly allows commercial use, modification, and distribution with attribution. No restrictions stated in model card. However, commercial use of any LLM depends on compliance with data usage terms, output liability, and deployment context. Verify with legal counsel regarding your specific commercial application (e.g., SaaS liability, data residency, GDPR).
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model weights and code are publicly available on HuggingFace and GitHub (ungated). Inference frameworks (vLLM, SGLang) are community-maintained; evaluate their security posture independently. FP8 quantization does not introduce known security vulnerabilities, but inference still requires sandboxing and access controls for production use. No security audit or threat model in provided data. Treat as with any large language model: validate outputs, monitor for prompt injection, and isolate inference endpoints.
Alternatives to consider
Llama 3.1 70B (or 405B)
Open-source, MIT-licensed, larger community, but higher VRAM requirements. No native reasoning mode. Better if you prioritize ecosystem size over efficient reasoning.
Qwen2.5 72B
Competitive benchmark performance, Apache 2.0 license, lower quantized VRAM footprint. No MoE or reasoning mode. Better if bilingual support is secondary and you want simpler inference.
Mixtral 8x7B (or 8x22B)
MoE architecture with lower total parameter count, easier to run on smaller GPU clusters. Apache 2.0 license. Lacks reasoning and tool-use specialization; consider if agent features are not critical.
Ship GLM-4.5-Air-FP8 with senior software developers
Start with our infrastructure consulting to size your GPU cluster, validate inference performance on your workload, and set up vLLM or SGLang pipelines. We'll also review commercial use compliance and fine-tuning costs.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
GLM-4.5-Air-FP8 FAQ
Can I use this model commercially without paying for a license?
What hardware do I need to run this model?
How does FP8 quantization affect output quality?
Can I fine-tune this model on my own data?
Software developers & web developers for hire
Need help beyond evaluating GLM-4.5-Air-FP8? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to deploy GLM-4.5-Air-FP8?
Start with our infrastructure consulting to size your GPU cluster, validate inference performance on your workload, and set up vLLM or SGLang pipelines. We'll also review commercial use compliance and fine-tuning costs.