GLM-4.7-FP8
GLM-4.7-FP8 is a 358B-parameter open-source language model from zai-org optimized for coding, tool use, and complex reasoning tasks. It is quantized to 8-bit floating point for reduced memory footprint and is available under the MIT license without access restrictions. The model shows competitive performance on coding benchmarks (SWE-bench, LiveCodeBench) and reasoning tasks (HLE, AIME), with claimed improvements in multilingual agentic coding and UI generation.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | zai-org |
| Parameters | 358.5B |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 72.2k |
| Likes | 124 |
| Last updated | 2025-12-23 |
| Source | zai-org/GLM-4.7-FP8 |
What GLM-4.7-FP8 is
GLM-4.7-FP8 is a 358.5B parameter mixture-of-experts (MoE) transformer trained on multilingual data (English, Chinese). It supports interleaved, preserved, and turn-level thinking modes for agentic workflows. The FP8 quantization reduces precision from standard float32/bfloat16 while maintaining usability via inference frameworks (vLLM, SGLang). Context length is not specified. Default inference uses temperature 1.0, top-p 0.95, and up to 131k tokens max output; task-specific settings vary.
Run GLM-4.7-FP8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="zai-org/GLM-4.7-FP8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE (not officially stated): FP8 quantization of 358B parameters requires approximately 180–220 GB VRAM for inference (vs. ~720 GB for bfloat16). Multi-GPU setup (8× H100 or A100 80GB) or tensor parallelism recommended. Single-GPU inference not feasible; batch serving scales with available VRAM.
Model card does not mention LoRA, QLoRA, or fine-tuning procedures. 358B scale suggests full-parameter training is prohibitive for most teams. Feasibility of LoRA depends on internal architecture documentation (not provided). Recommend contacting zai-org or checking GitHub repo for community-contributed fine-tuning recipes.
When to avoid it — and what to weigh
- Latency-critical or real-time inference scenarios — 358B parameters at FP8 still requires substantial VRAM and compute. Preserved thinking mode adds latency overhead for multi-turn workflows. If sub-100ms response times are required, consider smaller models.
- Resource-constrained edge or embedded deployments — FP8 quantization helps but 358B model is not suitable for mobile, IoT, or single-GPU setups without clustering. Quantization to lower precision (int4, int8) would require retraining or additional optimization work.
- Production deployments without vLLM/SGLang validation — Model card states vLLM and SGLang require their nightly/dev branches for support. Transformers library integration at version 4.57.3+ may not be stable. Requires thorough testing in your inference stack before committing to production.
- Specialized vertical domains without evaluation data — Benchmarks focus on coding, reasoning, and tool use. No evaluation provided for legal, medical, financial, or domain-specific fine-tuning. Unclear generalization to proprietary use cases.
License & commercial use
MIT License. MIT is a permissive OSI-approved license allowing commercial use, redistribution, and modification with attribution and liability disclaimer.
MIT license permits commercial use, including proprietary applications and closed-source deployment. No restrictions on commercial services, API endpoints, or monetization. No gating or access controls enforced. Suitable for building commercial products without additional licensing obligations, though users remain responsible for compliance with underlying training data sources and any local regulations.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Good |
| Assessment confidence | High |
No explicit security audit, threat model, or adversarial robustness evaluation provided. Large 358B model trained on multilingual data; typical LLM risks apply (prompt injection, jailbreaking, hallucination, toxic outputs). Preserved thinking mode may increase context window attack surface in multi-turn scenarios. FP8 quantization does not inherently reduce or increase security. Recommend standard hardening: input validation, output filtering, rate limiting, and monitoring if deployed in customer-facing systems. No mention of safety classifiers or content filtering.
Alternatives to consider
DeepSeek-V3.2
Comparable 358B+ parameter model; 73.1% on SWE-bench Verified (vs. GLM-4.7's 73.8%). Better on some reasoning benchmarks (HLE w/ tools: 40.8% vs. 42.8%). Considered a strong regional alternative with different training provenance.
Claude 3.5 Sonnet
77.2% on SWE-bench Verified (higher than GLM-4.7). Closed-source, hosted API only. Smaller effective parameter count, lower latency, battle-tested in production. Trade-off: no local deployment, per-token pricing, data privacy constraints.
Llama 3.1-405B (or smaller quantized variants)
Permissive Llama 2 Community License for commercial use. 405B base model or smaller quantized versions (70B) offer flexibility. Smaller variants suit resource-constrained environments. Ecosystem mature (ollama, llama.cpp support). Coding performance lower on SWE-bench but sufficient for many agentic workflows.
Ship GLM-4.7-FP8 with senior software developers
Explore managed inference options, benchmark on your dataset, or review the GitHub repository for local deployment via vLLM/SGLang. Contact zai-org for production support and custom integrations.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
GLM-4.7-FP8 FAQ
Can I use GLM-4.7-FP8 in a commercial product?
What are the realistic hardware and cost requirements for self-hosting?
What is the context length?
Is this model suitable for my specific domain (legal, medical, etc.)?
Work with a software development agency
Need help beyond evaluating GLM-4.7-FP8? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to Deploy GLM-4.7-FP8?
Explore managed inference options, benchmark on your dataset, or review the GitHub repository for local deployment via vLLM/SGLang. Contact zai-org for production support and custom integrations.