GLM-5.2-NVFP4
GLM-5.2-NVFP4 is a 753B-parameter Mixture-of-Experts language model quantized to 4-bit FP4 precision by NVIDIA. It activates 40B parameters at inference and supports up to 1M context length. Licensed under MIT, it is ready for commercial use. The model excels at reasoning, coding, long-context retrieval, and tool-use scenarios. It requires NVIDIA GPUs (B200/B300 tested) and runs via SGLang or vLLM.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | nvidia |
| Parameters | 381B |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 449.9k |
| Likes | 247 |
| Last updated | 2026-06-26 |
| Source | nvidia/GLM-5.2-NVFP4 |
What GLM-5.2-NVFP4 is
GLM-5.2-NVFP4 is a quantized MoE transformer using sparse attention (IndexShare indexer) and DSA architecture. Quantization targets weights and activations of linear operators in MoE experts; the shared expert remains unquantized. Baseline is GLM-5.2-FP8. Evaluation spans GPQA Diamond (reasoning), SciCode (coding), IFBench (instruction-following), AA-LCR (long-context recall), and τ²-Bench Telecom (agentic tool-use). Accuracy within 0.2–1.0 point of FP8 baseline across benchmarks. Requires transformers ≥5.3.0, tensor parallelism (8 GPUs typical), and NVIDIA CUDA ecosystem.
Run GLM-5.2-NVFP4 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="nvidia/GLM-5.2-NVFP4")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
NVIDIA B200 or B300 GPUs (tested hardware). Tensor parallelism typical at 8 GPUs minimum. Total parameters: 753B (40B activated per token). Quantized to NVFP4 (4-bit); estimated VRAM ~150–200 GB for 8-GPU setup (rough estimate; verify with vendor). Requires CUDA-capable infrastructure and Linux preferred OS. SGLang/vLLM engines handle KV-cache optimizations (fp8_e4m3 noted in vLLM config).
No fine-tuning guidance provided in model card. Quantization to NVFP4 and MoE expert parallelism create barriers to standard LoRA/QLoRA workflows. Feasibility for parameter-efficient tuning unknown; recommend internal evaluation using small-scale ablations. If tuning is required, consider starting from base GLM-5.2-FP8 and re-quantizing, or treat this checkpoint as inference-only.
When to avoid it — and what to weigh
- Latency-Critical Real-Time Systems — MoE and 753B total parameters require significant hardware and multi-GPU tensor parallelism. Inference latency at scale is not disclosed; verify against your SLA before committing.
- Single-GPU or CPU Deployments — Model requires NVIDIA B200/B300 GPUs with 8+ tensor parallelism. No CPU-only or lightweight edge-device variants provided. Infeasible for on-device or resource-constrained environments.
- Training or Fine-Tuning at Scale — No LoRA/QLoRA guidance provided. Quantization to NVFP4 and MoE architecture complicate gradient updates. Unknown feasibility for parameter-efficient tuning; treat as inference-only unless validated internally.
- Domains Requiring Toxicity/Bias Mitigation — Model card explicitly states training on internet data with toxic language and societal biases; may amplify bias in outputs. Requires post-processing or fine-tuning for safety-critical or bias-sensitive applications.
License & commercial use
MIT License (permissive, OSI-approved). Same license as base GLM-5.2 model. Governs use of model weights and derivatives.
Explicitly stated: 'This model is ready for commercial or non-commercial use.' MIT License permits commercial use, modification, and distribution with attribution. No gating restrictions. Suitable for proprietary or closed-source commercial deployment.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Strong |
| Assessment confidence | High |
Model card does not claim security. Training data origins and filtering undisclosed; potential for embedded vulnerabilities in base model. Trained on internet-sourced data, acknowledges toxic language and biases. No adversarial robustness, jailbreak, or prompt-injection analysis provided. Recommend: (1) isolate inference in sandboxed environment, (2) validate outputs against use-case policy, (3) report security concerns via NVIDIA vendor channel, (4) assume model can generate harmful content under adversarial input.
Alternatives to consider
Llama 3.1 (405B or smaller variants)
Open-source, similar scale reasoning/coding, broader ecosystem support. Llama 2 has commercial restrictions; Llama 3 terms require verification. No MoE; denser compute cost but simpler serving.
Mixtral 8x22B (or Mixtral 8x7B)
MoE-based, smaller footprint, Apache 2.0 license. Weaker on long-context and reasoning benchmarks, but lower deployment cost. Established vLLM/SGLang support.
Qwen2 (or Qwen2-MoE 60B)
Strong reasoning and coding, Apache 2.0 licensed, smaller than GLM-5.2. Requires verification of long-context capabilities; no 1M-token context guarantee.
Ship GLM-5.2-NVFP4 with senior software developers
Verify GPU infrastructure (B200/B300, 8+ parallel), test latency/throughput with SGLang or vLLM, and validate output quality on your reasoning or RAG workloads. Contact NVIDIA for enterprise support or security guidance.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
GLM-5.2-NVFP4 FAQ
Can I use GLM-5.2-NVFP4 in a commercial product?
What GPU hardware do I need?
Can I fine-tune this model?
What is the inference latency and throughput?
Work with a software development agency
DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If GLM-5.2-NVFP4 is part of your open-source llms roadmap, our team can implement, customize, migrate, and maintain it.
Ready to Deploy GLM-5.2-NVFP4?
Verify GPU infrastructure (B200/B300, 8+ parallel), test latency/throughput with SGLang or vLLM, and validate output quality on your reasoning or RAG workloads. Contact NVIDIA for enterprise support or security guidance.