GLM-4.7-AWQ
GLM-4.7-AWQ is a 358B-parameter quantized version of the GLM-4.7 large language model, optimized for coding, reasoning, and agentic tasks. It uses 4-bit AWQ quantization to reduce model size to ~181 GiB, enabling deployment on multi-GPU systems. The model supports vLLM serving with specialized MoE (mixture-of-experts) sharding and includes thinking modes for complex reasoning. It is distributed under the MIT license with no access gating.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | QuantTrio |
| Parameters | 358.3B |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 72.9k |
| Likes | 28 |
| Last updated | 2026-05-18 |
| Source | QuantTrio/GLM-4.7-AWQ |
What GLM-4.7-AWQ is
GLM-4.7-AWQ is a quantized derivative of zai-org/GLM-4.7 using AWQ (Activation-aware Weight Quantization) compression to 4-bit precision. The base model uses a mixture-of-experts architecture (GLM4_MOE). The quantized version weighs ~181 GiB on disk. It is optimized for vLLM serving (v0.14.0+) with support for tensor parallelism (TP=8 recommended with --enable-expert-parallel), speculative decoding (MTP method), and tool-calling. Context length and exact VRAM requirements are not specified in the card. Last modified 2025-12-24.
Run GLM-4.7-AWQ locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="QuantTrio/GLM-4.7-AWQ")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~181 GiB model weights (quantized). With activations and vLLM runtime overhead, expect ≥200–250 GiB total memory for batch size 32 at max-model-len=32768. Tensor parallelism TP=8 with expert-parallel sharding requires at least 8 high-end GPUs (e.g., 80GB A100/H100). Requires CUDA 12.8. Exact VRAM per GPU and minimum batch inference VRAM not stated in card.
No fine-tuning or LoRA guidance provided in the model card. The card does not indicate whether the quantized weights support LoRA, QLoRA, or parameter-efficient tuning. Quantized 4-bit models typically require specialized frameworks (e.g., bitsandbytes, AutoGPTQ). Requires review of base model (zai-org/GLM-4.7) for fine-tuning feasibility.
When to avoid it — and what to weigh
- Single-GPU or Resource-Constrained Environments — Model card specifies TP=8 with expert-parallel sharding. Deployment guidance assumes high-end multi-GPU clusters (8× GPUs likely needed for optimal performance). Not suitable for consumer hardware or edge devices.
- Sub-Second Latency Requirements — Mixture-of-experts models introduce routing and load-balancing overhead. While speculative decoding is supported, large context lengths and expert parallelism add latency. Real-time interactive use may be suboptimal.
- Minimal Inference Infrastructure or Simple Chatbot Use — This model is optimized for complex agentic tasks, coding, and reasoning. Simpler conversational or lightweight tasks do not justify the infrastructure complexity and resource cost.
- Unknown Hardware or vLLM Compatibility Issues — Requires CUDA 12.8, vLLM 0.14.0+, and specific environment variables (VLLM_USE_DEEP_GEMM=0, VLLM_USE_FLASHINFER_MOE_FP16=1). Compatibility with older NVIDIA drivers, AMD GPUs, or CPU-only inference is unknown and untested.
License & commercial use
MIT License. Permissive open-source license allowing commercial use, modification, and distribution with attribution. No proprietary restrictions stated.
MIT is a permissive OSI-approved license. Commercial use, deployment, and derivative models are explicitly permitted. However, consult your legal team regarding the base model (zai-org/GLM-4.7) license and any terms of service from the developer (QuantTrio/zai-org) when deploying in production. No commercial support, SLAs, or indemnification are implied by the license alone.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Moderate |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Good |
| Assessment confidence | Medium |
Quantized models using AWQ and MoE can be deployed on-premises, reducing data exposure vs. API-based services. However: (1) CUDA 12.8 and vLLM dependencies introduce supply-chain risk; (2) model card does not discuss input validation, prompt injection mitigations, or adversarial robustness; (3) no mention of model watermarking, backdoor testing, or safety evaluations; (4) remote-code execution flag (--trust-remote-code) is set in the example vLLM config, raising risk if serving untrusted inputs. Conduct threat modeling and safety testing before production deployment.
Alternatives to consider
Llama 3.1 405B (quantized) or Mixtral 8x22B
Open-source MoE models with broader community support, more documentation, and established vLLM/TGI deployments. Trade-off: may have different license terms (Llama Community) and potentially lower coding performance on SWE-bench.
DeepSeek-V3.2 or Gemini 3.0 Pro (proprietary)
Benchmark comparisons in the card show competitive or superior performance (e.g., Gemini 3.0 Pro: 90.1 on MMLU-Pro, 76.2 on SWE-bench). Avoid if private deployment is mandatory, but offers managed inference and support.
zai-org/GLM-4.7 (unquantized base model)
Full-precision model if VRAM and latency permit. Avoids quantization-related accuracy loss but requires significantly more hardware (~700+ GiB). Simpler serving without expert-parallel sharding complexity.
Ship GLM-4.7-AWQ with senior software developers
This model requires significant infrastructure (8+ GPUs, CUDA 12.8, vLLM). Consult our MLOps team to assess your hardware, plan quantization validation, and set up production monitoring for agentic workflows and coding tasks.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
GLM-4.7-AWQ FAQ
Can I commercially deploy GLM-4.7-AWQ in production?
What is the minimum hardware required?
How do I fine-tune this model?
Why does the startup command include --trust-remote-code?
Custom software development services
DEV.co helps companies turn open-source tools like GLM-4.7-AWQ into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source llms stack.
Ready to Deploy GLM-4.7-AWQ?
This model requires significant infrastructure (8+ GPUs, CUDA 12.8, vLLM). Consult our MLOps team to assess your hardware, plan quantization validation, and set up production monitoring for agentic workflows and coding tasks.