gpt-oss-20b-MXFP4-Q8
A 20B-parameter quantized version of OpenAI's GPT-OSS-20B model, converted to MLX format for efficient inference on Apple Silicon and compatible hardware. Supports 4-bit quantization (MXFP4-Q8) and is optimized for text generation tasks. Non-gated and Apache 2.0 licensed.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | mlx-community |
| Parameters | 20.9B |
| Context window | Unknown |
| License | apache-2.0 — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 371.5k |
| Likes | 69 |
| Last updated | 2026-03-19 |
| Source | mlx-community/gpt-oss-20b-MXFP4-Q8 |
What gpt-oss-20b-MXFP4-Q8 is
gpt-oss-20b-MXFP4-Q8 is a quantized derivative of openai/gpt-oss-20b, converted to MLX framework (0.27.0) using 4-bit MXFP4-Q8 quantization. Deployed as safetensors format. 20.9B parameters. Designed for MLX inference pipeline with native chat template support. No context length specification provided.
Run gpt-oss-20b-MXFP4-Q8 locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="mlx-community/gpt-oss-20b-MXFP4-Q8")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~10–12 GB VRAM (4-bit quantized 20B params). Optimized for Apple Silicon (M1/M2/M3+). CPU inference feasible on 64 GB+ systems; speed will be slower than GPU. Exact requirements not specified in card; verify against mlx-lm documentation and target hardware.
Base model fine-tuning readiness unknown; card does not specify LoRA/QLoRA compatibility or adaptation instructions. MLX framework may support parameter-efficient tuning, but no evidence in provided data. Requires experimentation or mlx-community consultation.
When to avoid it — and what to weigh
- GPU-only deployments — MLX format is Apple Silicon/CPU-centric; requires port or re-quantization for NVIDIA/AMD GPU clusters. Serving via vLLM or TGI may not be straightforward.
- Maximum latency-sensitive production — 20B parameters quantized to 4-bit is a memory-bandwidth tradeoff; throughput characteristics vs. proprietary inference engines unknown. Benchmark before committing SLA.
- Extensive fine-tuning on novel domains — Base model training data and capabilities not documented; viability of LoRA/QLoRA adaptation on specialized tasks requires experimentation.
- Enterprise SLA without vendor support — mlx-community is community-maintained; no commercial SLA, security patching timeline, or guaranteed uptime assurances.
License & commercial use
Apache 2.0 license. Permissive OSI-approved open-source license permitting commercial use, modification, and redistribution with attribution.
Apache 2.0 permits commercial deployment. However, base model (openai/gpt-oss-20b) terms must be independently verified; ensure OpenAI's original license or terms of use do not impose additional restrictions. Derivative quantization is not a license upgrade—upstream obligations apply. Recommend legal review before production use.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Moderate |
| Documentation | Limited |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Good |
| Assessment confidence | Medium |
No explicit security audit, adversarial robustness assessment, or prompt-injection mitigations documented. Community quantization introduces an additional layer of deviation from base model; potential for unintended behavioral changes during quantization. Model card does not address data privacy, model card does not address input/output validation requirements, or guardrails. Security posture not established; treat as experimental.
Alternatives to consider
Llama 2 (7B/13B, quantized via llama.cpp)
Broader community tooling, more mature quantization pipelines (GGUF), better GPU/CPU portability, and clearer fine-tuning guidance. More adoption and benchmarks.
Mistral 7B (quantized)
Smaller footprint, faster inference, extensive MLX/llama.cpp support, well-documented. Better trade-off for resource-constrained environments.
OpenAI API (gpt-4o, gpt-4-turbo)
Managed service with SLA, security audit trail, and production support. Avoids infrastructure burden if data residency and cost are acceptable.
Ship gpt-oss-20b-MXFP4-Q8 with senior software developers
Test this model on your Apple Silicon hardware using mlx-lm. Benchmark latency and accuracy against your production requirements before committing. Verify base model license terms with legal before commercial use.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
gpt-oss-20b-MXFP4-Q8 FAQ
Can I use this model commercially?
What hardware do I need to run this locally?
Is fine-tuning supported?
How does quantization affect accuracy and latency?
Work with a software development agency
Need help beyond evaluating gpt-oss-20b-MXFP4-Q8? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source llms integrations — and maintain them long-term.
Ready to deploy gpt-oss-20b-MXFP4-Q8?
Test this model on your Apple Silicon hardware using mlx-lm. Benchmark latency and accuracy against your production requirements before committing. Verify base model license terms with legal before commercial use.