Phi-3-medium-128k-instruct
Phi-3-Medium-128K-Instruct is a 14 billion parameter open-source language model from Microsoft, optimized for memory-constrained and latency-sensitive deployments. It supports 128K token context length, excels at code and math reasoning, and is available under the MIT license with no access restrictions. Suitable for enterprise self-hosted deployments and custom AI applications requiring strong inference efficiency.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Developer | microsoft |
| Parameters | 14B |
| Context window | Unknown |
| License | mit — OSI-approved |
| Modality / task | text-generation |
| Gated on HuggingFace | No |
| Downloads | 42.9k |
| Likes | 389 |
| Last updated | 2025-12-10 |
| Source | microsoft/Phi-3-medium-128k-instruct |
What Phi-3-medium-128k-instruct is
A 13.96B parameter instruction-tuned transformer model trained on synthetic and filtered public data, incorporating supervised fine-tuning and direct preference optimization. Supports 128K context window and 32,064 token vocabulary. Requires trust_remote_code=True in transformers v4.40.2+. Available in HuggingFace, ONNX, and format variants. Last updated December 10, 2025.
Run Phi-3-medium-128k-instruct locally
Load the open weights with 🤗 Transformers and generate — the same model, self-hosted.
from transformers import pipelinepipe = pipeline("text-generation", model="microsoft/Phi-3-medium-128k-instruct")out = pipe("Explain retrieval-augmented generation in one sentence.", max_new_tokens=128)print(out[0]["generated_text"])Swap in vLLM or Ollama for production-grade serving. DEV.co can stand up the inference stack.
How you'd run it
A typical self-hosted path — open weights, an inference server, your application.
DEV.co builds each layer — from GPU infrastructure to the application.
Best use cases
Running & fine-tuning it
ESTIMATE: ~28 GB VRAM for full precision (FP32/FP16); ~14-16 GB with bfloat16 or 8-bit quantization on single GPU (A100 40GB, H100, or dual A6000/L40S). Inference optimizable with ONNX or quantization tools. Exact VRAM depends on batch size and context length usage—verify with target hardware.
Model card indicates tokenizer supports extended vocabulary up to 32,064 tokens via placeholder tokens. LoRA/QLoRA feasible for instruction tuning or domain adaptation. Card does not detail fine-tuning frameworks or PEFT integration—requires experimental validation. Recommend starting with QLoRA for resource-constrained environments.
When to avoid it — and what to weigh
- Non-English primary workloads — Card states non-English languages experience degraded performance due to English-dominant training. Not suitable for multilingual production systems where non-English quality is critical.
- Real-time ultra-low latency requirements (<50ms) — 14B model requires substantial inference time even on high-end hardware. Consider smaller models (Phi-3-mini) if sub-50ms latency is mandatory.
- High-risk safety-critical applications without validation — Card notes model limitations remain despite safety post-training, including potential stereotypes and representation harms. Requires adversarial testing and mitigation before deployment in sensitive contexts.
- Specialized vertical domains without fine-tuning — Model trained on general-purpose data. Legal, medical, or domain-specific applications require evaluation and custom fine-tuning to meet regulatory/accuracy standards.
License & commercial use
MIT License. Permissive, OSI-approved open-source license permitting commercial use, modification, and redistribution with attribution.
MIT license explicitly permits commercial use. Card states 'intended for broad commercial and research use.' No gating or restrictions noted. Developers must independently ensure compliance with applicable laws (privacy, trade, export controls). Card disclaims liability for downstream harms—conduct internal risk assessment for production use.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
Card does not detail security testing, adversarial robustness, or data poisoning mitigations. Model trained on filtered public data—risk of inherited biases and potentially harmful content from source material. Inference via trust_remote_code=True requires trust in HuggingFace and Microsoft execution environments. For sensitive applications, conduct red-teaming and jailbreak testing before production deployment.
Alternatives to consider
Phi-3-mini-128k-instruct
Same family, smaller (3.8B), faster inference, lower VRAM. Trade-off: reduced reasoning capability on complex code/math tasks.
Mistral-7B-Instruct-v0.3
7B open model, MIT-compatible license, strong code and reasoning benchmarks. Larger than Phi-3-medium but better availability of fine-tuning tooling and community support.
LLaMA 2 (13B) or Code Llama (13B)
Comparable parameter count; LLaMA 2 has broader community adoption and tooling. Code Llama explicitly optimized for code tasks. Trade-off: proprietary license requires review; different tokenization ecosystem.
Ship Phi-3-medium-128k-instruct with senior software developers
Phi-3-Medium offers MIT licensing, strong reasoning, and efficient inference. Evaluate quantization and fine-tuning strategies for your infrastructure. Review responsible AI considerations and conduct red-teaming before production deployment.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
Phi-3-medium-128k-instruct FAQ
Can I use this model commercially?
What GPU(s) do I need to run this locally?
Does this support languages other than English?
How do I fine-tune this for my domain?
Software developers & web developers for hire
From first prototype to production, DEV.co delivers software development services around tools like Phi-3-medium-128k-instruct. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across open-source llms and beyond.
Ready to Deploy a Lightweight, Permissive LLM?
Phi-3-Medium offers MIT licensing, strong reasoning, and efficient inference. Evaluate quantization and fine-tuning strategies for your infrastructure. Review responsible AI considerations and conduct red-teaming before production deployment.