vllm-ascend
vLLM Ascend is a community-maintained plugin that enables running large language models on Huawei's Ascend NPU hardware using the vLLM inference framework. It supports Transformer-based, Mixture-of-Experts, embedding, and multi-modal models across multiple Ascend hardware generations.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Repository | vllm-project/vllm-ascend |
| Owner | vllm-project |
| Primary language | C++ |
| License | Apache-2.0 — OSI-approved |
| Stars | 2.4k |
| Forks | 1.6k |
| Open issues | 2.3k |
| Latest release | v0.18.0 (2026-04-30) |
| Last updated | 2026-07-08 |
| Source | https://github.com/vllm-project/vllm-ascend |
What vllm-ascend is
A C++ hardware plugin implementing the vLLM hardware-pluggable interface (RFC) for Ascend NPU inference. Requires Python 3.10–3.11, CANN 9.0.0, PyTorch 2.10.0 with torch-npu, and vLLM matching version. Actively developed with multiple maintained branches tracking vLLM releases.
Get the vllm-ascend source
Clone the repository and explore it locally.
git clone https://github.com/vllm-project/vllm-ascend.gitcd vllm-ascend# follow the project's README for install & configurationNeed it deployed, integrated, or customized instead? DEV.co ships production installs.
Best use cases
Implementation considerations
- Verify your Ascend hardware generation (Atlas 800I A2, Atlas A2 Training, Atlas 800I A3, Atlas A3 Training, or Atlas 300I Duo experimental) and CANN version compatibility before deployment.
- Pin exact dependency versions: Python 3.10–3.11, CANN 9.0.0, PyTorch 2.10.0, torch-npu 2.10.0, and matching vLLM version to avoid incompatibilities.
- Use main branch for vLLM main-branch tracking or releases/vX.Y.Z branches for stable point releases; confirm CI commitment status before production use.
- Consult [QuickStart](https://docs.vllm.ai/projects/ascend/en/latest/quick_start.html) and [Installation](https://docs.vllm.ai/projects/ascend/en/latest/installation.html) guides for environment setup and validation.
- Monitor 2,328 open issues; high issue count suggests active but not fully mature ecosystem; review known issues for your use case.
When to avoid it — and what to weigh
- Requires non-Ascend hardware — This plugin is Ascend-specific. If your infrastructure uses NVIDIA, AMD, or Intel accelerators, choose CUDA/ROCm backends or alternative LLM serving frameworks.
- Python 3.12+ or incompatible PyTorch versions — Strict dependency on Python < 3.12 and PyTorch 2.10.0 + torch-npu 2.10.0. Upgrading core dependencies may break compatibility; verify compatibility before adoption.
- Unsupported model architectures — Check [supported models documentation](https://docs.vllm.ai/projects/ascend/en/latest/user_guide/support_matrix/supported_models.html) before proceeding. Custom or experimental architectures may not be supported.
- Non-Linux operating systems — Only Linux is supported. Windows and macOS deployments are not viable with this plugin.
License & commercial use
Apache License 2.0 (Apache-2.0). Permissive OSI-approved license allowing modification, distribution, and private use under standard Apache 2.0 terms (liability limitations, trademark use restrictions, and patent grants apply).
Apache 2.0 is a permissive open-source license generally suitable for commercial use, including proprietary deployments and closed-source derivatives, provided Apache 2.0 terms (license notice, copyright attribution, and patent grant acknowledgment) are honored. No vendor lock-in or commercial licensing restrictions apparent from license ID. However, review your legal/compliance requirements and consult license text for detailed patent and liability terms before large-scale commercial deployment.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | High |
| DEV.co fit | Good |
| Assessment confidence | High |
No explicit security audit or vulnerability disclosure process documented in provided data. Standard open-source risks apply: dependency supply chain (CANN, PyTorch, vLLM), model loading from untrusted sources, and Ascend hardware-specific attack surface. No claims of cryptographic hardening or formal verification. Review CANN and Ascend NPU security advisories independently. Treat as research/production-ready under standard security vetting practices; not a security-hardened product.
Alternatives to consider
vLLM with CUDA/ROCm backends
If you use NVIDIA or AMD hardware instead of Ascend, vLLM's native CUDA and ROCm backends are more mature, better supported, and require no additional plugins.
Ascend native inference frameworks (MindSpore Lite, MindIE)
MindSpore Lite or Huawei's proprietary MindIE may offer tighter Ascend integration and optimizations but lack the vLLM community ecosystem and broader model compatibility.
TensorRT-LLM (NVIDIA), GPTQ, or HF Transformers (CPU/GPU)
If portability across hardware or framework agnostic inference is critical, alternatives like TensorRT-LLM, GPTQ quantization, or Hugging Face Transformers provide broader compatibility, though at potential performance cost on Ascend.
Build on vllm-ascend with DEV.co software developers
Review the QuickStart guide, verify hardware/software prerequisites, and consult user stories for Ascend integration patterns with LLaMA-Factory and other tools.
Talk to DEV.coRelated on DEV.co
Explore the category and the services that help you build with it.
vllm-ascend FAQ
Do I need to use a specific vLLM version?
What Ascend hardware is supported?
Can I use Python 3.12 or different PyTorch versions?
Is commercial use allowed?
Software development & web development with DEV.co
DEV.co helps companies turn open-source tools like vllm-ascend into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your ai frameworks stack.
Ready to deploy LLMs on Ascend?
Review the QuickStart guide, verify hardware/software prerequisites, and consult user stories for Ascend integration patterns with LLaMA-Factory and other tools.