LMCache
LMCache is an open-source KV cache management layer for LLM inference that stores and reuses cached key-value data across requests to reduce computation and improve response speed. It works with multiple serving engines, hardware platforms, and storage backends, decoupling cache management from the inference process itself.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Repository | LMCache/LMCache |
| Owner | LMCache |
| Primary language | Python |
| License | Apache-2.0 — OSI-approved |
| Stars | 10.1k |
| Forks | 1.5k |
| Open issues | 391 |
| Latest release | v0.5.1 (2026-07-06) |
| Last updated | 2026-07-08 |
| Source | https://github.com/LMCache/LMCache |
What LMCache is
LMCache provides a daemon-based KV cache layer supporting persistent tiered offloading (GPU, CPU RAM, local/remote storage), prefix and non-prefix cache reuse via CacheBlend, PD disaggregation, pluggable SERDE transforms, and observability. It integrates with vLLM, PyTorch, and supports NVIDIA/AMD/ARM hardware with backends including Redis, S3, and custom storage.
Get the LMCache source
Clone the repository and explore it locally.
git clone https://github.com/LMCache/LMCache.gitcd LMCache# follow the project's README for install & configurationNeed it deployed, integrated, or customized instead? DEV.co ships production installs.
Best use cases
Implementation considerations
- Daemon lifecycle and fault tolerance: LMCache runs as a separate process; ensure monitoring, restart policies, and graceful shutdown are configured to avoid cache loss.
- Storage backend selection and tiering: Choose appropriate backends (RAM, SSD, Redis, S3) based on cost, latency, and capacity trade-offs; remote backends introduce network overhead.
- KV cache invalidation and versioning: Define policies for cache TTL, model version changes, and query pattern shifts to prevent stale or incorrect cache hits.
- Integration with serving engine: Confirm vLLM, PyTorch, or other engine version compatibility; custom model modifications (e.g., SERDE transforms) require development and testing.
- Observability and monitoring: Deploy metrics collection (request hit rate, cache lifecycle, performance) to validate ROI and diagnose cache efficiency.
When to avoid it — and what to weigh
- Simple single-turn, stateless inference with no overlapping queries — If requests are isolated and unpredictable, KV cache reuse provides minimal benefit; added daemon overhead may reduce throughput compared to in-process caching.
- Resource-constrained environments (embedded, edge with minimal memory/storage) — LMCache requires a separate daemon process and tiered storage infrastructure; minimal deployments should use in-GPU caching instead.
- Strict latency-sensitive applications where daemon overhead is unacceptable — Any inter-process communication and remote storage access add latency; ultra-low-latency use cases may prefer tight integration with inference engine.
- Immature or custom serving engines without LMCache integration — Requires integration work from serving engine maintainers; custom or proprietary inference stacks may need substantial engineering to adopt.
License & commercial use
Licensed under Apache License 2.0 (Apache-2.0), a permissive OSI-approved license allowing commercial use, modification, and distribution with the requirement to include a copy of the license and note any changes made.
Apache-2.0 permits commercial use and modification. However, evaluate whether the project's evolving architecture and integration dependencies (vLLM, PyTorch, third-party backends) align with your support model and liability framework. Tensormesh provides backing; assess whether that implies commercial support or is purely community-driven.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
LMCache runs as a daemon with access to cached KV data in storage backends. No explicit security audit or CVE history is stated in provided data. Consider: (1) Network isolation of daemon and storage backend from untrusted clients; (2) authentication/encryption for remote storage access (Redis, S3); (3) cache poisoning risk if multiple tenants or models share storage without proper namespacing; (4) review third-party storage backend security posture. Requires security review before sensitive multi-tenant deployment.
Alternatives to consider
vLLM (built-in prefix caching)
vLLM includes prefix caching; lighter-weight if serving engine crashes are acceptable and tiered offloading/multi-engine reuse is not needed. LMCache decouples cache from engine for higher availability.
Ollama or GGML (local CPU inference with persistent caching)
Simpler deployment for single-machine, offline use cases with no remote storage or multi-turn optimization. LMCache targets scalable, persistent, networked inference.
Ray Serve + custom KV cache layer
Ray provides cluster scheduling and actor-based caching; can be hand-tuned for KV reuse. LMCache offers higher-level abstractions and vendor-neutral storage integration out of the box.
Build on LMCache with DEV.co software developers
LMCache is a strong fit for organizations building scalable, multi-turn, or RAG-heavy LLM services. Assess daemon overhead and storage integration effort for your specific workload. Start with a benchmarking POC using docs and community resources.
Talk to DEV.coRelated on DEV.co
Explore the category and the services that help you build with it.
LMCache FAQ
Does LMCache work with my inference engine?
What storage backends does LMCache support?
How much performance improvement should I expect?
Is LMCache production-ready?
Software developers & web developers for hire
From first prototype to production, DEV.co delivers software development services around tools like LMCache. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across ai frameworks and beyond.
Evaluate LMCache for Your Inference Stack
LMCache is a strong fit for organizations building scalable, multi-turn, or RAG-heavy LLM services. Assess daemon overhead and storage integration effort for your specific workload. Start with a benchmarking POC using docs and community resources.