{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/11"},{"label":"Information Analysis","url":"https://skillfed.io/packages/category/scientific-engineering-information-analysis/4"}],"enrichment":{"capability":"LMCache is a KV cache management layer that stores, reuses, and monitors transformer key-value caches across LLM inference requests, reducing time-to-first-token and improving throughput for long-context workloads.","skillfed_tags":["llm-inference","cache-optimization","distributed-systems"],"use_cases":["Multi-turn conversational AI: cache KV blocks from earlier turns to skip prefill recomputation on follow-up messages.","RAG and knowledge-augmented generation: reuse cached context embeddings across queries with the same knowledge base.","Agentic workloads: share KV caches across multiple reasoning steps and tool calls within a single session.","Long-context inference: offload KV caches to CPU or remote storage when GPU memory is exhausted, enabling longer prompts.","Batch serving with cache sharing: reduce total prefill cost across requests that share common prompt prefixes.","Cross-engine failover: persist KV caches so inference can resume on a different serving engine without recomputation."],"what_it_does":"LMCache decouples KV cache management from inference engines, storing transformer caches persistently across GPU, CPU, and remote storage tiers. It enables reuse of cached key-value blocks across requests and sessions, reducing redundant prefill computation for multi-turn conversations, agentic workloads, and retrieval-augmented generation (RAG). The layer runs as a standalone daemon, survives engine restarts, and provides observability through Kubernetes metrics, cache hit rates, and token-level diagnostics.\n\nThe package integrates with mainstream open-source serving engines and supports pluggable storage backends (CPU RAM, local SSD, Redis, S3-compatible systems, and others) and transport layers (NVLink, RDMA, TCP). It includes non-prefix KV reuse via CacheBlend for selective recomputation, prefill-decode disaggregation, and a flexible serialization interface for compression and token dropping. Vendor neutrality means you can switch storage or serving engines without losing cached KV data.","worth_installing":"Yes, with conditions. Install if you run LLM inference at scale with multi-turn, RAG, or long-context workloads and want to reduce latency and cost. Active maintenance, permissive license, and strong community adoption are positive signals. Medium install friction (37 dependencies, Linux-only wheels) and alpha status mean you should test integration with your serving engine and storage backend before production deployment. No known vulnerabilities."},"id":"lmcache","links":{"html":"https://skillfed.io/packages/lmcache","md":"https://skillfed.io/packages/lmcache.md","pypi":"https://pypi.org/project/lmcache/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-05","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"lmcache","python_support":"supports_current","summary":"A LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios."},"popularity":{"monthly_downloads":83225,"position":14090,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.5.3"}