{"categories":[{"label":"Distributed Computing","url":"https://skillfed.io/packages/category/system-distributed-computing"}],"enrichment":{"capability":"Provides Python bindings for a high-performance distributed key-value cache engine designed for LLM and graph reasoning inference workloads, with multi-tier memory pooling and cross-device data access.","skillfed_tags":["llm-inference","distributed-cache","ascend-hardware"],"use_cases":["Accelerate LLM inference by pooling KV cache across multiple nodes with direct cross-device memory access.","Implement vLLM-Ascend backend KV pool storage for large-language-model serving.","Enable prefix-cache acceleration for inference workloads on Ascend hardware.","Build distributed graph reasoning inference systems requiring high-bandwidth, low-latency KV storage.","Manage multi-tier memory (HBM/DDR/SSD) for inference workloads with automatic hot/cold data migration."],"what_it_does":"memcache-hybrid is a Python wrapper around a distributed key-value cache engine built for LLM and graph reasoning inference acceleration. It exposes object-oriented APIs for batch and non-batch put/get/exist/remove operations on KV blocks, backed by a multi-tier memory pool spanning HBM, DDR, and SSD with automatic data migration between tiers. The package depends on memfabric_hybrid for low-latency, high-bandwidth cross-device and cross-machine data access using hardware-specific paths (RDMA, SDMA, shared memory) on Ascend and Kunpeng processors.\n\nThe system is architected around two core components: MetaService, which manages cluster-wide memory allocation and node membership, and LocalService, which provides memory regions and acts as the client-side library. MetaService supports both single-node and Kubernetes-based HA deployments. The package is extremely new\u2014first released in February 2026\u2014and targets inference workloads where KV cache pooling and efficient multi-node memory access are critical bottlenecks.","worth_installing":"Yes, with conditions. Install if you are deploying LLM or graph reasoning inference on Ascend or Kunpeng hardware and need distributed KV cache pooling with multi-tier memory support. Do not install if: (1) your organization cannot adopt Mulan PSL v2 licensing, (2) you require production-grade stability (the package is 6 months old with no long-term track record), (3) you lack the infrastructure to deploy and manage MetaService and LocalService components, or (4) you are not targeting Ascend/Kunpeng hardware where the performance benefits are realized."},"id":"memcache-hybrid","links":{"html":"https://skillfed.io/packages/memcache-hybrid","md":"https://skillfed.io/packages/memcache-hybrid.md","pypi":"https://pypi.org/project/memcache-hybrid/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-11","license_spdx":null,"license_treatment":"unclear","name":"memcache-hybrid","python_support":"supports_current","summary":"Python bindings for memcache_hybrid implemented with pybind11"},"popularity":{"monthly_downloads":711014,"position":5258,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"1.2.0"}