{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/8"},{"label":"Distributed Computing","url":"https://skillfed.io/packages/category/system-distributed-computing/2"}],"enrichment":{"capability":"Mooncake Transfer Engine provides high-performance, zero-copy RDMA-based data transfer for distributed KV cache management in large-scale LLM inference and training systems.","skillfed_tags":["llm-inference","distributed-systems","gpu-acceleration"],"use_cases":["Disaggregated LLM inference on multi-node GPU clusters to separate compute-intensive prefill from decode operations","Distributed KV cache pooling to reduce per-node memory pressure and enable higher request throughput","Zero-copy cross-GPU weight and embedding transfer during large-scale distributed training","Hierarchical KV cache storage with intelligent offloading across device, host, and remote tiers","Multimodal inference with efficient cross-instance sharing of encoder outputs (e.g., Vision Transformer embeddings)"],"what_it_does":"Mooncake Transfer Engine is a component of the Mooncake infrastructure platform designed for large-scale LLM inference and training. It implements a KV cache-centric disaggregated architecture that separates prefill and decode compute clusters while using underutilized CPU, DRAM, and SSD resources to build a distributed KV cache pool. The engine provides zero-copy RDMA-based transfer of KV cache data across GPU clusters, enabling efficient cross-instance sharing of cached key-value states.\n\nThe package is tightly integrated with major LLM serving frameworks including vLLM, SGLang, TensorRT LLM, and LMDeploy. It serves as a backend for distributed KV cache management, hierarchical caching across device/host/remote storage tiers, and disaggregated prefill-decode inference patterns. The transfer engine handles large-scale model updates and multimodal embedding distribution in production training and inference pipelines.","worth_installing":"Yes, if you are deploying large-scale LLM inference or training on multi-node GPU clusters and need efficient distributed KV cache management. The package is production-stable, actively maintained, and integrates with established frameworks (vLLM, SGLang, TensorRT LLM). Requires CUDA 13 and Python 3.10+; not suitable for single-node or CPU-only setups."},"id":"mooncake-transfer-engine-cuda13","links":{"html":"https://skillfed.io/packages/mooncake-transfer-engine-cuda13","md":"https://skillfed.io/packages/mooncake-transfer-engine-cuda13.md","pypi":"https://pypi.org/project/mooncake-transfer-engine-cuda13/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-25","license_spdx":null,"license_treatment":"permissive","name":"mooncake-transfer-engine-cuda13","python_support":"supports_current","summary":"A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training. (CUDA 13 version)"},"popularity":{"monthly_downloads":158000,"position":10738,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.3.12.post1"}