---
id: mooncake-transfer-engine
version: "0.3.12.post1"
license: unclear
license_treatment: permissive
maintenance: active
---

# mooncake-transfer-engine — A KVCache-centric Disaggregated Architecture for large-scale LLM inference and training.

License: permissive · Maintenance: active · Downloads: 271.0K/mo

## What it is and what it does

Mooncake Transfer Engine is a low-level data transfer component for disaggregated LLM serving architectures. It enables zero-copy, high-throughput movement of KV cache and model weights across distributed GPU clusters using RDMA and P2P protocols. The engine sits at the core of Mooncake, a platform that separates prefill and decode compute into independent clusters while pooling KV cache across otherwise idle CPU, DRAM, and SSD resources.

The package is typically not used directly but integrated into higher-level serving frameworks—vLLM, SGLang, TensorRT LLM, LMDeploy, and others have built connectors and backends around it. Its runtime dependencies are aiohttp, requests, and msgpack, suggesting async HTTP communication and serialization for remote cache coordination. It is production-grade software powering Kimi, a leading LLM service, and has been deployed at scale on hundreds of GPUs.

Use it for:

- Disaggregated prefill-decode inference: separate prefill and decode clusters while sharing KV cache via Transfer Engine
- Large-scale model training: distribute model weights across thousands of GPUs with fast P2P updates (e.g., 1T-parameter models in ~20 seconds)
- Multimodal embedding caching: cross-instance sharing of Vision Transformer embeddings to avoid redundant GPU computation
- Hierarchical KV cache storage: offload cache to host DRAM or remote storage with intelligent prefetch and eviction
- Encoder-decoder decoupling: separate compute-intensive encoders from language model nodes with zero-copy embedding transfer

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Mooncake Transfer Engine provides high-performance, zero-copy data transfer for distributed KV cache management in large-scale LLM inference and training, using RDMA and P2P mechanisms to move cache and model weights across GPU clusters.

Yes, if you are building or deploying a disaggregated LLM inference or training system at scale and your framework (vLLM, SGLang, TensorRT LLM, LMDeploy) has native Mooncake support. Do not install directly for single-machine use; it is an infrastructure component for multi-node GPU clusters. Active maintenance, no known vulnerabilities, and permissive license make it safe to adopt in production.

## Install

pip install mooncake-transfer-engine
uv add mooncake-transfer-engine
poetry add mooncake-transfer-engine

## Installing mooncake-transfer-engine

Before you install: Medium install friction due to compiled wheels for specific Python versions (3.10–3.13) and Linux platforms (aarch64, x86_64). Active maintenance with recent commits and stable production status; last release 20 days old.

License in practice: Permissive license allows commercial and private use with minimal restrictions, typical for infrastructure projects in the AI serving ecosystem.

Quickstart:

pip install mooncake-transfer-engine
import mooncake_transfer_engine
# Use with vLLM, SGLang, or other LLM serving frameworks via their Mooncake connectors

Requires Linux (manylinux_2_28), Python 3.10 or later, and NVIDIA CUDA environment; designed for GPU cluster deployment, not single-machine use.

Verify before relying:

- Specific RDMA hardware requirements and whether fallback to TCP is supported
- Performance overhead or latency characteristics for different transfer sizes
- Compatibility matrix with specific vLLM, SGLang, and TensorRT LLM versions

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 271.0K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

kv cache transfer llm, rdma distributed inference, disaggregated llm serving, gpu cluster data transfer, distributed kv cache pool, llm-inference, gpu-cluster, kv-cache

[View on SkillFed](https://skillfed.io/packages/mooncake-transfer-engine) · [View on PyPI](https://pypi.org/project/mooncake-transfer-engine/)