cocoindex

What it is and what it does

CocoIndex is a Python library that builds and maintains live indexes of source data—codebases, documents, PDFs, Slack messages, and videos—for AI agents and LLM applications. Rather than re-processing entire datasets on each update, it uses an incremental engine to identify and recompute only the changed portions (the delta), keeping derived indexes fresh without the overhead of full re-embedding or re-indexing cycles. The library is built on a Rust core for production-grade performance and includes connectors for common sources (local filesystems, databases) and targets (PostgreSQL, vector stores).

You declare your transformation logic in Python using decorators and async functions, then CocoIndex handles the incremental scheduling, change propagation, and state management. It's designed for data engineering workflows in AI—particularly RAG pipelines and agent context management—where stale or incomplete data undermines agent reasoning. The package supports modern Python (3.11–3.14), runs on macOS, Linux, and Windows, and carries no known security vulnerabilities.

Use it for:

• Keep a RAG vector store continuously fresh as source documents change, re-embedding only modified or new chunks.
• Maintain a live codebase index for AI coding agents, updating only files that have changed since the last run.
• Build an incremental ETL pipeline that transforms and loads data into a data warehouse, processing only deltas.
• Index meeting notes and Slack messages in real-time for agent context without re-processing the entire conversation history.
• Reconcile changes across joined or looked-up records in a data pipeline without touching unchanged rows.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

CocoIndex maintains a live, incrementally-updated index of codebases, documents, and other sources for AI agents and LLM applications, recomputing only the changed portions rather than re-processing everything.

Yes, if you are building AI agents or LLM applications that need fresh context and want to avoid the cost and latency of full re-indexing on every update. The incremental engine, Rust-backed performance, and Apache 2.0 license make it production-ready. Install friction is moderate due to dependencies, but wheels are widely available. No security vulnerabilities are known. Not necessary if your data is small, infrequently updated, or you don't need real-time freshness.

Install

cocoindex on PyPI

pip install cocoindex

uv add cocoindex

poetry add cocoindex

Installing cocoindex

Before you install

Medium install friction due to 8 runtime dependencies including numpy and psutil, but wheels are available for modern Python versions (3.11–3.14) across macOS, Linux, and Windows. Package is actively maintained with a release just 2 days old.

License in practice

Licensed under Apache 2.0 (permissive), which allows commercial use, modification, and distribution with minimal restrictions—suitable for both open-source and proprietary projects.

Quickstart

pip install cocoindex

import cocoindex as coco
from cocoindex.connectors import localfs, postgres
from cocoindex.ops.text import RecursiveSplitter

@coco.fn(memo=True)
async def index_file(file, table):
    for chunk in RecursiveSplitter().split(await file.read_text()):
        table.declare_row(text=chunk.text)

@coco.fn
async def main(src):
    table = await postgres.mount_table_target(PG, table_name="docs")
    await coco.mount_each(index_file, localfs.walk_dir(src).items(), table)

coco.App(coco.AppConfig(name="docs"), main, src="./docs").update_blocking()

Requires Python 3.11 or later; async/await syntax required; depends on external connectors (e.g., PostgreSQL) for target storage.

Verify before relying

• Whether the Rust engine and incremental computation actually reduce re-embedding cost in practice for large corpora.
• Performance characteristics and scalability limits when processing petabyte-scale stores.
• Availability and maturity of connectors beyond localfs and postgres.
• Whether the package is suitable for real-time streaming or primarily batch-oriented workflows.

Package facts

Evidence: cocoindex-1.0.20-cp311-abi3-macosx_10_12_x86_64.whl; cocoindex-1.0.20-cp311-abi3-macosx_11_0_arm64.whl; cocoindex-1.0.20-cp311-abi3-manylinux_2_28_aarch64.whl; cocoindex-1.0.20-cp311-abi3-manylinux_2_28_x86_64.whl; cocoindex-1.0.20-cp311-abi3-win_amd64.whl; cocoindex-1.0.20-cp314-cp314t-macosx_11_0_arm64.whl; cocoindex-1.0.20-cp314-cp314t-manylinux_2_28_aarch64.whl; cocoindex-1.0.20-cp314-cp314t-manylinux_2_28_x86_64.whl; cocoindex-1.0.20-cp314-cp314t-win_amd64.whl

Keywords: indexing, real-time, incremental, pipeline, search, ai, etl, rag, dataflow, context-engineering

Tags

More Python Modules packages

Further reading