cocoindex-io

cocoindex-io/cocoindex

Incremental engine for long horizon agents 🌟 Star if you like it!

View on GitHub Website / Demo Back to listing

11.8kstars 926forks 100+contributors Rust

README.md

<picture>
 <source media="(prefers-color-scheme: dark)" srcset="https://cocoindex.io/blobs/github/homepage/enterprise-hero-dark.svg">
 <source media="(prefers-color-scheme: light)" srcset="https://cocoindex.io/blobs/github/homepage/enterprise-hero-light.svg">
 <img src="https://cocoindex.io/blobs/github/homepage/enterprise-hero-light.svg" alt="Enterprise corpus — codebase, Slack, meeting notes, and documentation — flowing continuously through the CocoIndex incremental sync engine into a production AI agent with always-fresh context. Only the Δ (delta) is reprocessed on every change. Keywords: RAG pipeline, agent memory, enterprise retrieval, AI agent context, live indexing, retrieval-augmented generation, production LLM apps, streaming ETL, incremental ingestion." width="100%" draggable="false"/>
 </picture>

<h1 align="center">Your agents deserve fresh context.</h1>

Star us&nbsp;❤️&nbsp;→&nbsp;<a href="https://github.com/cocoindex-io/cocoindex" title="Star CocoIndex on GitHub — open-source incremental indexing framework for AI agents"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cocoindex.io/blobs/github/homepage/star-btn-small-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cocoindex.io/blobs/github/homepage/star-btn-small-light.svg"><img src="https://cocoindex.io/blobs/github/homepage/star-btn-small-light.svg" alt="Star CocoIndex on GitHub — open-source Python framework for RAG, vector search, and live agent context" height="36" align="absmiddle"/></picture></a> &nbsp;·&nbsp;
 <a href="https://cocoindex.io" title="Visit cocoindex.io — the CocoIndex homepage"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cocoindex.io/blobs/github/homepage/coco-inline-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cocoindex.io/blobs/github/homepage/coco-inline-light.svg"><img src="https://cocoindex.io/blobs/github/homepage/coco-inline-light.svg" alt="cocoindex.io — the CocoIndex homepage: incremental data pipelines for AI agents" height="36" align="absmiddle"/></picture></a> &nbsp;·&nbsp;
 <a href="https://cocoindex.io/docs" title="Read the CocoIndex documentation — guides, quickstart, connectors, transformations, and API reference"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cocoindex.io/blobs/github/homepage/docs-inline-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cocoindex.io/blobs/github/homepage/docs-inline-light.svg"><img src="https://cocoindex.io/blobs/github/homepage/docs-inline-light.svg" alt="CocoIndex documentation — quickstart, connectors, ops, transformations, target stores, RAG and knowledge graph recipes" height="36" align="absmiddle"/></picture></a> &nbsp;·&nbsp;
 <a href="https://discord.com/invite/zpA9S2DR7s" title="Join the CocoIndex Discord — community chat, showcase, release notes, help and support"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cocoindex.io/blobs/github/homepage/discord-inline-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cocoindex.io/blobs/github/homepage/discord-inline-light.svg"><img src="https://cocoindex.io/blobs/github/homepage/discord-inline-light.svg" alt="Join the CocoIndex Discord community — help, showcase, release notes, and live chat with maintainers" height="36" align="absmiddle"/></picture></a>

CocoIndex turns codebases, meeting notes, inboxes, Slack, PDFs, and videos into live, continuously fresh context for your AI agents and LLM apps to reason over effectively — with minimal incremental processing. Get your production AI agent ready in 10 minutes with reliable, continuously fresh data — no stale batches, no context gap


 Incremental · only the delta &nbsp;·&nbsp; Any scale · parallel by default &nbsp;·&nbsp; Declarative · Python, 5 min

[![stars](https://img.shields.io/github/stars/cocoindex-io/cocoindex?style=flat-square&label=stars&color=FB6A76)](https://github.com/cocoindex-io/cocoindex)
[![downloads](https://img.shields.io/pepy/dt/cocoindex?style=flat-square&label=downloads&color=16A534)](https://pepy.tech/projects/cocoindex)
[![pypi](https://img.shields.io/pypi/v/cocoindex?style=flat-square&label=pypi&color=E59A63)](https://pypi.org/project/cocoindex/)
[![python](https://img.shields.io/badge/python-3.10--3.13-3572A5?style=flat-square)](https://www.python.org/)
[![rust](https://img.shields.io/badge/rust-core-db6d28?style=flat-square)](https://www.rust-lang.org/)
[![license](https://img.shields.io/badge/license-Apache--2.0-5B5BD6?style=flat-square)](https://opensource.org/licenses/Apache-2.0)
[![discord](https://img.shields.io/discord/1314801574169673738?style=flat-square&logo=discord&logoColor=white&label=discord&color=5865F2)](https://discord.com/invite/zpA9S2DR7s)

[![CI](https://img.shields.io/github/actions/workflow/status/cocoindex-io/cocoindex/CI.yml?event=push&style=flat-square&label=CI)](https://github.com/cocoindex-io/cocoindex/actions/workflows/CI.yml)
[![release](https://img.shields.io/github/actions/workflow/status/cocoindex-io/cocoindex/release.yml?event=push&style=flat-square&label=release)](https://github.com/cocoindex-io/cocoindex/actions/workflows/release.yml)
[![links](https://img.shields.io/github/actions/workflow/status/cocoindex-io/cocoindex/links.yml?event=push&style=flat-square&label=link%20check)](https://github.com/cocoindex-io/cocoindex/actions/workflows/links.yml)

</div>

[Deutsch](https://readme-i18n.com/cocoindex-io/cocoindex?lang=de) |
[English](https://readme-i18n.com/cocoindex-io/cocoindex?lang=en) |
[Español](https://readme-i18n.com/cocoindex-io/cocoindex?lang=es) |
[français](https://readme-i18n.com/cocoindex-io/cocoindex?lang=fr) |
[日本語](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ja) |
[한국어](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ko) |
[Português](https://readme-i18n.com/cocoindex-io/cocoindex?lang=pt) |
[Русский](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ru) |
[中文](https://readme-i18n.com/cocoindex-io/cocoindex?lang=zh)

</div>

<h2 align="center">Built with CocoIndex ❤️</h2>

<a href="https://cocoindex.io/cocoindex-code" title="CocoIndex-code — flagship MCP server for AI coding agents: AST-aware, incremental, semantic code index. Claude Code and Cursor see your whole repo instantly."><picture><source media="(prefers-color-scheme: dark)" srcset="https://cocoindex.io/blobs/github/homepage/cocoindex-code-hero-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cocoindex.io/blobs/github/homepage/cocoindex-code-hero-light.svg"><img src="https://cocoindex.io/blobs/github/homepage/cocoindex-code-hero-light.svg" alt="CocoIndex-code — flagship MCP server for AI coding agents. AST-aware incremental semantic code index that keeps live call graphs, symbols, vectors, and chunks fresh on every commit. 70% fewer tokens per turn, 80-90% cache hits on re-index, sub-second freshness. Supports Python, TypeScript, Rust, and Go. Features: Δ-only incremental processing, semantic search by meaning (not grep), call graphs and blast-radius analysis, global repo view for duplicates and architecture. Build coding agents (generate, refactor) and code-review agents (catch, approve). One install — Claude Code, Cursor, and other MCP-aware agents see your whole repository instantly. Keywords: MCP server, coding agent, code intelligence, AST chunking, semantic code search, call graph, vector embedding, repository context, Claude Code, Cursor, incremental indexing, blast radius." width="100%"/></picture></a>

<a href="examples">See all 20+ examples · updated every week →</a>

<h3 align="center">Get started</h3>

```sh
pip install -U cocoindex
```

Declare *what* should be in your target — CocoIndex keeps it in sync forever, recomputing only the Δ.

```python
import cocoindex as coco
from cocoindex.connectors import localfs, postgres
from cocoindex.ops.text import RecursiveSplitter

@coco.fn(memo=True)                          # ← cached by hash(input) + hash(code)
async def index_file(file, table):
    for chunk in RecursiveSplitter().split(await file.read_text()):
        table.declare_row(text=chunk.text, embedding=embed(chunk.text))

@coco.fn
async def main(src):
    table = await postgres.mount_table_target(PG, table_name="docs")
    table.declare_vector_index(column="embedding")
    await coco.mount_each(index_file, localfs.walk_dir(src).items(), table)

coco.App(coco.AppConfig(name="docs"), main, src="./docs").update_blocking()
```

Run once to backfill. Re-run anytime — only the changed files re-embed.

Building with an AI coding agent? 
 Drop in our <a href="skills/cocoindex/">CocoIndex skill</a> so your agent writes correct v1 code — concepts, APIs, patterns, all in one file. 
 See <a href="https://cocoindex.io/docs/getting_started/ai_coding_agents/">Use with AI coding agents</a> for install steps.

<h2 align="center">React — for data engineering</h2>

<a href="https://cocoindex.io/react-cocoindex">See the React ↔ CocoIndex mental model →</a>

<h2 align="center">Incremental engine for long-horizon agents</h2>

Data transformation for any engineer, designed for AI workloads — 
 with a smart incremental engine for always-fresh, explainable data.

<h2 align="center">Why incremental?</h2>

Your agents are only as good as the data they see. Batch pipelines drift stale. CocoIndex stays live — and only runs the Δ.

<h2 align="center">What can you build?</h2>

<a href="examples" title="Browse all 20+ CocoIndex examples on GitHub — code, PDF, HN, knowledge graph, podcast, CSV-to-Kafka, image, and more">See all 20+ examples · updated every week →</a>

Working starters from <a href="examples">the examples tree</a> — clone, plug your source, ship.

Building something with CocoIndex? We want to see it. Tag <a href="https://x.com/cocoindex_io" title="Tag @cocoindex_io on X to showcase your CocoIndex project">@cocoindex_io</a> on X or drop a link in <a href="https://discord.com/invite/zpA9S2DR7s" title="Share your project in the CocoIndex Discord #showcase channel">#showcase</a> on Discord. We'll boost it. 🥥

<h2 align="center">Community</h2>

We are so excited to meet you. 
 Every typo fix, new connector, doc tweak, or full-on rewrite makes CocoIndex better. 
 Come hang out — big PRs and small ones, both welcome.

📝 <a href="https://cocoindex.io/docs/contributing/guide">Read the contributing guide</a> &nbsp;·&nbsp;
 🐛 <a href="https://github.com/cocoindex-io/cocoindex/labels/good%20first%20issue">good first issues</a> &nbsp;·&nbsp;
 💬 <a href="https://discord.com/invite/zpA9S2DR7s">Say hi on Discord</a>

<h2 align="center">CocoIndex Enterprise</h2>

<h3 align="center">Large corpus — built for enterprise scale.</h3>

Incremental compute is the only way to keep large corpora fresh without re-embedding them every cycle. 
 CocoIndex scales from a single repo to petabyte-scale stores — parallel by default, delta-only by design.

<h3 align="center">Process once. Reconcile forever.</h3>

When a source changes, CocoIndex identifies the affected records, propagates the change 
 across joins and lookups, updates the target, and retires stale rows — 
 without touching anything that didn't change.

<h3 align="center">Built on a Rust engine.</h3>

The core is Rust — production-grade from day zero. 
 Parallel chunking, zero-copy transforms where possible, and failure isolation 
 so one bad record doesn't stall the flow.

Repository info

Default branch: main
Latest version: v1.1.1
Created: 04/03/2025
Last activity: 05/11/2026
License: Apache-2.0
Open issues: 99
Watchers: 45

Contributors

georgeh0

1.1k contributions

badmonster0

715 contributions

dependabot[bot]

102 contributions

lemorage

32 contributions

Haleshot

30 contributions