airbyte-cdk

What it is and what it does

Airbyte CDK is a Python framework for building connectors that integrate data sources into Airbyte's data pipeline platform. It abstracts the complexity of implementing Airbyte's protocol by providing base classes, helpers, and a declarative manifest language for defining how to extract, transform, and stream data from APIs, databases, and file systems.

The package supports three main approaches: a low-code declarative manifest (recommended for most connectors), a concurrent CDK for high-throughput scenarios, and a file-based CDK for cloud storage sources. It handles authentication, pagination, rate limiting, schema inference, and protocol serialization, letting developers focus on source-specific logic rather than boilerplate. The framework is actively maintained and supports modern Python versions.

Use it for:

• Build a REST API connector to sync data from a SaaS platform into a data warehouse using the declarative manifest.
• Develop a custom Python connector for a proprietary or complex data source requiring conditional logic and custom transformations.
• Create a file-based connector to extract data from S3, Azure Blob Storage, or Google Cloud Storage.
• Implement a high-throughput connector that reads data concurrently from multiple partitions or slices.
• Extend an existing Airbyte connector with custom authentication, error handling, or retry logic.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A framework for building Airbyte connectors that extract data from HTTP APIs and other sources, supporting both declarative (low-code) and programmatic Python approaches.

Yes, if you are building connectors for the Airbyte ecosystem. The framework is actively maintained, carries no known vulnerabilities, and provides well-documented patterns for both low-code and custom connector development. Install friction is low and Python support is current. Not relevant if you need a standalone data-fetching library outside Airbyte.

Install

airbyte-cdk on PyPI

pip install airbyte-cdk

uv add airbyte-cdk

poetry add airbyte-cdk

Installing airbyte-cdk

Before you install

Low install friction with a pure-Python wheel distribution. Actively maintained with a release 3 days old. Supports current Python versions (3.10–3.13) and carries 41 runtime dependencies, mostly standard data-processing and HTTP libraries.

License in practice

MIT license permits commercial and private use with minimal restrictions; you may use, modify, and distribute this package freely as long as you include the original license notice.

Quickstart

pip install airbyte-cdk

from airbyte_cdk.sources import Source
from airbyte_cdk.sources.declarative import DeclarativeSource

# Use DeclarativeSource for low-code connector or subclass Source for custom logic

Requires Python 3.10 or later; designed for building connectors within the Airbyte ecosystem, not a standalone data-fetching library.

Verify before relying

• Whether the 41 runtime dependencies are all required for basic connector development or only for advanced features.
• Performance characteristics when building connectors that handle high-throughput data sources.
• Maturity level beyond 'Alpha' classification—how stable is the public API across minor releases?

Package facts

Evidence: airbyte_cdk-7.25.1-py3-none-any.whl

Keywords: airbyte, connector-development-kit, cdk

Tags

More Scientific/Engineering packages