kafka

What it is and what it does

kafka-python is a pure-Python client for Apache Kafka that lets you produce and consume messages from Kafka brokers without external compiled dependencies. It provides two main APIs: KafkaConsumer for reading messages (with support for consumer groups and dynamic partition assignment on brokers 0.9+) and KafkaProducer for writing messages asynchronously. Both are designed to behave similarly to the official Java client, with Pythonic additions like consumer iterators. The library handles serialization, compression (gzip natively; lz4 and snappy via optional packages), and protocol-level operations.

The package is best suited for Kafka 0.9 and newer brokers, though it remains backward-compatible with 0.8.0. It supports thread-safe producer usage but recommends multiprocessing for consumers. No runtime dependencies means installation is straightforward, and the large download volume and production-stable status indicate it is widely used in production systems.

Use it for:

• Build a consumer application that subscribes to a Kafka topic and processes messages in a consumer group with automatic partition rebalancing
• Implement an asynchronous message producer that sends events to Kafka with optional compression and key-based partitioning
• Set up multi-consumer message processing by manually assigning different partitions to consumer instances using configuration management
• Integrate custom serialization (JSON, msgpack, etc.) for message values and keys in producer and consumer workflows
• Probe and test Kafka broker compatibility by using the protocol layer to check broker versions (0.8.0 to 0.11)

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A pure-Python client library for Apache Kafka that provides high-level producer and consumer APIs designed to mirror the official Java client, with support for consumer groups, message serialization, and compression.

Yes. kafka-python is production-stable, widely adopted (254199 monthly downloads, top 15000 on PyPI), has zero known vulnerabilities, and carries a permissive Apache License 2.0. Installation is frictionless with no runtime dependencies. The only caveat is that the last release was 2017-10-07; verify that this aligns with your support expectations and that the package still meets your target Kafka broker and Python version requirements.

Install

kafka on PyPI

pip install kafka

uv add kafka

poetry add kafka

Installing kafka

Before you install

Low friction: pure Python wheel with no runtime dependencies. Actively maintained with recent commits (2026-08-07) and a large user base (254199 monthly downloads). Production-stable classifier and 5901 repository stars indicate mature, widely-adopted code.

License in practice

Apache License 2.0 is permissive: you can use this library freely in commercial and private projects, modify it, and distribute it, provided you include a copy of the license and state significant changes. No copyleft obligations.

Quickstart

pip install kafka

from kafka import KafkaConsumer
consumer = KafkaConsumer('my_favorite_topic', group_id='my_favorite_group')
for msg in consumer:
    print(msg)

from kafka import KafkaProducer
producer = KafkaProducer(bootstrap_servers='localhost:1234')
producer.send('foobar', b'some_message_bytes')

Requires a running Kafka broker accessible at the specified bootstrap_servers address; optional compression support (lz4, snappy) requires additional system libraries.

Verify before relying

• Whether the package supports modern Python versions beyond 3.6 (classifiers list up to 3.6, but current ecosystem has moved further)
• Current status of optional compression dependencies (lz4, snappy) and whether they remain actively maintained
• Whether the last release date (2017-10-07) reflects true maintenance status or if updates are released through a different channel

Package facts

Evidence: kafka-1.3.5-py2.py3-none-any.whl

Keywords: apache, kafka

Tags

More Python Modules packages