aws-kinesis-agg

What it is and what it does

aws_kinesis_agg provides Python classes to pack multiple Kinesis user records into aggregated records using the Kinesis Aggregated Record Format, and to unpack aggregated records back into individual user records. On the producer side, it offers both iterative and callback-based aggregation modes, letting you batch records efficiently before sending to Kinesis; on the consumer side, it deaggregates records received from Kinesis (including those from the Kinesis Producer Library) into their original form.

The package is designed to reduce transmission costs and improve throughput by combining multiple small records into larger ones that fit within a single PutRecord request. It depends only on protobuf for serialization. The documentation cautions that aggregation is suitable only for low-value, non-critical messages; it is not recommended for sensitive data or scenarios where every message must be guaranteed delivery, especially when using the Kinesis Client Library or AWS Lambda for processing.

Use it for:

• Batch small telemetry or log events into aggregated Kinesis records to reduce API call costs and improve producer throughput.
• Deaggregate records received from Kinesis streams or Lambda events that were created by the Kinesis Producer Library.
• Build a producer application that packs records iteratively, checking byte size or record count to trigger transmission when full.
• Implement callback-based aggregation to automatically send packed records to Kinesis as soon as they reach target size.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Aggregates and deaggregates Kinesis records using the Kinesis Aggregated Record Format, allowing efficient batch transmission of data to Amazon Kinesis.

Yes, if you are building a Kinesis producer and need to optimize throughput and costs through record aggregation, and your data is non-critical or can tolerate occasional loss. No, if you require active maintenance, modern Python version support guarantees, or are processing sensitive data where every message must be reliably delivered. The dormant maintenance status and lack of recent updates mean you should verify compatibility with your protobuf and Python versions before adopting.

Install

aws-kinesis-agg on PyPI

pip install aws-kinesis-agg

uv add aws-kinesis-agg

poetry add aws-kinesis-agg

Installing aws-kinesis-agg

Before you install

Low install friction with a single runtime dependency (protobuf). Maintenance is dormant—last release was 2022-06-10 and last commit 2024-10-04—so expect no active bug fixes or feature updates, though the codebase is stable and marked Production/Stable.

License in practice

Apache-2.0 is permissive; you can use this in commercial and proprietary projects with minimal restrictions, provided you include a copy of the license and note any modifications.

Quickstart

pip install aws_kinesis_agg

import aws_kinesis_agg.aggregator

kinesis_aggregator = aws_kinesis_agg.aggregator.RecordAggregator()
result = kinesis_aggregator.add_user_record('partition_key', b'data', None)
if result:
    pk, ehk, data = result.get_contents()
    # Send to Kinesis

Requires protobuf to be installed (handled automatically by pip); Python 2.7 or 3.6+ support as stated in classifiers, but compatibility with newer versions is unverified.

Verify before relying

• Whether the package remains compatible with current versions of protobuf and modern Python versions beyond 3.6.
• Real-world data loss scenarios mentioned in the documentation and whether they apply to your use case.

Package facts

Evidence: aws_kinesis_agg-1.2.3-py3-none-any.whl

Keywords: aws, kinesis, aggregation, deaggregation, kpl

Tags

More Python Modules packages