pandavro

What it is and what it does

Pandavro bridges Apache Avro and pandas, letting you serialize DataFrames to Avro files and deserialize Avro files back into DataFrames. It depends on fastavro for the low-level Avro I/O, pandas for the DataFrame abstraction, and numpy for type handling. The package infers Avro schemas from DataFrame dtypes, mapping numpy and pandas types (booleans, integers, floats, strings, timestamps, and complex types like records and arrays) to their Avro equivalents. All columns are treated as nullable by default, and nested records and arrays are supported.

The package handles most common data types but has known limitations: it does not support Avro enums, maps, or unions; it cannot infer non-nested schemas with DataFrame indexes; and naive datetime columns will not roundtrip correctly without explicit timezone information due to fastavro's design. You can provide a custom schema instead of relying on inference, and you can load nullable pandas datatypes (Int64Dtype, StringDtype, etc.) deterministically by passing na_dtypes=True to read_avro.

Use it for:

• Export pandas DataFrames to Avro format for storage or transmission in data pipelines.
• Load Avro files from external sources (e.g., Kafka, data lakes) into pandas for analysis.
• Convert between pandas and Avro schemas when integrating with systems that use Avro as a standard format.
• Preserve nullable integer and string types when round-tripping DataFrames through Avro serialization.
• Infer Avro schemas automatically from DataFrame structure without manual schema definition.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Reads and writes Apache Avro files to and from pandas DataFrames, handling schema inference and type conversion between Avro and numpy/pandas types.

Yes, if you need to serialize pandas DataFrames to Avro or read Avro files into pandas. Install friction is low and the MIT license is permissive. Maintenance is aging (352 days since last release), but the repository is active and there are no known vulnerabilities. Be aware of the datetime timezone gotcha and the limitations on complex types and schema nesting before committing to it for a large project.

Install

pandavro on PyPI

pip install pandavro

uv add pandavro

poetry add pandavro

Installing pandavro

Before you install

Low install friction with a pure-Python wheel. Maintenance status is aging—last release was 352 days ago—but the repository remains active and unarchived with recent commits.

License in practice

MIT license permits commercial and private use with minimal restrictions; you may use, modify, and distribute pandavro freely provided you include the license notice.

Quickstart

pip install pandavro

import pandavro as pdx
import pandas as pd

df = pd.DataFrame({'col': [1, 2, 3]})
pdx.to_avro('file.avro', df)
df_read = pdx.read_avro('file.avro')

Requires Python >= 3.9.0. Naive datetime columns must not contain timezone information; always use timezone-aware timestamps to avoid roundtrip errors due to fastavro's system timezone interpretation.

Verify before relying

• Whether the 352-day release gap reflects stable maintenance or reduced active development.
• Performance characteristics when handling large DataFrames or complex nested schemas.
• Compatibility with recent pandas and numpy versions beyond the stated support window.

Package facts

Evidence: pandavro-1.9.0-py3-none-any.whl

Tags

More Information Analysis packages