fastparquet

What it is and what it does

fastparquet is a Python implementation of the Apache Parquet columnar file format, designed to integrate with pandas and numpy for big data workflows. It reads Parquet files into pandas DataFrames and writes DataFrames back to Parquet, supporting features like column selection, categorical encoding, compression, and row-group partitioning. The library has been used implicitly by Dask, Pandas, and intake-parquet.

The package is now in maintenance mode: as of March 2026, the project is being retired because pandas 3.0 now depends explicitly on pyarrow, eliminating the original rationale for fastparquet's existence. Continued use is anticipated only for those still on pandas 2.x. Installation requires numpy, pandas, and cramjam; optional compression support includes gzip, snappy, brotli, lz4, and zstandard by default.

Use it for:

• Read Parquet files produced by Spark or Hive into pandas DataFrames for local analysis.
• Write pandas DataFrames to Parquet format for efficient columnar storage and downstream consumption by big data tools.
• Load specific columns or apply categorical encoding when reading large Parquet files to reduce memory footprint.
• Partition data into row groups and apply compression when writing Parquet files for distributed processing.
• Integrate Parquet I/O into existing pandas-based data pipelines without introducing a pyarrow dependency.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

fastparquet reads and writes Apache Parquet files in Python, offering a native implementation that integrates with pandas, numpy, and other data processing libraries.

Yes, if you are on pandas 2.x and need a lightweight Parquet reader/writer without pyarrow. No, if you are adopting pandas 3.0 or later—use pyarrow instead, which pandas now depends on explicitly. The project is being retired and will receive no further development; evaluate your pandas version and long-term maintenance needs before committing to it.

Install

fastparquet on PyPI

pip install fastparquet

uv add fastparquet

poetry add fastparquet

Installing fastparquet

Before you install

Medium install friction due to compiled wheels across multiple platforms and Python versions (3.10–3.14). Active maintenance with recent releases, though the project is being retired as of March 2026 following pandas 3.0 changes and pandas' explicit dependency on pyarrow.

License in practice

Apache License 2.0 is permissive; you may use, modify, and distribute fastparquet freely in commercial and private projects, provided you include a copy of the license and note any changes.

Quickstart

from fastparquet import ParquetFile, write
import pandas as pd

# Read
pf = ParquetFile('myfile.parq')
df = pf.to_pandas()

# Write
write('outfile.parq', df)

Requires numpy and pandas installed; building from source requires a C compiler toolchain and cython >= 0.29.23.

Verify before relying

• Whether the project retirement announced in March 2026 affects long-term support or security patches for pandas 2.x users.
• Performance characteristics compared to pyarrow for specific workloads or file sizes.
• Full list of supported compression algorithms and any optional dependencies beyond those documented.

Package facts

Evidence: fastparquet-2026.5.0-cp310-cp310-macosx_10_9_universal2.whl; fastparquet-2026.5.0-cp310-cp310-macosx_11_0_arm64.whl; fastparquet-2026.5.0-cp310-cp310-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl; fastparquet-2026.5.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl; fastparquet-2026.5.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl; fastparquet-2026.5.0-cp310-cp310-musllinux_1_2_i686.whl; fastparquet-2026.5.0-cp310-cp310-musllinux_1_2_x86_64.whl; fastparquet-2026.5.0-cp310-cp310-win32.whl; fastparquet-2026.5.0-cp310-cp310-win_amd64.whl; fastparquet-2026.5.0-cp311-cp311-macosx_10_9_universal2.whl; fastparquet-2026.5.0-cp311-cp311-macosx_11_0_arm64.whl; fastparquet-2026.5.0-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl; fastparquet-2026.5.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl; fastparquet-2026.5.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl; fastparquet-2026.5.0-cp311-cp311-musllinux_1_2_i686.whl; fastparquet-2026.5.0-cp311-cp311-musllinux_1_2_x86_64.whl; fastparquet-2026.5.0-cp311-cp311-win32.whl; fastparquet-2026.5.0-cp311-cp311-win_amd64.whl; fastparquet-2026.5.0-cp312-cp312-macosx_10_13_universal2.whl; fastparquet-2026.5.0-cp312-cp312-macosx_11_0_arm64.whl

Tags

More Database packages