hdf5plugin

What it is and what it does

hdf5plugin bridges h5py and the HDF5 compression filter ecosystem by registering a suite of compression algorithms (Blosc, Blosc2, BitShuffle, BZip2, FciDecomp, LZ4, Sperr, SZ, SZ3, Zfp, ZStd) so they become available as compression options when creating or reading HDF5 datasets. It ships as pre-built wheels for macOS, Linux, and Windows, or can be compiled from source to enable platform-specific optimizations.

Once imported, hdf5plugin makes these filters transparent to h5py workflows. This is useful for scientific and data-intensive applications where the default gzip compression is insufficient or where specialized algorithms offer better performance or compression ratios.

Use it for:

• Store large scientific datasets with Blosc multithreaded compression for faster I/O in HDF5 files.
• Compress floating-point simulation or measurement data using specialized filters for controlled precision.
• Use fast compression algorithms for low-latency compression in real-time data acquisition pipelines writing to HDF5.
• Enable BitShuffle preprocessing before compression to improve compression ratios on binary data.
• Migrate existing HDF5 files using non-standard filters to systems where hdf5plugin makes those filters readable.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Registers HDF5 compression filters (Blosc, Blosc2, BitShuffle, BZip2, FciDecomp, LZ4, Sperr, SZ, SZ3, Zfp, ZStd) for use with h5py, enabling compressed dataset storage in HDF5 files.

Yes, if you work with HDF5 files and need compression beyond gzip. The package is actively maintained, has no known vulnerabilities, and offers a straightforward import-and-use interface. Verify that the embedded filter licenses align with your project before committing, and be aware that source compilation may be needed to unlock platform-specific performance gains.

Install

hdf5plugin on PyPI

pip install hdf5plugin

uv add hdf5plugin

poetry add hdf5plugin

Installing hdf5plugin

Before you install

Medium install friction due to compiled binary wheels for multiple platforms (macOS, Linux, Windows, ARM). Active maintenance with a release 50 days ago. Single runtime dependency on h5py.

License in practice

License treatment is unclear: the package itself is MIT-licensed, but it embeds multiple HDF5 compression filters under different open-source licenses. Verify the specific licenses of embedded filters match your project's requirements before use.

Quickstart

pip install hdf5plugin

import hdf5plugin
import h5py

with h5py.File('data.h5', 'w') as f:
    f.create_dataset('compressed', data=data, compression='blosc')

Requires h5py to be installed and functional; h5py itself requires HDF5 libraries on the system.

Verify before relying

• Which specific open-source licenses apply to each embedded compression filter and whether they are compatible with your project.
• Whether pre-built wheels include AVX2 and OpenMP optimizations or if source compilation is needed for performance.
• Exact filter IDs or names required to specify each compression algorithm when creating datasets.

Package facts

Evidence: hdf5plugin-7.0.0-py3-none-macosx_10_13_x86_64.whl; hdf5plugin-7.0.0-py3-none-macosx_11_0_arm64.whl; hdf5plugin-7.0.0-py3-none-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; hdf5plugin-7.0.0-py3-none-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl; hdf5plugin-7.0.0-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; hdf5plugin-7.0.0-py3-none-win_amd64.whl

Tags

More Python Modules packages