hickle

What it is and what it does

Hickle is a serialization library that stores Python objects—especially numpy arrays—directly to HDF5 files instead of pickle's binary format. It wraps h5py to provide a pickle-like API (dump/load) while leveraging HDF5's hierarchical structure, compression filters, and cross-platform readability. The main appeal is that HDF5 files can be read by most programming languages and scientific tools, not just Python, and data can be transparently compressed using LZF or GZIP.

The package is designed as a drop-in replacement for pickle when your use case is data storage rather than serializing arbitrary Python objects. It excels at handling large numpy arrays and supports custom loaders for user-defined classes (via dedicated loader functions in version 4.x and later). Trade-offs: HDF5 adds a system dependency (libhdf5), and while hickle handles many common types, arbitrary Python object serialization falls back to pickle, defeating some of the cross-language benefit.

Use it for:

• Store large numpy arrays with transparent gzip or LZF compression, reducing file size while keeping data readable by non-Python tools.
• Export scientific data (arrays, structured data) to HDF5 for sharing with MATLAB, R, or C/C++ codebases without format conversion.
• Replace pickle in data pipelines where serialization speed matters and you control the data types being stored.
• Archive simulation or experimental results in a format that preserves metadata and structure across programming languages.
• Dump pandas DataFrames or scipy objects to HDF5 using hickle's built-in loaders for those types.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Hickle serializes Python objects to HDF5 files as a pickle alternative, enabling cross-language data interchange and optional compression of numpy arrays and other data structures.

Yes, if you need cross-language data interchange or HDF5-specific features (compression, chunking, checksums). The low install friction, active maintenance, and lack of known vulnerabilities make it safe to adopt. However, verify the license first—it is marked OSI-approved but the exact terms are not in the metadata. Not worth installing if you only need Python-to-Python serialization (pickle is simpler) or if you cannot add the h5py system dependency.

Install

hickle on PyPI

pip install hickle

uv add hickle

poetry add hickle

Installing hickle

Before you install

Low install friction with only two runtime dependencies (h5py and numpy). The package is actively maintained with recent commits and has been in production use since 2014, supporting current Python versions (3.7–3.11).

License in practice

License treatment is unclear—no SPDX identifier or raw license text is available in the metadata. Verify the actual license before adopting in proprietary or restricted-license contexts.

Quickstart

import hickle as hkl
import numpy as np

array_obj = np.ones(32768, dtype='float32')
hkl.dump(array_obj, 'test.hkl', mode='w', compression='gzip')
array_loaded = hkl.load('test.hkl')

Requires h5py and numpy; h5py itself depends on HDF5 system libraries (libhdf5), which must be installed separately on your system.

Verify before relying

• Exact license identifier and terms—metadata lists 'License :: OSI Approved' but provides no SPDX or raw license text.
• Performance comparison with pickle on typical workloads—description claims faster than pickle but provides no benchmarks.
• Compatibility of HDF5 files across different hickle versions and h5py releases.

Package facts

Evidence: hickle-5.0.3-py3-none-any.whl

Keywords: pickle, hdf5, data storage, data export

Tags

More Python Modules packages