compress-pickle

What it is and what it does

compress_pickle is a thin wrapper around Python's standard pickle module that automatically routes serialized data through one of several compression libraries before writing to disk or file-like objects. It exposes the same four core functions as pickle—dump, load, dumps, loads—but adds a compression parameter to select gzip, bz2, lzma, zipfile, or lz4 (optional). The package requires Python >= 3.6 and has no runtime dependencies for the standard compression protocols; lz4 support is available as an optional install.

The package is straightforward in scope: it solves the common problem of reducing pickle file size without requiring custom serialization logic. It works with any picklable Python object and integrates directly into existing pickle workflows by replacing the import and function calls. However, the project has been abandoned since early 2022, meaning no bug fixes, security patches, or compatibility updates will be released.

Use it for:

• Reduce storage footprint when saving large trained machine learning models or data structures to disk.
• Compress Python object snapshots in data pipelines or caching layers where space efficiency matters.
• Archive pickled Python objects in long-term storage or backups with minimal code changes.
• Serialize and compress intermediate results in scientific computing workflows without custom serialization.
• Store pickled state in resource-constrained environments where disk space is limited.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Wraps Python's standard pickle serialization with transparent compression using gzip, bz2, lzma, zipfile, or optionally lz4, reducing storage size for pickled objects.

Yes, if you need straightforward pickle compression and are comfortable with an abandoned package. The code is simple, has no runtime dependencies for standard compression, and zero known vulnerabilities. Install only if your use case is stable and you do not expect future Python or compression library changes to break it. For new projects requiring ongoing maintenance, consider your risk tolerance.

Install

compress-pickle on PyPI

pip install compress-pickle

uv add compress-pickle

poetry add compress-pickle

Installing compress-pickle

Before you install

Low friction install with no runtime dependencies. However, the package is abandoned—last release was 2021-09-21 and last commit 2022-02-16—so expect no maintenance or security updates going forward.

License in practice

MIT license permits commercial and private use with minimal restrictions, making it safe to adopt from a licensing standpoint.

Quickstart

pip install compress_pickle

import compress_pickle

compress_pickle.dump(obj, path, compression='gzip')
obj = compress_pickle.load(path, compression='gzip')

Requires Python >= 3.6. Optional lz4 support requires separate install: pip install compress_pickle[lz4]

Verify before relying

• Whether lz4 optional extra is actively maintained or if its dependency has known issues.
• Compatibility with pickle protocol versions beyond those tested in 2021.
• Performance characteristics compared to alternatives with compression.

Package facts

Evidence: compress_pickle-2.1.0-py3-none-any.whl

Tags

More Utilities packages