pgzip

What it is and what it does

pgzip replaces Python's built-in gzip module with a multi-threaded alternative that uses block indexing to parallelize compression and decompression. It maintains full compatibility with standard gzip files by leveraging the FEXTRA feature defined in the gzip specification, so compressed files can be read by any standard gzip tool. The package exposes the same API as the standard gzip module (open, compress, decompress), making it a drop-in replacement for I/O-bound compression workloads.

The main trade-off is that only the three core methods (open, compress, decompress) are well-tested; other GzipFile methods like seek() and tell() have not been thoroughly validated. Performance gains depend heavily on available CPU cores and I/O bandwidth. The project is actively maintained but explicitly seeking additional maintainers.

Use it for:

• Compressing large text files or datasets where I/O and CPU parallelism can reduce wall-clock time.
• Batch processing pipelines that need to compress or decompress files concurrently using a gzip-compatible format.
• Replacing standard gzip in data workflows where compression throughput is a bottleneck and standard gzip files must remain readable.
• CLI-based compression tasks via the command-line interface with configurable thread count and compression level.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

pgzip is a multi-threaded gzip implementation that accelerates compression and decompression by parallelizing block-indexed gzip operations, maintaining compatibility with standard gzip files.

Yes, if you need faster gzip compression/decompression and can tolerate that seek/tell are not fully tested. The low install friction, permissive license, active maintenance, and zero security vulnerabilities make it safe to adopt. Not necessary if you rarely compress large files or if your bottleneck is not CPU/I/O bound.

Install

pgzip on PyPI

pip install pgzip

uv add pgzip

poetry add pgzip

Installing pgzip

Before you install

Low friction: pure Python wheel with no runtime dependencies. Active maintenance as of 2026-03-02, though maintainers are actively seeking additional contributors.

License in practice

MIT License (permissive) — you can use, modify, and distribute pgzip freely in commercial and private projects with minimal restrictions.

Quickstart

import pgzip

# Compress with 8 threads
with pgzip.open("file.txt.gz", "wt", thread=8) as f:
    f.write("data")

# Decompress
with pgzip.open("file.txt.gz", "rt", thread=8) as f:
    data = f.read()

Requires Python 3.10 or later.

Verify before relying

• Whether seek() and tell() methods work reliably despite the documented warning about incomplete testing.
• Real-world performance gains on machines with fewer cores or on I/O-constrained systems.
• Stability and compatibility with all gzip file variants and edge cases beyond standard FEXTRA blocks.

Package facts

Evidence: pgzip-0.4.0-py3-none-any.whl

Tags

More Python Modules packages