mgzip

What it is and what it does

mgzip is a multi-threaded implementation of Python's gzip module that parallelizes compression and decompression across CPU cores. It uses a block-indexed GZIP format (via the FEXTRA field defined in GZIP spec 4.3) to enable concurrent processing while remaining fully compatible with standard GZIP readers. The module replaces gzip's GzipFile class and open/compress/decompress functions with threaded equivalents.

The package is designed for scenarios where large files need fast compression or decompression on multi-core systems. It trades some compatibility (seek/tell not yet supported) for speed gains. The implementation is in Beta status and actively maintained, though the author notes that real-world performance depends on IO and Python interpreter limitations rather than pure CPU scaling.

Use it for:

• Compress large log files or data dumps quickly on multi-core servers before archival or transfer.
• Decompress GZIP archives in parallel to speed up data pipeline initialization on systems with spare CPU capacity.
• Replace standard gzip calls in existing code with minimal changes to gain multi-threaded performance.
• Process block-indexed GZIP files created by mgzip itself for faster random-access decompression.
• Reduce compression time in batch data processing workflows where IO is not the bottleneck.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

mgzip provides a drop-in replacement for Python's standard gzip module that uses multi-threading to compress and decompress files in parallel, compatible with standard GZIP format.

Yes, if you need faster GZIP compression/decompression on multi-core systems and can work around the missing seek/tell methods. Low install friction, no dependencies, MIT-licensed, and actively maintained. Not suitable if you rely on seek/tell or need production-grade stability—the package is still Beta.

Install

mgzip on PyPI

pip install mgzip

uv add mgzip

poetry add mgzip

Installing mgzip

Before you install

Low install friction with no runtime dependencies. Actively maintained as of 2026-02-28 with recent development activity.

License in practice

MIT License permits commercial and private use with minimal restrictions; suitable for most projects.

Quickstart

import mgzip

with mgzip.open("file.txt.gz", "wt", thread=8, blocksize=200000000) as f:
    f.write("data")

with mgzip.open("file.txt.gz", "rt", thread=8) as f:
    data = f.read()

Requires Python 3.8 or later; seek() and tell() are not yet supported.

Verify before relying

• Whether seek() and tell() methods will be implemented in future releases
• Real-world performance gains on systems with fewer than 24 CPU cores
• Compatibility with compressed files created by other multi-threaded gzip implementations

Package facts

Evidence: mgzip-0.2.5-py3-none-any.whl

Tags

More Python Modules packages