indexed-gzip

What it is and what it does

indexed_gzip is a Python extension that replaces the standard gzip.GzipFile class with IndexedGzipFile, which enables efficient random access to gzip-compressed files. Instead of decompressing from the start of the file every time you seek to a new location, it builds an index of seek points (mappings between compressed and uncompressed positions) with accompanying decompression state. This allows you to decompress only the data needed to reach your target location, typically requiring far less work than the standard library's approach.

The package is designed for reading large gzip files where random access is needed—particularly NIFTI neuroimaging files, though it works with any gzip file. It integrates seamlessly with nibabel for medical imaging workflows and supports index import/export to pre-compute and cache indexes for frequently accessed files. No runtime dependencies are required beyond Python itself.

Use it for:

• Reading specific slices or regions from large compressed neuroimaging (NIFTI) files without decompressing the entire file.
• Accessing arbitrary byte ranges in large gzip archives where seek performance is critical.
• Integrating with nibabel to load compressed medical images with automatic index building on first access.
• Pre-computing and caching indexes for large gzip files to avoid rebuild overhead on repeated access.
• Replacing gzip.GzipFile in existing code that needs faster random access without changing the API.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides fast random access to gzip-compressed files by building an index of seek points, allowing you to jump to any location in the uncompressed data without decompressing from the beginning.

Yes, if you regularly read from large gzip files and need random access. The permissive Zlib license and active maintenance make it safe to adopt. Install friction is moderate due to compilation requirements, but pre-built wheels cover common platforms. No security vulnerabilities are known. Not necessary if you only read sequentially or work with small files.

Install

indexed-gzip on PyPI

pip install indexed-gzip

uv add indexed-gzip

poetry add indexed-gzip

Installing indexed-gzip

Before you install

Medium install friction due to compiled C/Cython components requiring a build step, but pre-built wheels are available for common platforms (macOS, Linux, Windows across multiple architectures). Last release was 249 days ago with active repository maintenance.

License in practice

Licensed under Zlib (permissive), allowing free use, modification, and distribution with minimal restrictions—suitable for both open-source and proprietary projects.

Quickstart

import indexed_gzip as igzip

myfile = igzip.IndexedGzipFile('big_file.gz')
myfile.seek(234195)
data = myfile.read(1048576)

Requires Python >= 3.7; C/Cython extension must compile or a pre-built wheel must be available for your platform.

Verify before relying

• Whether the index-building overhead is acceptable for your typical file sizes and access patterns.
• Performance gains compared to standard gzip.GzipFile for your specific use case and seek patterns.

Package facts

Evidence: indexed_gzip-1.10.3-cp310-cp310-macosx_10_9_universal2.whl; indexed_gzip-1.10.3-cp310-cp310-macosx_10_9_x86_64.whl; indexed_gzip-1.10.3-cp310-cp310-macosx_11_0_arm64.whl; indexed_gzip-1.10.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl; indexed_gzip-1.10.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl; indexed_gzip-1.10.3-cp310-cp310-manylinux_2_28_i686.whl; indexed_gzip-1.10.3-cp310-cp310-musllinux_1_2_aarch64.whl; indexed_gzip-1.10.3-cp310-cp310-musllinux_1_2_i686.whl; indexed_gzip-1.10.3-cp310-cp310-musllinux_1_2_x86_64.whl; indexed_gzip-1.10.3-cp310-cp310-win32.whl; indexed_gzip-1.10.3-cp310-cp310-win_amd64.whl; indexed_gzip-1.10.3-cp311-abi3-macosx_10_9_universal2.whl; indexed_gzip-1.10.3-cp311-abi3-macosx_10_9_x86_64.whl; indexed_gzip-1.10.3-cp311-abi3-macosx_11_0_arm64.whl; indexed_gzip-1.10.3-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl; indexed_gzip-1.10.3-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl; indexed_gzip-1.10.3-cp311-abi3-manylinux_2_28_i686.whl; indexed_gzip-1.10.3-cp311-abi3-musllinux_1_2_aarch64.whl; indexed_gzip-1.10.3-cp311-abi3-musllinux_1_2_i686.whl; indexed_gzip-1.10.3-cp311-abi3-musllinux_1_2_x86_64.whl

Tags

More Compression packages