xopen

What it is and what it does

xopen is a file-opening utility that extends Python's built-in open() function to handle compressed files transparently. When you pass a filename with a recognized compression extension (.gz, .bz2, .xz, .zst), xopen automatically detects the format and opens it with an appropriate decompressor. For writing, the format is inferred from the filename extension; for reading, it can detect format from either the extension or the file's magic bytes. The package intelligently delegates to the most efficient available backend—preferring specialized libraries like python-isal and python-zlib-ng for gzip, external tools like pigz and xz for parallel compression, and falling back to Python's standard library (gzip, lzma, bz2) when specialized backends are unavailable.

The API is intentionally minimal: a single xopen() function that accepts the same parameters as built-in open() plus compression-specific options like compresslevel and threads. By default, xopen spawns additional threads (up to four) to offload compression work from the main Python thread, though this can be disabled by setting threads=0. The package also ensures reproducible gzip output by stripping timestamps from headers. It's designed for workflows that need to read or write compressed files without coupling to a specific compression library or worrying about which backend is available.

Use it for

• Processing large compressed log files or data archives without decompressing to disk first.
• Writing compressed output in a script without manually choosing and configuring a compression backend.
• Reading files where compression format is unknown, relying on magic-byte detection.
• Parallel compression/decompression of large files by leveraging multi-threaded backends.
• Reproducible data pipelines that generate gzip files with consistent output across runs.

Worth the install?

AI-flagged interpretation of the facts on this page. Verify before relying on it.

Install

xopen on PyPI

Before you install

Low install friction with a pure-Python wheel. Three optional compression backends (isal, zlib-ng, backports.zstd) are listed as runtime dependencies but the package gracefully falls back to Python built-ins if unavailable. Active maintenance with a recent release (72 days old).

Requires Python 3.10 or later. Zstandard support requires either the zstd command-line tool, backports.zstd package, or Python 3.14+ standard library compression.zstd module.

License in practice

MIT license permits commercial and private use with minimal restrictions—you may use, modify, and distribute xopen freely provided you include the license notice.

Quickstart

from xopen import xopen

# Read a gzip file
with xopen("file.txt.gz") as f:
    content = f.read()

# Write with compression level and threading control
with xopen("file.txt.xz", mode="wb", threads=0, compresslevel=3) as f:
    f.write(b"Hello")

Package facts

Evidence: xopen-2.1.0-py3-none-any.whl

Tags

More Utilities packages

See also bz2file · compress-pickle · pyzstd · indexed-gzip · dnaio · zipfile-zstd · pgzip · smart-open · mgzip · zstd