xopen

What it is and what it does

xopen is a file-opening utility that extends Python's built-in open() function to handle compressed files transparently. When you pass a filename with a recognized compression extension (.gz, .bz2, .xz, .zst), xopen automatically detects the format and opens it with an appropriate decompressor. For writing, the format is inferred from the filename extension; for reading, it can detect format from either the extension or the file's magic bytes. The package intelligently delegates to the most efficient available backend—preferring specialized libraries like python-isal and python-zlib-ng for gzip, external tools like pigz and xz for parallel compression, and falling back to Python's standard library (gzip, lzma, bz2) when specialized backends are unavailable.

The API is intentionally minimal: a single xopen() function that accepts the same parameters as built-in open() plus compression-specific options like compresslevel and threads. By default, xopen spawns additional threads (up to four) to offload compression work from the main Python thread, though this can be disabled by setting threads=0. The package also ensures reproducible gzip output by stripping timestamps from headers. It's designed for workflows that need to read or write compressed files without coupling to a specific compression library or worrying about which backend is available.

Use it for:

• Processing large compressed log files or data archives without decompressing to disk first.
• Writing compressed output in a script without manually choosing and configuring a compression backend.
• Reading files where compression format is unknown, relying on magic-byte detection.
• Parallel compression/decompression of large files by leveraging multi-threaded backends.
• Reproducible data pipelines that generate gzip files with consistent output across runs.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

xopen provides a drop-in replacement for Python's built-in open() that transparently handles compressed files in gzip, bzip2, xz, and Zstandard formats, automatically selecting efficient backends.

Yes. xopen is actively maintained, has no known vulnerabilities, uses a permissive MIT license, and installs with low friction. It solves a genuine problem—transparent compressed file handling—with a minimal, stable API. The optional backends (isal, zlib-ng, backports.zstd) gracefully degrade if unavailable, so the package works out of the box. Suitable for any workflow involving compressed files.

Install

xopen on PyPI

pip install xopen

uv add xopen

poetry add xopen

Installing xopen

Before you install

Low install friction with a pure-Python wheel. Three optional compression backends (isal, zlib-ng, backports.zstd) are listed as runtime dependencies but the package gracefully falls back to Python built-ins if unavailable. Active maintenance with a recent release (72 days old).

License in practice

MIT license permits commercial and private use with minimal restrictions—you may use, modify, and distribute xopen freely provided you include the license notice.

Quickstart

from xopen import xopen

# Read a gzip file
with xopen("file.txt.gz") as f:
    content = f.read()

# Write with compression level and threading control
with xopen("file.txt.xz", mode="wb", threads=0, compresslevel=3) as f:
    f.write(b"Hello")

Requires Python 3.10 or later. Zstandard support requires either the zstd command-line tool, backports.zstd package, or Python 3.14+ standard library compression.zstd module.

Verify before relying

• Performance characteristics compared to direct use of individual compression libraries for specific formats.
• Behavior when optional backends (isal, zlib-ng, backports.zstd) are not installed—exact fallback chain and performance impact.

Package facts

Evidence: xopen-2.1.0-py3-none-any.whl

Tags

More Utilities packages