xmhuffman

What it is and what it does

xmhuffman is a Cython extension that decodes the canonical-Huffman compressed string dictionaries stored inside Power BI `.pbix` files and Excel Power Pivot workbooks. These files use the xVelocity/Vertipaq column-store format, which compresses string data using Huffman coding. The package provides a tight C kernel that replaces what would otherwise be a per-symbol Python loop, delivering significant performance gains on real workloads.

The library exposes a primary entry point `decode_page()` that takes a bitstream, code-length array, string offsets, and total bit count—all extracted directly from Vertipaq dictionary page metadata—and returns a list of decoded byte strings. It handles both single-charset and multi-charset pages, with optional in-extension byte-pair swapping and charset reinsertion. Lower-level helpers like `build_table()` and `decode_with_table()` allow amortization across multiple pages. No runtime dependencies; zero external imports required.

Use it for:

• Extract and analyze data from Power BI `.pbix` files by decompressing the embedded Vertipaq string dictionaries.
• Parse Excel Power Pivot workbooks (`xl/model/item.data`) to recover the original string values from compressed storage.
• Build tools that inspect or transform Power BI data models without opening Power BI Desktop.
• Parallelize large-scale `.pbix` extraction using `concurrent.futures.ThreadPoolExecutor` with per-page decoding.
• Reverse-engineer or audit the string content of Power BI reports for compliance or data discovery.
• Integrate Huffman decompression into data pipeline tools that consume Power BI artifacts.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Decodes canonical-Huffman compressed string dictionaries from xVelocity/Vertipaq column stores used in Power BI `.pbix` files and Excel Power Pivot workbooks via a fast Cython extension.

Yes, if you need to extract or analyze data from Power BI `.pbix` files or Excel Power Pivot workbooks. The package solves a specific, narrow problem—Vertipaq Huffman decompression—with no dependencies and prebuilt wheels for common platforms. It is actively maintained, MIT-licensed, and offers substantial performance gains on real workloads. Install only if you are working with these Microsoft formats; it has no utility for general-purpose Huffman decoding or other compression tasks.

Install

xmhuffman on PyPI

pip install xmhuffman

uv add xmhuffman

poetry add xmhuffman

Installing xmhuffman

Before you install

Medium install friction due to compiled wheels; prebuilt binaries available for Python 3.9, 3.10, 3.11, 3.12 on macOS, Linux (x86_64 and aarch64), and Windows. Building from source requires a C compiler and Cython ≥ 3.0. Package is actively maintained with recent releases.

License in practice

MIT License permits unrestricted use, modification, and distribution with minimal restrictions—suitable for both open-source and commercial projects.

Quickstart

import xmhuffman

# Decode a Vertipaq dictionary page
strings = xmhuffman.decode_page(
    bitstream,           # compressed_string_buffer (bytes)
    encode_array_128,    # 128-byte nibble-packed code-length array
    offsets,             # per-string start bit offsets
    total_bits,          # store_total_bits (end of last string)
    swap=True            # apply byte-pair swap inside extension
)
# Output: list[bytes]

Requires Python ≥ 3.8; building from source requires a C compiler and Cython ≥ 3.0.

Verify before relying

• Actual performance gains on real-world `.pbix` files beyond the benchmark examples provided.
• Thread-safety guarantees and behavior under high concurrent load in production scenarios.
• Compatibility with Python versions beyond those with prebuilt wheels.

Package facts

Evidence: xmhuffman-0.3.0-cp310-cp310-macosx_11_0_universal2.whl; xmhuffman-0.3.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; xmhuffman-0.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; xmhuffman-0.3.0-cp310-cp310-win_amd64.whl; xmhuffman-0.3.0-cp311-cp311-macosx_11_0_universal2.whl; xmhuffman-0.3.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; xmhuffman-0.3.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; xmhuffman-0.3.0-cp311-cp311-win_amd64.whl; xmhuffman-0.3.0-cp312-cp312-macosx_11_0_universal2.whl; xmhuffman-0.3.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; xmhuffman-0.3.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; xmhuffman-0.3.0-cp312-cp312-win_amd64.whl; xmhuffman-0.3.0-cp313-cp313-macosx_11_0_universal2.whl; xmhuffman-0.3.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; xmhuffman-0.3.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; xmhuffman-0.3.0-cp313-cp313-win_amd64.whl; xmhuffman-0.3.0-cp39-cp39-macosx_11_0_universal2.whl; xmhuffman-0.3.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; xmhuffman-0.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; xmhuffman-0.3.0-cp39-cp39-win_amd64.whl

Tags

More Python Modules packages