faust-cchardet

What it is and what it does

faust-cchardet is a maintained fork of cChardet, a high-speed character encoding detector that wraps the uchardet C++ library. It identifies the encoding of binary data by analyzing byte patterns, supporting dozens of languages and encodings from ASCII and UTF variants to CJK, Cyrillic, Arabic, and European character sets. The package is designed for both one-shot detection (call detect() on a byte string) and streaming detection (feed chunks to a UniversalDetector instance).

This fork exists to keep the project building on modern Python (3.10+) and Windows toolchains, shipping prebuilt wheels so most users avoid needing a C++ compiler. It bundles uchardet by default for better detection accuracy, though it can optionally link a system uchardet library. The public API is unchanged from upstream—you import as cchardet—but the PyPI distribution is renamed to faust-cchardet. The package is thread-safe for stateless detect() calls but not for shared UniversalDetector instances.

Use it for:

• Detect encoding of uploaded files or network streams before parsing or displaying them.
• Preprocess text data in data pipelines to normalize encoding before NLP or ML processing.
• Identify encoding of legacy documents or logs in mixed-encoding archives.
• Stream large files incrementally to detect encoding without loading them entirely into memory.
• Handle international user-generated content where encoding is unknown or misreported.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Detects character encoding in binary data with high speed, supporting dozens of languages and encodings through a C++ binding to uchardet.

Yes, if you need reliable encoding detection and can accept the copyleft license terms. The package is actively maintained, has no known vulnerabilities, supports current Python versions, and ships prebuilt wheels for common platforms, making installation frictionless for most users. The bundled uchardet build offers better accuracy than the system library. Avoid if your project forbids GPL/LGPL dependencies or if you require Python < 3.10.

Install

faust-cchardet on PyPI

pip install faust-cchardet

uv add faust-cchardet

poetry add faust-cchardet

Installing faust-cchardet

Before you install

Medium install friction due to compiled C++ extension, but prebuilt wheels are available for common platforms (Linux x86_64/aarch64, macOS x86_64/arm64, Windows x86_64) across Python 3.10–3.14, reducing compilation needs for most users. Actively maintained as of 2026-08-10.

License in practice

Licensed under MPL-1.1 OR GPL-2.0-or-later OR LGPL-2.1-or-later (copyleft). Use is permitted but derivative works or distributions may require source disclosure and compatibility with GPL/LGPL terms depending on which license applies to your use.

Quickstart

pip install faust-cchardet

import cchardet

with open('file.bin', 'rb') as f:
    result = cchardet.detect(f.read())
    print(result)

Requires Python >= 3.10. On platforms without prebuilt wheels, a C++ compiler and build tools are needed.

Verify before relying

• Actual throughput (MB/s) claims in the description are specific to the bundled uchardet build; performance with system uchardet may differ.
• The 16.2% false-positive rate for non-UTF-8 detection with system uchardet is documented as an upstream issue; whether this affects your use case depends on your input data.
• Whether prebuilt wheels cover your specific platform/Python combination (e.g., PyPy on Linux is listed but not all combinations may be published).
• Confidence scores returned by detect() and their typical ranges across different encodings and input sizes.

Package facts

Evidence: faust_cchardet-3.2.0-cp310-cp310-macosx_10_9_x86_64.whl; faust_cchardet-3.2.0-cp310-cp310-macosx_11_0_arm64.whl; faust_cchardet-3.2.0-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; faust_cchardet-3.2.0-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; faust_cchardet-3.2.0-cp310-cp310-musllinux_1_2_aarch64.whl; faust_cchardet-3.2.0-cp310-cp310-musllinux_1_2_x86_64.whl; faust_cchardet-3.2.0-cp310-cp310-win_amd64.whl; faust_cchardet-3.2.0-cp311-cp311-macosx_10_9_x86_64.whl; faust_cchardet-3.2.0-cp311-cp311-macosx_11_0_arm64.whl; faust_cchardet-3.2.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; faust_cchardet-3.2.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; faust_cchardet-3.2.0-cp311-cp311-musllinux_1_2_aarch64.whl; faust_cchardet-3.2.0-cp311-cp311-musllinux_1_2_x86_64.whl; faust_cchardet-3.2.0-cp311-cp311-win_amd64.whl; faust_cchardet-3.2.0-cp312-cp312-macosx_10_13_x86_64.whl; faust_cchardet-3.2.0-cp312-cp312-macosx_11_0_arm64.whl; faust_cchardet-3.2.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; faust_cchardet-3.2.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; faust_cchardet-3.2.0-cp312-cp312-musllinux_1_2_aarch64.whl; faust_cchardet-3.2.0-cp312-cp312-musllinux_1_2_x86_64.whl

Keywords: cython, chardet, charsetdetect

Tags

More Libraries packages