unicode-segmentation-rs

What it is and what it does

unicode-segmentation-rs is a Python library that wraps Rust implementations of Unicode text processing algorithms. It splits text into grapheme clusters (user-perceived characters, handling complex emojis and combining marks), words, sentences, and line-break opportunities according to Unicode standards. It also calculates display width for terminal or monospace rendering, and provides gettext PO file wrapping with proper handling of escape sequences and CJK characters.

The package has no Python runtime dependencies and installs as a compiled extension, making it fast for text processing tasks common in localization, terminal UI, and multilingual applications. It supports modern Python versions (3.10+) and is actively maintained.

Use it for:

• Split text into grapheme clusters to correctly count user-perceived characters in strings with complex emojis or combining diacritics.
• Find word and sentence boundaries for text analysis, search indexing, or natural language processing pipelines.
• Calculate display width of text for terminal wrapping, monospace layout, or UI rendering without measuring individual glyphs.
• Wrap localization strings for gettext PO files while preserving escape sequences and respecting CJK character boundaries.
• Identify legal line-break opportunities in multilingual text for text editors or document formatters.
• Process Arabic, Japanese, Chinese, and other non-Latin scripts with Unicode-compliant segmentation rules.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides Unicode text segmentation (graphemes, words, sentences, line breaks) and display width calculation via Python bindings to Rust crates, following Unicode standards.

Yes, with a license caveat. The package is actively maintained, has no known vulnerabilities, installs cleanly on common platforms, and solves a real problem (correct Unicode text segmentation) that pure Python solutions handle poorly. However, verify the license status before use in proprietary projects, since the metadata does not declare one.

Install

unicode-segmentation-rs on PyPI

pip install unicode-segmentation-rs

uv add unicode-segmentation-rs

poetry add unicode-segmentation-rs

Installing unicode-segmentation-rs

Before you install

Medium install friction due to compiled wheels; however, prebuilt binaries are available for common platforms (x86_64, ARM, PowerPC, s390x on Linux/macOS/Windows). Package is actively maintained with recent releases.

License in practice

License status is unclear—no SPDX identifier or raw license text is recorded in the package metadata. Verify the actual license before use in proprietary or restricted contexts.

Quickstart

pip install unicode-segmentation-rs

import unicode_segmentation_rs

text = "Hello 👨‍👩‍👧‍👦 World"
clusters = unicode_segmentation_rs.graphemes(text, is_extended=True)
print(clusters)

Requires Python 3.10 or later; precompiled wheels available for most platforms, but source builds require maturin and Rust.

Verify before relying

• Whether the package is licensed under an open-source or proprietary license (metadata shows 'unclear').
• Performance characteristics and memory overhead for very large text inputs.
• Whether PyPy support (listed in classifiers) is fully tested and stable.

Package facts

Evidence: unicode_segmentation_rs-0.3.3-cp310-abi3-macosx_10_12_x86_64.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-macosx_11_0_arm64.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-manylinux_2_5_i686.manylinux1_i686.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_aarch64.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_armv7l.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_i686.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-musllinux_1_2_x86_64.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-win32.whl; unicode_segmentation_rs-0.3.3-cp310-abi3-win_amd64.whl; unicode_segmentation_rs-0.3.3-cp314-cp314t-macosx_10_12_x86_64.whl; unicode_segmentation_rs-0.3.3-cp314-cp314t-macosx_11_0_arm64.whl; unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl; unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl; unicode_segmentation_rs-0.3.3-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl

Tags

More Python Modules packages