jieba

What it is and what it does

Jieba is a Chinese text segmentation library that breaks Chinese sentences into individual words. It offers three segmentation modes: precise mode for accurate analysis, full mode for exhaustive word extraction, and search-engine mode for improved recall in retrieval tasks. The library supports both simplified and traditional Chinese, allows users to inject custom dictionaries, and has no external runtime dependencies.

The package is widely used in Chinese NLP pipelines but has been dormant since January 2020. While it maintains high GitHub visibility and download counts, the lack of recent maintenance means it may have compatibility issues with modern Python versions and may not reflect current Chinese language conventions or terminology.

Use it for:

• Tokenizing Chinese text for machine learning models that require word-level input features.
• Building Chinese search engines or information retrieval systems using search-engine segmentation mode.
• Preprocessing Chinese documents for NLP analysis, sentiment analysis, or text classification tasks.
• Extracting keywords from Chinese text by combining segmentation with frequency analysis.
• Processing domain-specific Chinese text by adding custom words to the dictionary.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Jieba segments Chinese text into words using multiple algorithms (precise, full, and search-engine modes) and supports both simplified and traditional Chinese with custom dictionary injection.

Yes, with conditions. Jieba remains the de facto standard for Chinese word segmentation in Python and has proven reliability for its core task. However, install it only if you are working with Chinese text and can verify compatibility with your Python version—the package is dormant and may not work on very recent Python releases without testing. Check the GitHub repository for community forks or patches if you encounter issues.

Install

jieba on PyPI

pip install jieba

uv add jieba

poetry add jieba

Installing jieba

Before you install

High install friction: the package is dormant (last release January 2020, last commit August 2024) and has not been updated for years. Despite 35112 GitHub stars and 2.4M monthly downloads, the lack of recent maintenance and unspecified Python version support create uncertainty about compatibility with current Python releases.

License in practice

MIT license is permissive and imposes no restrictions on use, modification, or distribution in proprietary or open-source projects.

Quickstart

pip install jieba

import jieba
result = jieba.cut('我爱自然语言处理')
print(list(result))

No runtime dependencies, but package is dormant since 2020 and Python version support is unspecified; compatibility with modern Python versions is uncertain.

Verify before relying

• Whether the package works reliably on Python 3.8 and later versions given the lack of updates since 2020.
• Whether the built-in dictionary is current for modern Chinese usage and terminology.
• Performance characteristics and memory usage for large-scale text segmentation tasks.

Package facts

Evidence: jieba-0.42.1.tar.gz

Keywords: NLP

Tags

More Text Processing packages