SudachiPy

What it is and what it does

SudachiPy is a Python binding to Sudachi.rs, a Japanese morphological analyzer that breaks Japanese text into individual morphemes (words/word components) and annotates each with grammatical information. It provides three tokenization granularity modes (A, B, C) to control how aggressively text is split, and returns morpheme objects with methods to access surface forms, dictionary forms, reading forms, part-of-speech tags, and normalized variants. The package includes both a command-line interface for shell pipelines and a Python API for programmatic use.

The package is distributed as precompiled wheels for modern Python versions on macOS, Windows, and Linux (x86_64 and aarch64). It has no pure-Python runtime dependencies—the heavy lifting is done by the compiled Sudachi.rs backend. A dictionary must be installed separately as a companion package. The project is actively maintained and widely used in Japanese NLP workflows.

Use it for:

• Tokenize Japanese text for downstream NLP tasks like named entity recognition or sentiment analysis
• Build a command-line pipeline to process Japanese documents with multiple granularity levels
• Extract normalized forms and reading annotations for Japanese text in search or indexing systems
• Integrate Japanese morphological analysis into a larger text processing or machine learning pipeline

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

SudachiPy is a Python binding for Sudachi.rs, a Japanese morphological analyzer that tokenizes Japanese text into morphemes with part-of-speech tags, normalized forms, and reading information.

Yes, if you need Japanese morphological analysis. SudachiPy is actively maintained, has no security vulnerabilities, uses a permissive license, and provides both CLI and Python API interfaces. The main gotcha is the separate dictionary dependency and medium install friction from compiled wheels, but prebuilt binaries cover common platforms. Suitable for production use in Japanese NLP workflows.

Install

sudachipy on PyPI

pip install sudachipy

uv add sudachipy

poetry add sudachipy

Installing SudachiPy

Before you install

Medium install friction due to compiled binary wheels. Prebuilt wheels are provided for macOS (10.14+), Windows, and Linux x86_64/aarch64, but ARM-based macOS requires Rust toolchain and Cargo. Package is actively maintained with recent releases.

License in practice

Licensed under Apache-2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions.

Quickstart

pip install sudachipy

from sudachipy import Dictionary, SplitMode

tokenizer = Dictionary().create()
morphemes = tokenizer.tokenize("国会議事堂前駅", SplitMode.A)
print([m.surface() for m in morphemes])

Requires a separate dictionary package to be installed; dictionary files download on first use (approximately 70MB for the core edition).

Verify before relying

• Whether Python version constraints exist beyond what the fact sheet indicates
• Performance characteristics for large-scale tokenization workloads
• Whether user dictionaries can be easily integrated for domain-specific terms
• Specific dictionary package names and their installation requirements

Package facts

Evidence: sudachipy-0.6.11-cp310-cp310-macosx_10_12_universal2.whl; sudachipy-0.6.11-cp310-cp310-macosx_10_12_x86_64.whl; sudachipy-0.6.11-cp310-cp310-macosx_11_0_arm64.whl; sudachipy-0.6.11-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl; sudachipy-0.6.11-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl; sudachipy-0.6.11-cp310-cp310-win_amd64.whl; sudachipy-0.6.11-cp311-cp311-macosx_10_12_universal2.whl; sudachipy-0.6.11-cp311-cp311-macosx_10_12_x86_64.whl; sudachipy-0.6.11-cp311-cp311-macosx_11_0_arm64.whl; sudachipy-0.6.11-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl; sudachipy-0.6.11-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl; sudachipy-0.6.11-cp311-cp311-win_amd64.whl; sudachipy-0.6.11-cp312-cp312-macosx_10_13_universal2.whl; sudachipy-0.6.11-cp312-cp312-macosx_10_13_x86_64.whl; sudachipy-0.6.11-cp312-cp312-macosx_11_0_arm64.whl; sudachipy-0.6.11-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl; sudachipy-0.6.11-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl; sudachipy-0.6.11-cp312-cp312-win_amd64.whl; sudachipy-0.6.11-cp313-cp313-macosx_10_13_universal2.whl; sudachipy-0.6.11-cp313-cp313-macosx_10_13_x86_64.whl

Tags

More Linguistic packages