pyvi

What it is and what it does

Pyvi is a Vietnamese natural language processing toolkit that handles tokenization, part-of-speech tagging, and accent manipulation. It uses conditional random fields as its underlying algorithm and reports an F1 score of 0.985 for tokenization and 0.925 for POS tagging. The package depends on scikit-learn and sklearn-crfsuite for its machine learning operations.

The toolkit is designed for developers working with Vietnamese text who need to split sentences into tokens, identify grammatical roles (adjectives, nouns, verbs, etc.), or normalize accents. It integrates with spacy.io and handles common Vietnamese text issues like redundant spacing. However, the package has been dormant since mid-2021, with no recent updates or active maintenance.

Use it for:

• Tokenize Vietnamese sentences into words for downstream NLP pipelines or text analysis.
• Tag Vietnamese words with their grammatical roles (noun, verb, adjective, etc.) for linguistic analysis.
• Remove or add diacritical marks to Vietnamese text for normalization or text processing workflows.
• Prepare Vietnamese text for machine learning models that require tokenized and tagged input.
• Integrate Vietnamese language support into spacy.io-based NLP applications.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides Vietnamese language processing tools including tokenization, part-of-speech tagging, and accent manipulation using conditional random fields.

Yes, if you need Vietnamese NLP and can tolerate dormant maintenance. The package has no known vulnerabilities, low install friction, and a permissive MIT license. However, verify that scikit-learn and sklearn-crfsuite versions remain compatible with your Python environment, and be aware that no updates have been released since 2021.

Install

pyvi on PyPI

pip install pyvi

uv add pyvi

poetry add pyvi

Installing pyvi

Before you install

Low install friction with a pure-Python wheel. Maintenance is dormant—last release was 2021-06-30 and the repository shows no recent activity, though it remains unarchived with a final commit on 2024-09-26.

License in practice

MIT license is permissive, allowing commercial and private use with minimal restrictions.

Quickstart

pip install pyvi

from pyvi import ViTokenizer, ViPosTagger, ViUtils

ViTokenizer.tokenize(u"Trường đại học bách khoa hà nội")
ViPosTagger.postagging(ViTokenizer.tokenize(u"Trường đại học Bách Khoa Hà Nội"))
ViUtils.remove_accents(u"Trường đại học bách khoa hà nội")

Verify before relying

• Whether scikit-learn and sklearn-crfsuite are downloaded as pre-trained models or require separate setup.
• Current compatibility with Python versions beyond 3.x, given classifiers list Python 2.6 and 2.7.

Package facts

Evidence: pyvi-0.1.1-py2.py3-none-any.whl

Keywords: Vietnamese, natural, language, processing

Tags

More Linguistic packages