{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/5"},{"label":"Linguistic","url":"https://skillfed.io/packages/category/text-processing-linguistic/2"}],"enrichment":{"capability":"Identifies the language of text in one of 97 languages using a pre-trained statistical model, with optimized speed for Python 3.6+.","skillfed_tags":["language-detection","nlp-preprocessing","cli-tool"],"use_cases":["Automatically categorize user-generated content by language for multilingual applications or content management systems.","Filter or route text documents to language-specific processing pipelines in NLP workflows.","Identify the dominant language in mixed-language documents or social media posts for downstream analysis.","Build language detection into a command-line data pipeline using the standalone langid tool.","Normalize confidence scores to probabilities for machine learning feature engineering or confidence thresholding."],"what_it_does":"py3langid is a modernized fork of langid.py that detects which of 97 languages a piece of text is written in. It returns both the ISO 639-1 language code and a confidence score. The package is designed to be a drop-in replacement for the original, with significant performance improvements: imports are about 30% faster, model loading is 25-30x faster, and classification itself is 5-6x faster on paragraphs.\n\nThe tool works as both a Python library and a command-line utility. It uses a pre-trained statistical model built from diverse sources (JRC-Acquis, ClueWeb 09, Wikipedia, Reuters RCV2, Debian i18n) and requires only numpy as a dependency. It supports language subsetting, probability normalization, and can be deployed as a WSGI web service.","worth_installing":"Yes, if you need fast language detection across 97 languages with minimal dependencies. The dormant maintenance status is not a blocker\u2014the package is stable, has no known vulnerabilities, and the last commit is recent enough. Install if you want a lightweight, performant drop-in replacement for the original langid.py; skip if you need active development or support for languages outside the pre-trained set."},"id":"py3langid","links":{"html":"https://skillfed.io/packages/py3langid","md":"https://skillfed.io/packages/py3langid.md","pypi":"https://pypi.org/project/py3langid/"},"maintenance":{"status":"dormant"},"meta":{"latest_release":"2024-06-18","license_spdx":null,"license_treatment":"permissive","name":"py3langid","python_support":"supports_current","summary":"Fork of the language identification tool langid.py, featuring a modernized codebase and faster execution times."},"popularity":{"monthly_downloads":437924,"position":6663,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.3.0"}