{"categories":[{"label":"Linguistic","url":"https://skillfed.io/packages/category/text-processing-linguistic"}],"enrichment":{"capability":"Unsupervised Korean natural language processing toolkit that extracts words and nouns from text, tokenizes sentences, and performs part-of-speech analysis without requiring training data.","skillfed_tags":["korean-nlp","unsupervised-learning","tokenization"],"use_cases":["Extract new or domain-specific Korean words from a corpus of news articles or social media posts without a pre-trained dictionary.","Tokenize Korean sentences into word sequences using learned word boundaries from a corpus of similar documents.","Identify nouns in Korean text by analyzing right-side character patterns (e.g., particles following noun candidates).","Build a custom Korean tokenizer by combining noun scores with word cohesion scores for domain-specific text.","Analyze L-R graph structure of Korean words to understand morphological patterns in a specific corpus."],"what_it_does":"soynlp is an unsupervised Korean NLP toolkit designed to extract linguistic structure from text without pre-trained models or labeled training data. It uses statistical patterns\u2014cohesion scores, branching entropy, and accessor variety\u2014to identify word boundaries, extract nouns, and tokenize sentences. The package includes WordExtractor for discovering new words in a corpus, multiple noun extractors (LRNounExtractor versions 1 and 2, NewsNounExtractor), and tokenizers that split sentences into word sequences based on learned word scores.\n\nThe toolkit works by analyzing how characters and substrings co-occur and branch within a corpus, learning left-right (L-R) structure patterns typical of Korean morphology. It is intended for processing document collections from a single domain or genre (news articles, movie reviews, social media) where homogeneous vocabulary and writing style yield better statistical signals. Dependencies are standard scientific Python libraries: numpy, scipy, scikit-learn, and psutil for memory monitoring during training.","worth_installing":"Yes, if you are processing Korean text and need unsupervised word/noun extraction or tokenization without external models. The low install friction and active maintenance make it accessible. However, GPLv3 licensing restricts proprietary use, and the package requires a reasonably large, homogeneous corpus to work well\u2014single documents or mixed-domain text will produce poor results. Not suitable for production systems requiring copyleft-incompatible licensing."},"id":"soynlp","links":{"html":"https://skillfed.io/packages/soynlp","md":"https://skillfed.io/packages/soynlp.md","pypi":"https://pypi.org/project/soynlp/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2019-08-25","license_spdx":null,"license_treatment":"copyleft","name":"soynlp","python_support":"unspecified","summary":"Unsupervised Korean Natural Language Processing Toolkits"},"popularity":{"monthly_downloads":757471,"position":5135,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.0.493"}