lexical-diversity

What it is and what it does

Lexical-diversity is a Python library for computing quantitative measures of vocabulary diversity in text. It provides implementations of multiple diversity metrics—including Type-Token Ratio (TTR), Root TTR, Log TTR, Mean Segmental TTR, Moving Average TTR, Hypergeometric Distribution D (HDD), and Measure of Lexical Textual Diversity (MTLD)—each suited to different text lengths and analytical goals. The package includes convenience functions for tokenization and lemmatization, allowing users to preprocess text before calculating diversity scores.

The library is designed for linguistic research, computational text analysis, and language assessment tasks where understanding vocabulary breadth and repetition patterns is important. It has no external runtime dependencies, making it lightweight to install. However, the project is dormant—last released in 2020 and last committed in 2023—so users should not expect active maintenance or updates for newer Python versions or linguistic standards.

Use it for:

• Assess vocabulary richness in student writing samples or language learner essays to measure linguistic development
• Compare lexical diversity across different text corpora or authors to identify stylistic or quality differences
• Analyze vocabulary variation in clinical speech samples to detect language disorders or cognitive decline
• Evaluate machine-generated text quality by measuring how repetitive or diverse its vocabulary is
• Conduct computational linguistics research requiring multiple diversity metrics on the same text

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Calculates lexical diversity metrics (TTR, MTLD, HDD, and others) on tokenized text to measure vocabulary richness and variation.

Yes, if you need lexical diversity metrics for research or analysis and accept dormant maintenance. The package is lightweight, dependency-free, and offers multiple well-established metrics. However, verify compatibility with your Python version before relying on it in production, and consider whether a more actively maintained alternative better fits your timeline.

Install

lexical-diversity on PyPI

pip install lexical-diversity

uv add lexical-diversity

poetry add lexical-diversity

Installing lexical-diversity

Before you install

Installs with no runtime dependencies. Maintenance is dormant—last release was 2020-03-04 and last commit 2023-09-15—but the repository remains active and the package is stable enough for research use.

License in practice

MIT license permits commercial and private use with minimal restrictions, making it suitable for academic and production contexts.

Quickstart

pip install lexical-diversity

from lexical_diversity import lex_div as ld

text = "The state was named for the Colorado River..."
tok = ld.tokenize(text)
flt = ld.flemmatize(tok)
print(ld.ttr(flt))
print(ld.mtld(flt))

Verify before relying

• Whether the package works with modern Python versions (3.9+) despite no explicit version constraint in metadata
• Performance characteristics on very large texts or batch processing scenarios

Package facts

Evidence: lexical_diversity-0.1.1-py3-none-any.whl

Tags

More Linguistic packages