wordfreq
Look up the frequencies of words in many languages, based on many sources of data.
Decision gist · record as of 2026-08-14
Yes, if you need multilingual word frequency data. The package is stable, well-maintained at a dormant pace (last commit January 2025), has no security issues, and covers a broad range of languages with multiple corpus sources. Install friction is low. The main consideration is whether you need active development—the package is not being enhanced, only kept functional. For lookup-only use cases, this is a solid choice.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.8 or later.
- Chinese, Japanese, and Korean word frequencies require additional installation steps beyond the base package.
- Low install friction with a pure-Python wheel distribution.
License · maintenance · safety
Apache-2.0 (permissive) — Licensed under Apache-2.0 (permissive), allowing commercial and private use with minimal restrictions beyond attribution and liability disclaimers.
last release 2023-11-21 (997 days) · last repo commit 2025-01-04 · 1,729 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 2,306,469 downloads/mo, #3,150 on PyPI
Alternatives
Verify before relying
pip install wordfreq
from wordfreq import word_frequency, zipf_frequency
freq = word_frequency('cafe', 'en')
zipf = zipf_frequency('the', 'en')- Whether the 'large' wordlists are automatically downloaded on first use or require manual setup.
- Memory footprint and load time for different language/wordlist combinations in production.
- How frequently the underlying corpus data is refreshed or updated.
What it is and what it does
wordfreq is a lookup library for word frequencies across over 40 languages, drawing from multiple corpus sources including Wikipedia, news, books, web text, Twitter, and Reddit. It provides two main query functions: word_frequency() returns a decimal frequency between 0 and 1, while zipf_frequency() returns a logarithmic scale where a Zipf value of 6 means the word appears once per thousand words. The library offers both 'small' wordlists (words appearing at least once per million words) and 'large' lists (once per 100 million words), with a 'best' default that uses large when available.
The package combines frequencies from multiple sources using a figure-skating-inspired method that discards outliers and averages the remainder, then rescales to preserve relative precision. It handles special cases like numbers (aggregating by digit shape and applying Benford's law for first-digit distribution) and supports Unicode text including non-Latin digit systems. Dependencies are lightweight: msgpack, langcodes, regex, ftfy, and locate. It runs on Python 3.8+ and has no known security vulnerabilities.
Use it for
- Build spell-checkers or autocomplete systems that rank suggestions by word prevalence in a target language.
- Filter or weight text analysis results by word frequency to focus on meaningful rather than common words.
- Estimate language proficiency or corpus difficulty by analyzing the frequency distribution of words in a text.
- Train NLP models with frequency-weighted word embeddings or vocabulary pruning across multiple languages.
- Validate or debug tokenization and language detection by checking whether token frequencies match expected patterns.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you need multilingual word frequency data.
The package is stable, well-maintained at a dormant pace (last commit January 2025), has no security issues, and covers a broad range of languages with multiple corpus sources. Install friction is low. The main consideration is whether you need active development—the package is not being enhanced, only kept functional. For lookup-only use cases, this is a solid choice.
Install
wordfreq on PyPI
Before you install
Low install friction with a pure-Python wheel distribution. Maintenance is dormant—last release was 2023-11-21 and the repository received its most recent commit on 2025-01-04, so the package is not actively developed but remains functional and receives occasional updates.
Requires Python 3.8 or later. Chinese, Japanese, and Korean word frequencies require additional installation steps beyond the base package.
License in practice
Licensed under Apache-2.0 (permissive), allowing commercial and private use with minimal restrictions beyond attribution and liability disclaimers.
Quickstart
pip install wordfreq
from wordfreq import word_frequency, zipf_frequency
freq = word_frequency('cafe', 'en')
zipf = zipf_frequency('the', 'en')Verify before relying
- Whether the 'large' wordlists are automatically downloaded on first use or require manual setup.
- Memory footprint and load time for different language/wordlist combinations in production.
- How frequently the underlying corpus data is refreshed or updated.
Package facts
| License | Apache-2.0 permissive |
| Python support | Supports the current Python release >=3.8,<4 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 5 packagesmsgpacklangcodesregexftfylocate |
| Maintenance | Dormant 997 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 2,306,469 / month, #3,150 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | License :: OSI Approved :: Apache Software LicenseProgramming Language :: Python :: 3Programming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9 |
Evidence: wordfreq-3.1.1-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “word frequency lookup”
- wordfreqwordfreq looks up word frequencies across over 40 languages from…
- preshedPreshed provides high-performance Cython hash tables for mapping…
- wordcloudGenerates word cloud visualizations from text input, displaying word…
Give your agent the search over MCP, or paste the wish link into any chat.
More Linguistic packages
Detects and normalizes text encoding from unknown or ambiguous sources, supporting all IANA character sets that Python's core library provides codecs for, with the ability to register custom codecs.
tiktoken is a fast BPE tokenizer that converts text into token sequences compatible with OpenAI models, supporting multiple encoding schemes including o200k_base and model-specific encodings.
Install it if you work with OpenAI APIs or need to understand token boundaries in GPT-family models.
Detects character encoding and language in byte sequences with high accuracy, supporting 99 encodings and returning confidence scores, language tags, and MIME types.
Install it if you need to detect character encoding or language in byte data; the rewrite makes it substantially faster and more accurate than its predecessors.
Converts Unicode text to ASCII by transliterating non-ASCII characters into their closest ASCII equivalents, with no runtime dependencies.
However, if transliteration quality or ongoing maintenance matters, consider unidecode instead despite its GPL-only license.
Lark is a parsing library that builds abstract syntax trees from context-free grammars, supporting multiple parsing algorithms (Earley, LALR(1), CYK) with automatic line and column tracking.
Python bindings to the tree-sitter parsing library, enabling incremental parsing and syntax tree analysis for source code.
See also syllapy · wordcloud · english-words · pyspellchecker · stop-words · num2words · indic-numtowords · wordsegment · stopwordsiso · wn