gensim

What it is and what it does

Gensim is a mature Python framework for unsupervised learning on large text corpora, specializing in topic modeling and document similarity. It implements memory-efficient, out-of-core algorithms including Latent Dirichlet Allocation (LDA), Latent Semantic Analysis (LSA), word2vec, and Hierarchical Dirichlet Process (HDP), designed to handle corpora larger than available RAM through streaming and generator-based processing. The library taps into optimized BLAS libraries via numpy and scipy for fast matrix operations while keeping the top-level API in pure Python.

Gensim is intended for the natural language processing and information retrieval community. It offers a simple streaming API for plugging in custom corpora, a transformation API for extending with new algorithms, and supports distributed computing for LSA and LDA across clusters. The package has been in production use since 2010 and is actively tested across Python 3.9–3.13.

Use it for:

• Train topic models (LDA, LSA) on large document collections to discover latent themes and organize unstructured text.
• Build word embeddings using word2vec to capture semantic relationships for downstream NLP tasks.
• Index and retrieve similar documents in a corpus using vector space models and similarity metrics.
• Process text corpora that exceed available RAM by streaming documents through memory-efficient generators.
• Implement custom NLP pipelines by chaining gensim transformations (e.g., TF-IDF, LSI) on document streams.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Gensim is a Python library for topic modeling, document indexing, and similarity retrieval on large text corpora, using algorithms like LDA, LSA, word2vec, and others.

Yes, with conditions. Gensim is production-stable and widely used for topic modeling and document similarity tasks, with no known vulnerabilities and broad Python version support. However, the aging maintenance status (300 days since last release) suggests the project is in steady-state rather than actively developed; evaluate whether its algorithms and API meet your needs before committing. Install friction is moderate due to compiled dependencies, but prebuilt wheels ease setup. The LGPL-2.1 copyleft license requires careful review if you plan proprietary distribution.

Install

gensim on PyPI

pip install gensim

uv add gensim

poetry add gensim

Installing gensim

Before you install

Medium install friction due to compiled dependencies (numpy, scipy); prebuilt wheels available for Python 3.9–3.13 on macOS, Linux, and Windows. Maintenance status is aging (300 days since last release), but the repository remains active with 16480 stars and continuous testing.

License in practice

Gensim is released under LGPL-2.1-only (copyleft). You may use it freely in open-source projects, but any modifications must be shared under the same license; closed-source or proprietary use requires careful review of your distribution model.

Quickstart

pip install gensim

import gensim.models
from gensim.corpora import Dictionary

# Create a dictionary and corpus from your documents
dictionary = Dictionary(texts)
corpus = [dictionary.doc2bow(text) for text in texts]

# Train an LDA model
model = gensim.models.LdaModel(corpus, num_topics=10, id2word=dictionary)

Requires numpy and scipy to be installed first; a fast BLAS library (MKL, ATLAS, or OpenBLAS) is recommended for performance but optional.

Verify before relying

• Whether the aging maintenance status (300 days since release) signals reduced active development or is typical for a stable, mature library.
• Current performance characteristics compared to newer topic modeling alternatives in the ecosystem.

Package facts

Evidence: gensim-4.4.0-cp310-cp310-macosx_10_9_x86_64.whl; gensim-4.4.0-cp310-cp310-macosx_11_0_arm64.whl; gensim-4.4.0-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; gensim-4.4.0-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; gensim-4.4.0-cp310-cp310-win_amd64.whl; gensim-4.4.0-cp311-cp311-macosx_10_9_x86_64.whl; gensim-4.4.0-cp311-cp311-macosx_11_0_arm64.whl; gensim-4.4.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; gensim-4.4.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; gensim-4.4.0-cp311-cp311-win_amd64.whl; gensim-4.4.0-cp312-cp312-macosx_10_13_x86_64.whl; gensim-4.4.0-cp312-cp312-macosx_11_0_arm64.whl; gensim-4.4.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; gensim-4.4.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; gensim-4.4.0-cp312-cp312-win_amd64.whl; gensim-4.4.0-cp313-cp313-macosx_10_13_x86_64.whl; gensim-4.4.0-cp313-cp313-macosx_11_0_arm64.whl; gensim-4.4.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; gensim-4.4.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; gensim-4.4.0-cp313-cp313-win_amd64.whl

Keywords: Singular Value Decomposition, SVD, Latent Semantic Indexing, LSA, LSI, Latent Dirichlet Allocation, LDA, Hierarchical Dirichlet Process, HDP, Random Projections, TFIDF, word2vec

Tags

More Artificial Intelligence packages