fasttext

What it is and what it does

fastText is a C++ library wrapped for Python that learns dense word representations (embeddings) and trains text classifiers. It uses subword information to build word vectors and supports both unsupervised learning (skipgram and CBOW models) and supervised text classification. The library is designed for efficiency and can handle large datasets.

You use fastText by calling train_unsupervised() to learn word vectors from raw text, or train_supervised() to train a classifier from labeled text. Once trained, you can retrieve word vectors, make predictions on new text, evaluate model performance, and save/load models to disk. The library requires numpy, pybind11, and setuptools as runtime dependencies. Input text must be UTF-8 encoded, and the library tokenizes on ASCII whitespace and control characters.

Use it for:

• Build word embeddings from large text corpora for downstream NLP tasks like clustering or similarity search.
• Train a text classifier to categorize documents or messages into predefined labels with minimal preprocessing.
• Compress trained classifier models using quantization to reduce file size for production deployment.
• Generate sentence representations by averaging subword vectors for fast semantic similarity comparisons.
• Experiment with different embedding models (skipgram vs. CBOW) on custom datasets without external dependencies.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

fastText is a library for learning word embeddings and training text classifiers using efficient algorithms based on subword information and bag-of-words methods.

Yes, if you need lightweight word embeddings or text classification and can tolerate an archived codebase. The library is stable and widely used, with no known vulnerabilities, but expect no new features or bug fixes. Install friction is moderate due to C++ compilation; pre-built wheels may not cover all platforms. Suitable for production use of existing models or training on stable datasets, but not for active development or cutting-edge NLP research.

Install

fasttext on PyPI

pip install fasttext

uv add fasttext

poetry add fasttext

Installing fasttext

Before you install

Medium install friction due to C++11 compilation requirements via pybind11. The repository is archived and the last commit was 2024-03-22, indicating the project is no longer actively maintained, though it remains functional for existing use cases.

License in practice

MIT license is permissive, allowing commercial and private use with minimal restrictions—you may use, modify, and distribute fastText freely provided you include the license notice.

Quickstart

pip install fasttext

import fasttext

# Train unsupervised word vectors
model = fasttext.train_unsupervised('data.txt', model='skipgram')
print(model['king'])  # retrieve word vector

# Or train supervised text classifier
model = fasttext.train_supervised('data.train.txt')
model.predict("sample text")

Requires a C++11-capable compiler and pybind11 for compilation during installation. UTF-8 encoded input text is mandatory; text must be unicode (Python 2) or str (Python 3).

Verify before relying

• Current compatibility with Python versions beyond 3.6 (classifiers list 3.4–3.6 but latest release is 2024-06-12)
• Whether pre-built wheels are available for all major platforms or if compilation is typically required

Package facts

Evidence: fasttext-0.9.3-cp39-cp39-macosx_14_0_arm64.whl

Tags

More Software Development packages