fasttext

What it is and what it does

fastText is a C++ library wrapped for Python that learns dense word representations (embeddings) and trains text classifiers. It uses subword information to build word vectors and supports both unsupervised learning (skipgram and CBOW models) and supervised text classification. The library is designed for efficiency and can handle large datasets.

You use fastText by calling train_unsupervised() to learn word vectors from raw text, or train_supervised() to train a classifier from labeled text. Once trained, you can retrieve word vectors, make predictions on new text, evaluate model performance, and save/load models to disk. The library requires numpy, pybind11, and setuptools as runtime dependencies. Input text must be UTF-8 encoded, and the library tokenizes on ASCII whitespace and control characters.

Use it for

• Build word embeddings from large text corpora for downstream NLP tasks like clustering or similarity search.
• Train a text classifier to categorize documents or messages into predefined labels with minimal preprocessing.
• Compress trained classifier models using quantization to reduce file size for production deployment.
• Generate sentence representations by averaging subword vectors for fast semantic similarity comparisons.
• Experiment with different embedding models (skipgram vs. CBOW) on custom datasets without external dependencies.

Worth the install?

AI-flagged interpretation of the facts on this page. Verify before relying on it.

Install

fasttext on PyPI

Before you install

Medium install friction due to C++11 compilation requirements via pybind11. The repository is archived and the last commit was 2024-03-22, indicating the project is no longer actively maintained, though it remains functional for existing use cases.

Requires a C++11-capable compiler and pybind11 for compilation during installation. UTF-8 encoded input text is mandatory; text must be unicode (Python 2) or str (Python 3).

License in practice

MIT license is permissive, allowing commercial and private use with minimal restrictions—you may use, modify, and distribute fastText freely provided you include the license notice.

Quickstart

pip install fasttext

import fasttext

# Train unsupervised word vectors
model = fasttext.train_unsupervised('data.txt', model='skipgram')
print(model['king'])  # retrieve word vector

# Or train supervised text classifier
model = fasttext.train_supervised('data.train.txt')
model.predict("sample text")

Package facts

Evidence: fasttext-0.9.3-cp39-cp39-macosx_14_0_arm64.whl

Tags

More Software Development packages

See also fasttext-numpy2 · fasttext-predict · fasttext-wheel · floret · fast-langdetect · fasttext-langdetect · model2vec · setfit · alt-profanity-check · sentence-transformers