---
id: clean-text
version: "0.7.1"
license: Apache-2.0
license_treatment: permissive
maintenance: aging
---

# clean-text — Functions to preprocess and normalize text.

License: permissive · Maintenance: aging · Downloads: 317.6K/mo

## What it is and what it does

clean-text is a text preprocessing library that normalizes messy user-generated content from the web and social media. It fixes unicode encoding errors, removes or replaces unwanted patterns (URLs, emails, phone numbers, IP addresses, code snippets), handles transliteration to ASCII, and supports language-specific rules for English and German. The package wraps ftfy for unicode repair and optionally uses unidecode for transliteration, falling back to Python's built-in unicodedata when unidecode is unavailable.

The library offers both a simple functional API (clean() for single strings, clean_texts() for batch processing with optional multiprocessing) and a scikit-learn compatible transformer for integration into ML pipelines. You can preserve specific text patterns using regex-based exceptions to prevent them from being modified during cleaning, and customize replacement tokens for different content types.

Use it for:

- Normalize scraped web or social media text before feeding it into NLP models or search indexing.
- Batch-clean large text datasets in parallel using n_jobs to speed up preprocessing on multi-core systems.
- Standardize user input in web applications by removing malformed unicode, URLs, and email addresses while preserving legitimate content.
- Prepare text for machine learning pipelines via the scikit-learn CleanTransformer for reproducible preprocessing steps.
- Transliterate accented or non-ASCII characters to ASCII equivalents for systems that require ASCII-only text.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Preprocesses and normalizes user-generated text by fixing unicode errors, removing or replacing URLs, emails, code snippets, and other patterns, with optional transliteration to ASCII and support for parallel batch processing.

Yes, if you need to clean messy user-generated text before NLP or indexing. The package is stable, permissively licensed, and has low install friction. The aging maintenance status (198 days since last release) is a minor concern but not a blocker for a mature text-cleaning utility. Consider it a solid choice for preprocessing pipelines, especially if you already use scikit-learn.

## Install

pip install clean-text
uv add clean-text
poetry add clean-text

## Installing clean-text

Before you install: Low friction install with two lightweight runtime dependencies (emoji and ftfy). Last release was 198 days ago; maintenance status is aging but the package remains functional for its core use case.

License in practice: Licensed under Apache-2.0 (permissive), allowing commercial and private use with minimal restrictions. No GPL obligation unless you explicitly install the optional unidecode extra.

Quickstart:

pip install clean-text

from cleantext import clean

result = clean("Yóù àré right!", fix_unicode=True, to_ascii=True, lower=True)

Requires Python 3.9 or later. Optional unidecode dependency (installed via clean-text[gpl]) provides better transliteration but carries GPL licensing; without it, falls back to Python's built-in unicodedata with slightly different output.

Verify before relying:

- Whether the aging maintenance status (198 days since last release) affects bug fixes or feature requests going forward.
- Real-world performance of parallel processing (n_jobs parameter) on typical workloads.
- Consistency and quality differences between transliteration with and without unidecode in production use.

## Package facts

- License: Apache-2.0 (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: aging
- Downloads: 317.6K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

text normalization preprocessing, clean user generated content, unicode text fixing, remove urls emails from text, text sanitization pipeline, transliterate unicode to ascii, batch text cleaning, text-preprocessing, nlp-utilities, data-cleaning

[View on SkillFed](https://skillfed.io/packages/clean-text) · [View on PyPI](https://pypi.org/project/clean-text/)