{"categories":[{"label":"Software Development","url":"https://skillfed.io/packages/category/software-development/3"},{"label":"Libraries","url":"https://skillfed.io/packages/category/software-development-libraries/2"},{"label":"Scientific/Engineering","url":"https://skillfed.io/packages/category/scientific-engineering/2"},{"label":"Python Modules","url":"https://skillfed.io/packages/category/software-development-libraries-python-modules/4"},{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/2"}],"enrichment":{"capability":"TensorFlow Text provides text preprocessing operations and tokenizers that run within the TensorFlow computation graph, enabling consistent text handling across training and inference without external preprocessing scripts.","skillfed_tags":["nlp","tensorflow-ecosystem","graph-based-preprocessing"],"use_cases":["Normalize and tokenize text within a TensorFlow model pipeline to ensure consistent preprocessing at training and serving time.","Split multilingual or punctuation-heavy text using Unicode script boundaries instead of simple whitespace splitting.","Extract byte offsets during tokenization to trace tokens back to their original positions in source text.","Build end-to-end NLP models in Keras that accept raw text and produce token sequences without external preprocessing.","Handle non-ASCII Unicode text with case folding and normalization forms (NFKC, NFKD) for case-insensitive matching."],"what_it_does":"TensorFlow Text is a library of text-related operations and tokenizers designed to run within TensorFlow's computation graph. It provides utilities for Unicode normalization, case folding, tokenization (whitespace, Unicode script-based, and character-level), n-gram generation, and other text preprocessing tasks commonly needed for NLP models. The key advantage is that all preprocessing happens inside the TensorFlow graph, ensuring identical behavior between training and inference without needing separate preprocessing scripts.\n\nThe library depends solely on TensorFlow and is distributed as precompiled wheels for multiple Python versions (3.9\u20133.13) and architectures (x86_64, aarch64, macOS ARM64). It handles UTF-8 strings natively and supports operations like byte offset tracking during tokenization, integration with tf.data pipelines, and Keras API compatibility for ragged tensor inputs.","worth_installing":"Yes. TensorFlow Text is actively maintained, widely used (top 5000 PyPI packages), has no known vulnerabilities, and solves a real problem\u2014keeping text preprocessing inside the TensorFlow graph. Install it if you are building NLP models with TensorFlow and need tokenization or text normalization; ensure your tensorflow-text version matches your TensorFlow minor version."},"id":"tensorflow-text","links":{"html":"https://skillfed.io/packages/tensorflow-text","md":"https://skillfed.io/packages/tensorflow-text.md","pypi":"https://pypi.org/project/tensorflow-text/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-03-10","license_spdx":null,"license_treatment":"permissive","name":"tensorflow-text","python_support":"unspecified","summary":"TF.Text is a TensorFlow library of text related ops, modules, and subgraphs."},"popularity":{"monthly_downloads":5745982,"position":2043,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"2.20.1"}