---
id: tensorflow-transform
version: "1.21.0"
license: Apache 2.0
license_treatment: permissive
maintenance: active
---

# tensorflow-transform — A library for data preprocessing with TensorFlow

License: permissive · Maintenance: active · Downloads: 187.3K/mo

## What it is and what it does

TensorFlow Transform is a preprocessing library that extends TensorFlow's single-example capabilities to support full-pass operations over entire datasets. It handles transformations that require seeing all data—such as computing mean and standard deviation for normalization, building vocabularies from all unique values, or assigning data to quantile-based buckets—then exports the result as a frozen TensorFlow graph for reuse in both training and serving pipelines.

The library uses Apache Beam for distributed computation (defaulting to local mode but supporting Google Cloud Dataflow and other runners) and PyArrow for efficient vectorized operations. By applying identical transformations at train and serve time, it eliminates training-serving skew, a common source of model degradation in production.

Use it for:

- Normalize numerical features by computing global mean and standard deviation across the entire training dataset.
- Generate a vocabulary from all unique string values in a column and map strings to integer IDs consistently.
- Assign continuous values to discrete buckets based on observed data quantiles or distribution.
- Preprocess data pipelines for TensorFlow Extended (TFX) workflows with distributed Apache Beam runners.
- Export preprocessing logic as a reusable TensorFlow graph to serve alongside trained models.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

TensorFlow Transform preprocesses data with full-pass operations like normalization, vocabulary generation, and bucketing, exporting a reusable TensorFlow graph for consistent training and serving.

Yes. TensorFlow Transform is production-stable, actively maintained, and essential for building ML pipelines that require full-pass preprocessing or consistent train-serve transformations. Install if you need stateful data transformations (normalization, vocabulary generation, bucketing) in TensorFlow workflows; skip if your preprocessing fits single-example operations.

## Install

pip install tensorflow-transform
uv add tensorflow-transform
poetry add tensorflow-transform

## Installing tensorflow-transform

Before you install: Low friction installation as a pure-Python wheel. Active maintenance with a recent release (64 days old) and ongoing repository activity. Requires TensorFlow and Apache Beam as core dependencies.

License in practice: Licensed under Apache 2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions.

Quickstart:

pip install tensorflow-transform

import tensorflow_transform as tft

# Use tft.scale_to_z_score() or tft.compute_and_apply_vocabulary()
# within a tf.Transform preprocessing function

Requires TensorFlow, Apache Beam, and PyArrow as runtime dependencies; Python 3.10 or later.

Verify before relying:

- Whether the package works with Google Cloud Dataflow or other Apache Beam runners without additional configuration.
- Performance characteristics and scalability limits for very large datasets.
- Compatibility with custom Apache Beam runners beyond the default local mode.

## Package facts

- License: Apache 2.0 (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 187.3K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

tensorflow data preprocessing, full-pass data transformation, normalize and bucket data, vocabulary generation pipeline, training serving skew prevention, apache beam tensorflow, distributed data preprocessing, ml-preprocessing, distributed-computing, tensorflow-ecosystem

[View on SkillFed](https://skillfed.io/packages/tensorflow-transform) · [View on PyPI](https://pypi.org/project/tensorflow-transform/)