sagemaker-feature-store-pyspark

What it is and what it does

This is a PySpark connector for Amazon SageMaker FeatureStore that lets you ingest data from Spark DataFrames directly into FeatureStore's online and offline stores. It provides a main interface for operations like data ingestion, feature definition loading from schema, and retrieval of failed ingestion records. The package bundles pre-built JARs for Spark versions 3.1 through 3.5 and automatically selects the correct one based on your installed PySpark version.

The connector is designed for use in Spark environments, including AWS EMR clusters and SageMaker Notebook instances. It supports optional Lake Formation credentials for cross-account access when the offline store is registered with AWS Lake Formation (PySpark 3.5+ only). The package requires PySpark and NumPy to be pre-installed, and compatibility between Python and PySpark versions is constrained by a documented matrix.

Use it for:

• Ingest training data from Spark DataFrames into SageMaker FeatureStore offline store to reduce WCU costs.
• Load feature definitions from a Spark DataFrame schema for use with the SageMaker CreateFeatureGroup API.
• Retrieve and debug records that failed during streaming ingestion to FeatureStore.
• Build feature pipelines on EMR clusters that feed data directly into SageMaker FeatureStore.
• Access offline store data via Lake Formation when registered with AWS Lake Formation (PySpark 3.5+).

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Ingest Spark DataFrames into Amazon SageMaker FeatureStore's online and offline stores, and load feature definitions from schema.

Yes. Low install friction, active maintenance, permissive Apache-2.0 license, no known vulnerabilities, and a focused feature set for SageMaker FeatureStore integration. Install if you need to ingest Spark DataFrames into FeatureStore; verify Python/PySpark version compatibility against the documented matrix before proceeding.

Install

sagemaker-feature-store-pyspark on PyPI

pip install sagemaker-feature-store-pyspark

uv add sagemaker-feature-store-pyspark

poetry add sagemaker-feature-store-pyspark

Installing sagemaker-feature-store-pyspark

Before you install

Low friction install with only setuptools as a runtime dependency. Active maintenance as of 2026-04-29. Requires PySpark and NumPy to be pre-installed; the package bundles pre-built JARs for Spark versions 3.1–3.5 and selects the correct one at runtime.

License in practice

Licensed under Apache-2.0 (permissive), allowing use in commercial and proprietary projects with minimal restrictions.

Quickstart

pip install sagemaker-feature-store-pyspark

from sagemaker_feature_store_pyspark import FeatureStoreManager

feature_store_manager = FeatureStoreManager()
feature_store_manager.ingest_data(
    input_data_frame=df,
    feature_group_arn="arn:aws:sagemaker:...:feature-group/your-feature-group",
    target_stores=["OfflineStore"]
)

PySpark (3.1–3.5) and NumPy must be installed before this package. Python version must match the PySpark version according to the compatibility matrix (e.g., Python 3.10 requires PySpark 3.2+).

Verify before relying

• Whether the package works with PySpark versions outside the 3.1–3.5 range or with Python versions outside 3.8–3.12.
• Performance characteristics and scalability limits for large DataFrame ingestion.
• Whether Lake Formation support (PySpark 3.5+ only) is production-ready or still experimental.

Package facts

Evidence: sagemaker_feature_store_pyspark-2.0.0-py3-none-any.whl

Keywords: ML, Amazon, AWS, AI, FeatureStore, SageMaker

Tags

More Artificial Intelligence packages