delta-spark

Python APIs for using Delta Lake with Apache Spark

delta-spark Permissive license Apache-2.0 Active ★ 8,938 v4.3.1 released 2026-07-08

Install

delta-spark on PyPI

pip

pip install delta-spark

uv

uv add delta-spark

poetry

poetry add delta-spark

Package facts

License	Apache-2.0 (permissive)
Python support	supports the current Python release (>=3.10)
Install friction	low — pure-Python wheel
Runtime dependencies	2 — pyspark, importlib_metadata
Maintenance	actively maintained — 36 days since the last release
Last repo commit	2026-08-13
First released	2021-05-20
Popularity	one of the top 1,000 most-downloaded packages on PyPI (30-day window, as of 2026-08-13)
Known vulnerabilities	none known (OSV.dev, checked 2026-08-13)

Evidence: delta_spark-4.3.1-py3-none-any.whl

Keywords: delta.io

Development Status :: 5 - Production/StableIntended Audience :: DevelopersLicense :: OSI Approved :: Apache Software LicenseOperating System :: OS IndependentProgramming Language :: Python :: 3Topic :: Software Development :: Libraries :: Python ModulesTyping :: Typed

About delta-spark

from the package's own PyPI description — quoted content, verbatim

Delta Lake

Delta Lake is an open source storage layer that brings reliability to data lakes. Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. Delta Lake runs on top of your existing data lake and is fully compatible with Apache Spark APIs.

This PyPi package contains the Python APIs for using Delta Lake with Apache Spark.

Installation and usage

Install using pip install delta-spark
To use the Delta Lake with Apache Spark, you have to set additional configurations when creating the SparkSession. See the online project web page for details.

Documentation

This README file only contains basic information related to pip installed Delta Lake. You can find the full documentation on the project web page

Read as markdown · JSON record · Source repository · Homepage · Docs

AI interpretation — verify before relying

AI-generated interpretation of the package facts above; every digit, version, license, or vulnerability id it cites is grounded in the facts already shown on this page

Delta Lake provides ACID transactions and unified streaming/batch processing on top of Apache Spark, bringing reliability and metadata handling to data lakes.

Active maintenance with a recent release 36 days ago and 8938 repository stars. Low install friction with only two runtime dependencies (pyspark and importlib_metadata), both lightweight.

Licensed under Apache-2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions beyond attribution and liability disclaimers.

Usage

pip install delta-spark

from delta.tables import DeltaTable
import pyspark.sql as sql

spark = sql.SparkSession.builder.appName("DeltaApp").getOrCreate()
delta_table = DeltaTable.forPath(spark, "/path/to/delta/table")

Requires Python >=3.10 and Apache Spark to be installed and configured; SparkSession must be initialized with Delta Lake configurations as documented on delta.io.

Verdict: Production-stable Delta Lake integration for Spark with active maintenance, permissive licensing, and low install friction. No known vulnerabilities and well-established in the top 1000 PyPI packages. Suitable for reliable data lake workloads requiring ACID guarantees.

Needs verification

Whether pyspark dependency version constraints are compatible with current Spark releases
Performance characteristics and scalability limits for very large metadata operations

spark acid transactionsdelta lake pythonstreaming batch processing sparkdata lake reliabilityspark metadata managementdelta io integrationapache spark storage layer