livy

What it is and what it does

Livy is a Python wrapper around Apache Livy, an open-source REST interface for Spark clusters. It lets you submit and manage Spark jobs from Python without direct cluster access, using HTTP requests under the hood. The package depends on pandas, requests, and dataclasses to handle data serialization, HTTP communication, and Python 3.6 compatibility.

The package is no longer maintained. Its last commit was 2022-01-05, and the latest release was 2021-01-27. If your Spark and Livy infrastructure has remained stable and compatible with that era, it may still work; otherwise, you risk incompatibility with newer cluster versions or missing security patches.

Use it for:

• Submit Python code to a remote Spark cluster for distributed execution without SSH or direct cluster access.
• Integrate Spark job submission into Python data pipelines or ETL workflows via REST.
• Prototype or test Spark jobs from a local development environment against a shared cluster.
• Build automation that triggers Spark computations from Python applications.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A Python client for Apache Livy that enables remote code execution on Spark clusters via Livy's REST interface.

No, unless you are locked into a legacy Spark/Livy setup from 2021 or earlier and cannot upgrade. The package is abandoned with no maintenance since 2022, no known vulnerabilities currently reported, but no assurance of compatibility with modern Spark or Livy versions. Consider an actively maintained alternative if possible.

Install

livy on PyPI

pip install livy

uv add livy

poetry add livy

Installing livy

Before you install

Low install friction with standard dependencies (pandas, requests, dataclasses), but the package is abandoned—last commit was 2022-01-05 and no releases since 2021-01-27. Maintenance has ceased for over three years.

License in practice

MIT license is permissive and imposes no restrictions on use, modification, or distribution in your own projects.

Quickstart

pip install -U livy

import livy
# Create a client and submit code to a Livy REST endpoint

Requires Python 3.6 or later and a running Livy REST interface endpoint.

Verify before relying

• Compatibility with recent Spark or Livy versions—last release predates current ecosystem standards by years.
• Whether dataclasses is a runtime dependency or a Python 3.6 compatibility shim no longer needed.
• Active community or fork status given the long maintenance gap.

Package facts

Evidence: livy-0.8.0-py3-none-any.whl

Tags

More Distributed Computing packages