---
id: koalas
version: "1.8.2"
license: http://www.apache.org/licenses/LICENSE-2.0
license_treatment: permissive
maintenance: dormant
---

# koalas — Koalas: pandas API on Apache Spark

License: permissive · Maintenance: dormant · Downloads: 1.1M/mo

## What it is and what it does

Koalas bridges pandas and Apache Spark by exposing a pandas-compatible DataFrame API that executes on Spark clusters. If you know pandas, you can write similar code against Koalas and have it run on distributed data without learning Spark's native API. The package wraps Spark's distributed execution while mimicking pandas' single-node interface, making it useful for teams that want to scale pandas workflows to big data without rewriting code.

However, Koalas is now in maintenance mode and officially superseded by PySpark's native pandas API layer in Spark 3.2+. The last release was in October 2021, and the repository has seen minimal activity since. For new projects targeting Spark 3.2 or later, the description recommends using PySpark directly instead.

Use it for:

- Scale existing pandas code to distributed Spark clusters without rewriting logic or learning Spark's native API
- Write a single codebase that works with pandas on small datasets (for testing) and Koalas on large Spark clusters
- Migrate legacy pandas workflows to big data infrastructure when locked into Spark 3.1 or below
- Prototype data transformations in pandas, then deploy them on Spark using Koalas with minimal code changes

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Koalas implements the pandas DataFrame API on top of Apache Spark, letting you write pandas-like code that runs on distributed Spark clusters instead of a single machine.

No, unless you are locked into Spark 3.1 or below and cannot upgrade. Koalas is dormant (last release October 2021, minimal commits since) and officially deprecated in favor of PySpark's native pandas API layer in Spark 3.2+. For new projects or upgradeable environments, use PySpark directly. For legacy systems on Spark 3.1, Koalas remains functional but will not receive updates.

## Install

pip install koalas
uv add koalas
poetry add koalas

## Installing koalas

Before you install: Low install friction with a pure-Python wheel. However, the package is dormant—last release was 2021-10-19 and last commit 2024-03-20—and the description explicitly states it is in maintenance mode, superseded by PySpark's native pandas API layer in Spark 3.2+.

License in practice: Licensed under Apache License 2.0 (permissive), which allows commercial and private use with minimal restrictions, though you must include a copy of the license and state significant changes.

Quickstart:

pip install koalas

import pandas as pd
import pyarrow
import numpy

pdf = pd.DataFrame({'x': range(3), 'y': ['a', 'b', 'b']})
# Convert to Koalas DataFrame and perform operations

Requires Apache Spark 3.1 or below; for Spark 3.2+, use PySpark directly. Also requires pandas, pyarrow, and numpy as runtime dependencies.

Verify before relying:

- Exact import path and API surface for Koalas DataFrame creation and operations
- Whether existing code will continue to work without modification if Spark is upgraded to 3.2+
- Compatibility with Python versions beyond 3.9 (classifiers support up to 3.9)

## Package facts

- License: http://www.apache.org/licenses/LICENSE-2.0 (permissive)
- Python support: capped_below_current
- Install friction: low
- Maintenance: dormant
- Downloads: 1.1M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

pandas api on spark, distributed dataframe spark, spark pandas compatibility, big data dataframe python, spark dataframe wrapper, pandas to spark migration, distributed pandas alternative, spark-integration, dataframe-api, deprecated

[View on SkillFed](https://skillfed.io/packages/koalas) · [View on PyPI](https://pypi.org/project/koalas/)