polars-cloud

What it is and what it does

Polars Cloud is a client library that bridges the open-source Polars DataFrame engine to a managed distributed compute service. It lets you write Polars queries using the familiar local API, then transparently execute them on remote infrastructure by calling `.remote()` on a query and providing a compute context. The library depends on polars and typing-extensions, and handles the serialization and orchestration needed to run your DataFrame operations at scale.

The package targets data engineers and analysts who want to scale beyond single-machine memory and CPU without rewriting their Polars code or learning a new API. It supports both CPU and GPU compute, works from notebooks, Airflow DAGs, Lambda functions, or any Python environment, and lets you monitor query progress through a dashboard. As of version 0.10.0, it is very early in its lifecycle (released 11 days ago), so production readiness and feature completeness should be validated before adoption.

Use it for:

• Scale Polars queries on large datasets stored in S3 or other cloud storage without rewriting code.
• Run compute-intensive aggregations and transformations from serverless environments like AWS Lambda.
• Execute parallel DataFrame operations from Airflow DAGs without managing Spark or Dask clusters.
• Prototype locally with Polars, then run the same query at scale on remote infrastructure.
• Combine multiple data sources and perform complex joins on datasets larger than local machine memory.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Polars Cloud extends the Polars DataFrame library to run queries on distributed remote infrastructure, allowing you to scale computations beyond a single machine while using the same Polars API.

Yes, with conditions. The package is permissively licensed and has no known vulnerabilities, making it safe to try. However, it is extremely new (0.10.0, released 11 days ago) with a 'Planning' development status, so treat it as early-stage software. Install if you want to experiment with distributed Polars execution and are willing to accept potential API changes; avoid if you need production-grade stability or have strict SLAs. Verify pricing, feature coverage, and performance characteristics before committing to it for critical workloads.

Install

polars-cloud on PyPI

pip install polars-cloud

uv add polars-cloud

poetry add polars-cloud

Installing polars-cloud

Before you install

Medium install friction: compiled wheels available for Python 3.10+ across macOS, Linux, and Windows architectures. Very recent release (11 days old) with active maintenance status, so long-term stability and community patterns remain unproven.

License in practice

MIT license (permissive) imposes no restrictions on use, modification, or distribution in commercial or private projects.

Quickstart

pip install polars_cloud

import polars as pl
import polars_cloud as pc

ctx = pc.ComputeContext(cpus=16, memory=64)
query = pl.scan_parquet("s3://data/").group_by("col").agg(pl.mean("val"))
query.remote(ctx).distributed().sink_parquet("s3://out/")

Requires Python 3.10 or later; remote execution requires a Polars Cloud account and compute context configuration.

Verify before relying

• Actual performance and cost characteristics of remote execution compared to local Polars or other distributed engines.
• Whether the distributed engine supports all Polars operations or has limitations on certain query patterns.
• Pricing model and free tier details for Polars Cloud compute resources.
• Integration maturity with common data sources and cloud storage systems beyond S3.

Package facts

Evidence: polars_cloud-0.10.0-cp310-abi3-macosx_10_12_x86_64.whl; polars_cloud-0.10.0-cp310-abi3-macosx_11_0_arm64.whl; polars_cloud-0.10.0-cp310-abi3-manylinux_2_28_aarch64.whl; polars_cloud-0.10.0-cp310-abi3-manylinux_2_28_x86_64.whl; polars_cloud-0.10.0-cp310-abi3-win_amd64.whl; polars_cloud-0.10.0-cp310-abi3-win_arm64.whl

Tags

More Scientific/Engineering packages