{"categories":[{"label":"Information Analysis","url":"https://skillfed.io/packages/category/scientific-engineering-information-analysis"}],"enrichment":{"capability":"Koalas implements the pandas DataFrame API on top of Apache Spark, letting you write pandas-like code that runs on distributed Spark clusters instead of a single machine.","skillfed_tags":["spark-integration","dataframe-api","deprecated"],"use_cases":["Scale existing pandas code to distributed Spark clusters without rewriting logic or learning Spark's native API","Write a single codebase that works with pandas on small datasets (for testing) and Koalas on large Spark clusters","Migrate legacy pandas workflows to big data infrastructure when locked into Spark 3.1 or below","Prototype data transformations in pandas, then deploy them on Spark using Koalas with minimal code changes"],"what_it_does":"Koalas bridges pandas and Apache Spark by exposing a pandas-compatible DataFrame API that executes on Spark clusters. If you know pandas, you can write similar code against Koalas and have it run on distributed data without learning Spark's native API. The package wraps Spark's distributed execution while mimicking pandas' single-node interface, making it useful for teams that want to scale pandas workflows to big data without rewriting code.\n\nHowever, Koalas is now in maintenance mode and officially superseded by PySpark's native pandas API layer in Spark 3.2+. The last release was in October 2021, and the repository has seen minimal activity since. For new projects targeting Spark 3.2 or later, the description recommends using PySpark directly instead.","worth_installing":"No, unless you are locked into Spark 3.1 or below and cannot upgrade. Koalas is dormant (last release October 2021, minimal commits since) and officially deprecated in favor of PySpark's native pandas API layer in Spark 3.2+. For new projects or upgradeable environments, use PySpark directly. For legacy systems on Spark 3.1, Koalas remains functional but will not receive updates."},"id":"koalas","links":{"html":"https://skillfed.io/packages/koalas","md":"https://skillfed.io/packages/koalas.md","pypi":"https://pypi.org/project/koalas/"},"maintenance":{"status":"dormant"},"meta":{"latest_release":"2021-10-19","license_spdx":null,"license_treatment":"permissive","name":"koalas","python_support":"capped_below_current","summary":"Koalas: pandas API on Apache Spark"},"popularity":{"monthly_downloads":1084450,"position":4389,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"1.8.2"}