{"categories":[{"label":"Python Modules","url":"https://skillfed.io/packages/category/software-development-libraries-python-modules/6"}],"enrichment":{"capability":"SparkMeasure provides a Python API to collect and analyze Apache Spark performance metrics, enabling interactive troubleshooting and batch job analysis of Spark workloads.","skillfed_tags":["spark-performance","metrics-collection","distributed-computing"],"use_cases":["Diagnose slow PySpark jobs by collecting task-level timing and resource metrics in a Jupyter notebook","Batch analysis of Spark workload performance using Flight Recorder mode to capture and replay metrics","Monitor Spark job metrics in real time and export them to InfluxDB or Prometheus for dashboarding","Understand shuffle, I/O, and memory behavior of large-scale Spark applications","Educational exploration of Spark's Listener interface and internal performance instrumentation"],"what_it_does":"SparkMeasure is a Python wrapper around a Scala-based performance instrumentation tool for Apache Spark. It lets you collect detailed runtime metrics from Spark jobs\u2014task execution times, shuffle metrics, memory usage, and other performance indicators\u2014either interactively in notebooks or in batch mode. The package is designed for developers and data engineers who need to diagnose slow or inefficient Spark workloads, understand resource consumption, and validate performance improvements.\n\nThe tool integrates with Spark's Listener interface to capture low-level execution events without modifying your job code significantly. It can export metrics to external systems like InfluxDB, Kafka, or Prometheus for centralized monitoring, and also serves as an educational resource for understanding how Spark's internal metrics work. Since the bulk implementation is in Scala, it runs efficiently within the JVM alongside your Spark jobs.","worth_installing":"Yes, if you work with Apache Spark and need performance visibility. Low install friction, no runtime dependencies, active maintenance, and permissive licensing make it a practical choice. Verify that your Spark environment and use case align with its requirements before adopting."},"id":"sparkmeasure","links":{"html":"https://skillfed.io/packages/sparkmeasure","md":"https://skillfed.io/packages/sparkmeasure.md","pypi":"https://pypi.org/project/sparkmeasure/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-05-19","license_spdx":null,"license_treatment":"permissive","name":"sparkmeasure","python_support":"supports_current","summary":"Python API for sparkMeasure, a tool for performance troubleshooting of Apache Spark workloads."},"popularity":{"monthly_downloads":2320115,"position":3142,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"0.28.0"}