{"categories":[{"label":"Distributed Computing","url":"https://skillfed.io/packages/category/system-distributed-computing"}],"enrichment":{"capability":"Provides utility functions and Jupyter integration helpers for the Sparkmagic project, enabling interactive Spark cluster access through Jupyter notebooks via REST-based Livy, Lighter, or Ilum servers.","skillfed_tags":["jupyter-integration","spark-utilities","distributed-computing"],"use_cases":["Supporting Sparkmagic's Jupyter kernel implementations to enable automatic Spark context creation and SQL query execution.","Providing shared utility functions across Sparkmagic's multiple authentication methods (Basic, Kerberos, custom).","Enabling visualization and dataframe capture workflows in Jupyter notebooks connected to remote Spark clusters.","Facilitating local-to-remote data transfer and Spark job configuration management from Jupyter.","Abstracting REST communication details when interacting with Livy or other Spark session servers."],"what_it_does":"hdijupyterutils is a utility library that underpins the Sparkmagic project, providing helper functions and Jupyter integration scaffolding for working with remote Spark clusters. It is designed to be used alongside Jupyter notebooks and the IPython ecosystem, offering utilities that abstract common operations needed when connecting to Spark via REST servers like Livy, Lighter, or Ilum.\n\nThe package depends on seven runtime components\u2014ipython, ipywidgets, ipykernel, jupyter, pandas, numpy, and notebook\u2014reflecting its deep integration with the Jupyter data science stack. It is primarily a support library; most users encounter it as a transitive dependency when installing Sparkmagic itself rather than as a standalone tool. The BSD 3-clause license permits unrestricted use in commercial and private projects.","worth_installing":"Yes, but typically as a transitive dependency of Sparkmagic rather than directly. If you are setting up Sparkmagic for interactive Spark work in Jupyter, this will be installed automatically. Install it explicitly only if you are developing Sparkmagic extensions or utilities. The aging maintenance status (403 days since last release) warrants monitoring, but no security vulnerabilities are known and the BSD license poses no restrictions."},"id":"hdijupyterutils","links":{"html":"https://skillfed.io/packages/hdijupyterutils","md":"https://skillfed.io/packages/hdijupyterutils.md","pypi":"https://pypi.org/project/hdijupyterutils/"},"maintenance":{"status":"aging"},"meta":{"latest_release":"2025-07-07","license_spdx":null,"license_treatment":"permissive","name":"hdijupyterutils","python_support":"unspecified","summary":"HdiJupyterUtils: Utils for Jupyter projects from HDInsight team"},"popularity":{"monthly_downloads":1609412,"position":3722,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"0.23.0"}