{"categories":[{"label":"Monitoring","url":"https://skillfed.io/packages/category/system-monitoring/4"}],"enrichment":{"capability":"Integrates Apache Hadoop Distributed File System (HDFS) and WebHDFS with Apache Airflow, enabling workflows to read, write, and manage files on Hadoop clusters.","skillfed_tags":["airflow-provider","hadoop-integration","data-orchestration"],"use_cases":["Orchestrate ETL pipelines that read raw data from HDFS, transform it, and write results back to the cluster.","Monitor HDFS file availability as a sensor in Airflow DAGs before triggering downstream processing tasks.","Integrate Hadoop-based data lakes with Airflow workflows for scheduled batch processing and data movement.","Build data validation tasks that check file existence, size, or format on HDFS before proceeding with pipeline stages."],"what_it_does":"This is an Apache Airflow provider package that adds HDFS and WebHDFS connectivity to Airflow workflows. It extends Airflow with operators, hooks, and sensors for interacting with Hadoop Distributed File System clusters, allowing you to read, write, and manage files as part of data pipelines. The package depends on apache-airflow, the hdfs client library, fastavro for Avro serialization, and pandas for dataframe operations.\n\nThe package is designed for teams running Airflow orchestration on top of Hadoop infrastructure. It abstracts away the complexity of HDFS client configuration and integrates directly into Airflow's task execution model, so you can define file operations declaratively in your DAGs. It supports Python 3.10 through 3.14 and is actively maintained by the Apache Airflow project.","worth_installing":"Yes, if you run Apache Airflow and need to orchestrate tasks on a Hadoop cluster. The package is actively maintained, has no known vulnerabilities, installs with low friction, and is part of the official Airflow provider ecosystem. Install it only if you have an HDFS cluster to connect to; it is not useful as a standalone tool."},"id":"apache-airflow-providers-apache-hdfs","links":{"html":"https://skillfed.io/packages/apache-airflow-providers-apache-hdfs","md":"https://skillfed.io/packages/apache-airflow-providers-apache-hdfs.md","pypi":"https://pypi.org/project/apache-airflow-providers-apache-hdfs/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-08","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"apache-airflow-providers-apache-hdfs","python_support":"supports_current","summary":"Provider package apache-airflow-providers-apache-hdfs for Apache Airflow"},"popularity":{"monthly_downloads":249279,"position":8652,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"4.12.2"}