{"categories":[{"label":"Utilities","url":"https://skillfed.io/packages/category/utilities/3"},{"label":"Application Frameworks","url":"https://skillfed.io/packages/category/software-development-libraries-application-frameworks/2"}],"enrichment":{"capability":"Provides a pure Python HDFS client library and command-line interface for communicating with Hadoop NameNodes and reading data from DataNodes using protobuf.","skillfed_tags":["hadoop","distributed-storage"],"use_cases":["Build Python data pipelines that read or write files directly to HDFS clusters without shelling out to commands.","Automate HDFS file management tasks (listing, checking, reading) from Python scripts in a Hadoop environment.","Integrate HDFS access into Python applications running on or near a Hadoop cluster for data processing workflows.","Replace shell-based HDFS access in lightweight Python tools that need to communicate with NameNodes."],"what_it_does":"Snakebite-py3 is a pure Python HDFS client that communicates with Hadoop NameNodes and DataNodes using protobuf. It provides both a library API and a command-line tool for interacting with HDFS clusters. The package is a Python 3 fork maintained by Internet Archive, originally from Spotify, and is designed to work with modern Hadoop versions (2.2.0 and up). It supports most NameNode operations and can read data from DataNodes, with optional CRC checking during transfer (disabled by default for performance).\n\nThe library depends on protobuf for serialization and argparse for CLI argument handling. It has been tested primarily against Cloudera CDH5 and Hortonworks HDP2.0 clusters. The package is useful for Python-based applications that need direct programmatic access to HDFS without relying on shell commands.","worth_installing":"Yes, if you need programmatic HDFS access from Python and are running Hadoop 2.2.0 or later (CDH5 or HDP2.0+). The package has low install friction, permissive licensing, and no known vulnerabilities. However, maintenance is aging (last release 542 days ago), so verify that it works with your specific Hadoop version before committing to production use."},"id":"snakebite-py3","links":{"html":"https://skillfed.io/packages/snakebite-py3","md":"https://skillfed.io/packages/snakebite-py3.md","pypi":"https://pypi.org/project/snakebite-py3/"},"maintenance":{"status":"aging"},"meta":{"latest_release":"2025-02-18","license_spdx":null,"license_treatment":"permissive","name":"snakebite-py3","python_support":"unspecified","summary":"Pure Python HDFS client"},"popularity":{"monthly_downloads":1714136,"position":3624,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"3.0.6"}