hf-xet
Fast transfer of large files with the Hugging Face Hub.
Decision gist · record as of 2026-08-14
Yes, if the parent package that depends on it requires it. It installs as a dependency automatically and requires no configuration. No vulnerabilities, active maintenance, and permissive license. Do not add it directly to project dependencies; let the parent package manage it.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.8 or later.
- Intended as an internal dependency, not for direct application use.
- Medium install friction due to compiled wheels for multiple platforms and Python versions.
License · maintenance · safety
Apache-2.0 (permissive) — Apache-2.0 permissive license allows commercial and private use with minimal restrictions.
last release 2026-08-03 (11 days) · last repo commit 2026-08-13 · 557 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 258,362,687 downloads/mo, #154 on PyPI
Alternatives
Verify before relying
pip install hf_xet
# hf-xet operates as an internal dependency and is not directly imported
# Its functionality is accessed through the parent package that depends on it- Specific performance gains or deduplication ratios compared to standard Git LFS
- Disk cache size limits and eviction policies
- Direct usage scenarios outside of the parent package context
What it is and what it does
hf-xet is a low-level storage client that extends file transfer capabilities with chunk-based deduplication and efficient caching. It implements xet storage, which breaks files into chunks, deduplicates them across binary files, and caches them locally on disk. The library provides Python bindings and handles concurrent communication with backend services.
This package is not meant to be used directly in application code. Instead, it operates as an internal dependency, transparently improving upload and download performance when invoked by its parent package. It maintains backward compatibility with Git LFS while adding chunk-level deduplication to reduce redundant storage and transfer.
Use it for
- Uploading large files with automatic chunk deduplication to reduce storage footprint
- Downloading files with local chunk-based caching to speed up repeated access
- Reducing redundancy when multiple files share common binary chunks
- Enabling faster incremental transfers when only portions of a large file change
- Leveraging concurrent transfers to backend services for improved throughput
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if the parent package that depends on it requires it.
It installs as a dependency automatically and requires no configuration. No vulnerabilities, active maintenance, and permissive license. Do not add it directly to project dependencies; let the parent package manage it.
Install
hf-xet on PyPI
Before you install
Medium install friction due to compiled wheels for multiple platforms and Python versions. Active maintenance with recent releases and no known vulnerabilities.
Requires Python 3.8 or later. Intended as an internal dependency, not for direct application use.
License in practice
Apache-2.0 permissive license allows commercial and private use with minimal restrictions.
Quickstart
pip install hf_xet
# hf-xet operates as an internal dependency and is not directly imported
# Its functionality is accessed through the parent package that depends on itVerify before relying
- Specific performance gains or deduplication ratios compared to standard Git LFS
- Disk cache size limits and eviction policies
- Direct usage scenarios outside of the parent package context
Package facts
| License | Apache-2.0 permissive |
| Python support | Supports the current Python release >=3.8 |
| Install friction | Medium. Platform-specific wheel |
| Runtime dependencies | None |
| Maintenance | Actively maintained 11 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 258,362,687 / month, #154 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableLicense :: OSI Approved :: Apache Software LicenseProgramming Language :: Python :: 3Programming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9Programming Language :: Python :: Free ThreadingProgramming Language :: Python :: Free Threading :: 2 - BetaProgramming Language :: Python :: Implementation :: CPythonProgramming Language :: Python :: Implementation :: PyPyProgramming Language :: RustTopic :: Scientific/Engineering :: Artificial Intelligence |
Evidence: hf_xet-1.6.0-cp314-cp314t-macosx_10_12_x86_64.whl; hf_xet-1.6.0-cp314-cp314t-macosx_11_0_arm64.whl; hf_xet-1.6.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl; hf_xet-1.6.0-cp314-cp314t-manylinux_2_28_aarch64.whl; hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_aarch64.whl; hf_xet-1.6.0-cp314-cp314t-musllinux_1_2_x86_64.whl; hf_xet-1.6.0-cp314-cp314t-win_amd64.whl; hf_xet-1.6.0-cp314-cp314t-win_arm64.whl; hf_xet-1.6.0-cp38-abi3-macosx_10_12_x86_64.whl; hf_xet-1.6.0-cp38-abi3-macosx_11_0_arm64.whl; hf_xet-1.6.0-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl; hf_xet-1.6.0-cp38-abi3-manylinux_2_28_aarch64.whl; hf_xet-1.6.0-cp38-abi3-musllinux_1_2_aarch64.whl; hf_xet-1.6.0-cp38-abi3-musllinux_1_2_x86_64.whl; hf_xet-1.6.0-cp38-abi3-win_amd64.whl; hf_xet-1.6.0-cp38-abi3-win_arm64.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “chunk deduplication storage”
- hf-xethf-xet provides chunk-based deduplication and efficient file transfer…
- icechunkIcechunk is a transactional storage engine for tensor/ND-array data…
- borgbackupBorgBackup is a command-line deduplicating backup tool that stores…
Give your agent the search over MCP, or paste the wish link into any chat.
More Artificial Intelligence packages
LiteLLM provides a unified Python interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others) using OpenAI-compatible API format, available as both a Python SDK and a self-hosted AI Gateway proxy server.
Install it if you need to work with multiple LLM providers or want to centralize LLM routing in your organization.
Client library and CLI tool for downloading, uploading, and managing models, datasets, and repositories on the Hugging Face Hub platform.
Install it if you work with Hugging Face Hub models or datasets.
LangChain provides a framework for building agents and LLM-powered applications by composing language models, tools, and memory through a unified API that abstracts over multiple model providers.
Tokenizers converts raw text into token sequences for NLP models, with support for training custom vocabularies and using pre-built tokenizers (BPE, WordPiece) optimized for speed via Rust.
Transformers provides a unified framework for loading, fine-tuning, and running state-of-the-art pretrained models across text, vision, audio, video, and multimodal tasks using PyTorch, JAX, or TensorFlow.
Install it if you need to run or train any transformer-based model for NLP, vision, audio, or multimodal tasks.
LangChain Core provides the foundational abstractions and interfaces for building LLM applications, enabling modular composition of language model chains, agents, and tools across the LangChain ecosystem.
Install it if you are building any LLM application that benefits from modular abstractions and ecosystem interoperability.
See also cached-path · hf-transfer · hf · spaces · datasets · pyspark-huggingface · kernels · hf-doc-builder · borgbackup