dvc-http

What it is and what it does

dvc-http is a plugin that extends DVC (Data Version Control) to use HTTP and HTTPS endpoints as remote storage backends. It allows you to store versioned data on any HTTP server and retrieve it as part of your DVC workflow, integrating with DVC's existing remote storage abstraction layer through fsspec.

The plugin sits between DVC and HTTP servers, handling the mechanics of pushing and pulling data over HTTP. It depends on fsspec for the storage abstraction and aiohttp-retry for resilient asynchronous HTTP requests. This is useful when your data lives on a web server or HTTP-compatible storage service and you want DVC to manage versions and access patterns.

Use it for:

• Store DVC-tracked datasets on a company HTTP server or web host without needing cloud provider SDKs.
• Access versioned data from HTTP endpoints in CI/CD pipelines where only HTTP access is available.
• Use DVC with legacy or custom HTTP-based data repositories that don't have native cloud storage support.
• Retrieve specific versions of data from HTTP remotes in reproducible machine learning workflows.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Enables DVC to store and retrieve data over HTTP/HTTPS by providing a remote storage backend that integrates with DVC's data versioning system.

Yes, if you use DVC and need to store or retrieve data over HTTP. The plugin is actively maintained, has low install friction, carries no security vulnerabilities, and uses a permissive license. Install it only if HTTP is your actual remote storage backend; it's not needed for local or cloud-native workflows.

Install

dvc-http on PyPI

pip install dvc-http

uv add dvc-http

poetry add dvc-http

Installing dvc-http

Before you install

Low install friction with only two runtime dependencies (fsspec and aiohttp-retry). Actively maintained with recent commits; last release was 975 days ago but the repository remains active and not archived.

License in practice

Licensed under Apache License 2.0 (permissive), which allows commercial use, modification, and distribution with minimal restrictions—suitable for most projects.

Quickstart

pip install dvc-http

import dvc.api
data = dvc.api.get_url('path/to/data', repo='.')

Requires DVC to be installed and configured; the HTTP remote must be set up in DVC's configuration before use.

Verify before relying

• Whether this plugin supports HTTP authentication (basic auth, bearer tokens, or custom headers) for private endpoints.
• Performance characteristics when handling large files or high-latency HTTP connections.
• Whether the plugin supports HTTP range requests for partial file retrieval.

Package facts

Evidence: dvc_http-2.32.0-py3-none-any.whl

Keywords: dvc, http

Tags

More WWW/HTTP packages