---
id: bdbag
version: "1.8.0"
license: unclear
license_treatment: permissive
maintenance: aging
---

# bdbag — Big Data Bag Utilities

License: permissive · Maintenance: aging · Downloads: 119.0K/mo

## What it is and what it does

bdbag is a toolkit for creating and managing BagIt packages—a standardized format for bundling research data with metadata, checksums, and references to remote files. It wraps and extends the Bagit-Python library and Bagit-Profiles-Validator, adding features like in-place bag updates, automatic archiving (ZIP, TAR, TGZ), and multi-protocol file retrieval (HTTP, FTP, S3, Google Cloud Storage, Globus). The package enforces BDBag and Bagit/RO profiles, which specify required manifests and serialization rules, making it useful for researchers and data repositories that need to exchange large, complex datasets in a reproducible, verifiable format.

The tool works both as a command-line utility and as a Python API, allowing integration into data pipelines. It depends on seven runtime libraries (pytz, tzlocal, certifi, packaging, requests, bagit, bagit-profile) and installs with low friction as a pure-Python wheel. It supports Python 3.8 through 3.12 and carries no known security vulnerabilities.

Use it for:

- Create and validate research data packages that conform to BagIt standards for submission to data repositories.
- Automatically generate manifest and fetch.txt entries for datasets that include remote files from multiple sources.
- Retrieve remote files referenced in a bag's fetch.txt using HTTP, FTP, S3, or Globus Transfer protocols.
- Update existing bags in place without re-creating the entire structure, useful for iterative data curation.
- Serialize bags into ZIP, TAR, or TGZ archives for distribution while preserving integrity metadata.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

bdbag creates, validates, and manages BagIt packages—standardized containers for research data that bundle files with checksums, metadata, and remote file references—supporting multiple serialization formats and transport protocols.

Yes, if you work with research data packaging or need to exchange large datasets in a standardized, verifiable format. The low install friction, lack of security issues, and support for current Python versions make it straightforward to adopt. The aging maintenance status (319 days since release) is a minor concern for a stable, feature-complete tool but warrants checking whether active development has simply paused or shifted elsewhere.

## Install

pip install bdbag
uv add bdbag
poetry add bdbag

## Installing bdbag

Before you install: Low install friction with a pure-Python wheel. Maintenance status is aging (319 days since last release), but the package supports current Python versions (3.8–3.12) and carries no known security vulnerabilities.

License in practice: Licensed under Apache License (permissive), allowing commercial and private use with minimal restrictions.

Quickstart:

pip install bdbag

from bdbag import bdbag
bdbag.make_bag('/path/to/data', 'my-bag')

Requires Python 3.8 or later; optional extra dependencies (boto, globus) available for S3 and Globus transfer support.

Verify before relying:

- Whether the aging maintenance status (319 days since release) reflects active stewardship or dormancy in a stable, feature-complete tool.
- Performance characteristics when handling very large datasets or deeply nested directory structures.
- Compatibility with the latest BagIt specification versions and any breaking changes in upstream bagit or bagit-profile libraries.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: aging
- Downloads: 119.0K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

bagit package creation and validation, research data packaging utilities, big data bag management, fetch.txt remote file handling, bagit profile compliance checking, data-packaging, research-data, bagit-standard

[View on SkillFed](https://skillfed.io/packages/bdbag) · [View on PyPI](https://pypi.org/project/bdbag/)