bdbag
Big Data Bag Utilities
Decision gist · record as of 2026-08-14
Yes, if you work with research data packaging or need to exchange large datasets in a standardized, verifiable format. The low install friction, lack of security issues, and support for current Python versions make it straightforward to adopt. The aging maintenance status (319 days since release) is a minor concern for a stable, feature-complete tool but warrants checking whether active development has simply paused or shifted elsewhere.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.8 or later; optional extra dependencies (boto, globus) available for S3 and Globus transfer support.
- Low install friction with a pure-Python wheel.
- Maintenance status is aging (319 days since last release), but the package supports current Python versions (3.8–3.12) and carries no known security vulnerabilities.
License · maintenance · safety
permissive license (permissive) — Licensed under Apache License (permissive), allowing commercial and private use with minimal restrictions.
last release 2025-09-29 (319 days)
0 known vulnerabilities (OSV.dev, 2026-08-14) · 118,968 downloads/mo, #12,095 on PyPI
Alternatives
Verify before relying
pip install bdbag
from bdbag import bdbag
bdbag.make_bag('/path/to/data', 'my-bag')- Whether the aging maintenance status (319 days since release) reflects active stewardship or dormancy in a stable, feature-complete tool.
- Performance characteristics when handling very large datasets or deeply nested directory structures.
- Compatibility with the latest BagIt specification versions and any breaking changes in upstream bagit or bagit-profile libraries.
What it is and what it does
bdbag is a toolkit for creating and managing BagIt packages—a standardized format for bundling research data with metadata, checksums, and references to remote files. It wraps and extends the Bagit-Python library and Bagit-Profiles-Validator, adding features like in-place bag updates, automatic archiving (ZIP, TAR, TGZ), and multi-protocol file retrieval (HTTP, FTP, S3, Google Cloud Storage, Globus). The package enforces BDBag and Bagit/RO profiles, which specify required manifests and serialization rules, making it useful for researchers and data repositories that need to exchange large, complex datasets in a reproducible, verifiable format.
The tool works both as a command-line utility and as a Python API, allowing integration into data pipelines. It depends on seven runtime libraries (pytz, tzlocal, certifi, packaging, requests, bagit, bagit-profile) and installs with low friction as a pure-Python wheel. It supports Python 3.8 through 3.12 and carries no known security vulnerabilities.
Use it for
- Create and validate research data packages that conform to BagIt standards for submission to data repositories.
- Automatically generate manifest and fetch.txt entries for datasets that include remote files from multiple sources.
- Retrieve remote files referenced in a bag's fetch.txt using HTTP, FTP, S3, or Globus Transfer protocols.
- Update existing bags in place without re-creating the entire structure, useful for iterative data curation.
- Serialize bags into ZIP, TAR, or TGZ archives for distribution while preserving integrity metadata.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you work with research data packaging or need to exchange large datasets in a standardized, verifiable format.
The low install friction, lack of security issues, and support for current Python versions make it straightforward to adopt. The aging maintenance status (319 days since release) is a minor concern for a stable, feature-complete tool but warrants checking whether active development has simply paused or shifted elsewhere.
Install
bdbag on PyPI
Before you install
Low install friction with a pure-Python wheel. Maintenance status is aging (319 days since last release), but the package supports current Python versions (3.8–3.12) and carries no known security vulnerabilities.
Requires Python 3.8 or later; optional extra dependencies (boto, globus) available for S3 and Globus transfer support.
License in practice
Licensed under Apache License (permissive), allowing commercial and private use with minimal restrictions.
Quickstart
pip install bdbag
from bdbag import bdbag
bdbag.make_bag('/path/to/data', 'my-bag')Verify before relying
- Whether the aging maintenance status (319 days since release) reflects active stewardship or dormancy in a stable, feature-complete tool.
- Performance characteristics when handling very large datasets or deeply nested directory structures.
- Compatibility with the latest BagIt specification versions and any breaking changes in upstream bagit or bagit-profile libraries.
Package facts
| License | permissive license permissive |
| Python support | Supports the current Python release <4,>=3.8 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 7 packagespytztzlocalcertifipackagingrequestsbagitbagit-profile |
| Maintenance | Aging 319 days since the last release |
| First released | |
| Downloads | 118,968 / month, #12,095 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Intended Audience :: DevelopersIntended Audience :: Science/ResearchLicense :: OSI Approved :: Apache Software LicenseOperating System :: MacOS :: MacOS XOperating System :: Microsoft :: WindowsOperating System :: POSIXProgramming Language :: PythonProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9 |
Evidence: bdbag-1.8.0-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “research data packaging utilities”
- bdbagbdbag creates, validates, and manages BagIt packages—standardized…
- rocraterocrate is a Python library for creating and consuming Research…
- PyScaffoldPyScaffold generates a complete Python project template with…
Give your agent the search over MCP, or paste the wish link into any chat.
More Utilities packages
Converts domain names between Unicode and ASCII-compatible encoding (Punycode) according to IDNA 2008 and Unicode Technical Standard 46, with security validation and broader script coverage than the standard library.
Install it if you work with internationalized domain names, need to validate domains, or use HTTP clients that depend on it transitively.
Detects and normalizes text encoding from unknown or ambiguous sources, supporting all IANA character sets that Python's core library provides codecs for, with the ability to register custom codecs.
Setuptools is a Python build backend and package management tool that handles building, distributing, and installing Python packages, including support for C/C++ extension modules.
Pluggy provides a plugin system that lets you define hook specifications and register implementations to be called in sequence, enabling extensible Python applications without tight coupling.
Install it if you're building an extensible application or framework.
Pygments is a syntax highlighter that colorizes source code and text in over 500 languages and formats, outputting to HTML, LaTeX, RTF, SVG, images, or ANSI terminal sequences.
Install it if you need to display or transform source code.
Six provides utility functions to write Python code that runs on both Python 2.7 and Python 3.3+, smoothing over language differences between the two versions.
See also bagit · bagit-profile · big · dbs3-client · rosbags · roc-validator · interchange · refgenie · django-dbbackup · peppy