abx-dl

What it is and what it does

abx-dl is a unified CLI interface to the ArchiveBox plugin ecosystem for downloading and extracting content from URLs. It runs a collection of specialized tools (wget, browser automation, media extractors, etc.) in a single command, organizing output by plugin into a structured directory with a JSON log. The tool is designed for scraping, OSINT, digital preservation, and backup workflows where you want multiple extraction methods applied automatically without orchestrating separate commands.

It works by loading plugins from the abx-plugins package, resolving required external binaries through a multi-provider system (env, pip, npm, brew, apt, cargo, browser-specific), and executing them in phases: install (binary resolution), crawl setup (expensive per-crawl processes), and snapshot (per-URL extraction). Configuration is managed through environment variables and a persistent config file, with per-plugin timeouts and binary path overrides available.

Use it for:

• Archive a webpage with HTML, PDF, screenshot, and extracted text in one command.
• Download video playlists with metadata and subtitles using the ytdlp plugin.
• Extract article content, favicon, and outlinks from a URL for OSINT or research.
• Batch-download multiple URLs with consistent output structure for digital preservation.
• Configure and pre-install plugin dependencies for reproducible archiving workflows.
• Customize plugin selection by output type (e.g., --output=pdf,html,video) or by name.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A CLI tool that auto-detects and downloads content from URLs using a plugin-based architecture, supporting multiple extraction methods (HTML, PDFs, screenshots, metadata, media) in a single command.

Yes. Active, well-maintained, low-friction install with no compiled dependencies. MIT license is permissive. Solves a real problem—unified multi-tool downloading—for archiving, scraping, and OSINT workflows. Plugin architecture is flexible and extensible. No known vulnerabilities. Main gotcha: external binaries (wget, chrome, etc.) must be available or auto-installed; test on your target platform first.

Install

abx-dl on PyPI

pip install abx-dl

uv add abx-dl

poetry add abx-dl

Installing abx-dl

Before you install

Low friction: pure Python wheel with no compiled dependencies. Active maintenance (last commit 2026-08-14, released 2026-08-14). Depends on 10 runtime packages including rich, pydantic, and requests—all stable, widely-used libraries. Requires Python 3.12–3.14.

License in practice

MIT license (permissive): you can use, modify, and distribute this package freely in commercial or private projects with minimal restrictions.

Quickstart

pip install abx-dl
abx-dl --plugins=title,wget 'https://example.com'
# Downloads to current directory with index.jsonl + plugin-specific subdirs

Requires Python 3.12 or later. Many plugins require external binaries (wget, chrome, etc.); abx-dl can auto-install these, but --no-install skips that step.

Verify before relying

• Whether plugin auto-installation works reliably across all supported platforms (Linux, macOS, Windows).
• Performance characteristics when downloading large or media-heavy URLs.
• Stability and error recovery when plugins fail or timeout.

Package facts

Evidence: abx_dl-1.12.132-py3-none-any.whl

Keywords: scraping, crawling, downloading, internet archiving, web archiving, digipres, warc, preservation, backups, archiving, web, bookmarks, puppeteer, browser, download

Tags

More Python Modules packages