piper

What it is and what it does

Piper is a Python library that wraps shell command execution to add workflow management features on top of standard subprocess calls. It's designed for multi-step pipelines that chain together command-line tools, where robustness and reproducibility matter. The package handles restarting failed pipelines from checkpoints, validates output files, tracks resource usage via psutil, and logs execution details through logmuse. It integrates with pipestat for status reporting and yacman for configuration management.

The library targets developers building multi-step workflows that call external tools—common in bioinformatics, data science, and systems administration. Rather than writing shell scripts or reinventing error handling, you define pipeline steps in Python and let piper manage execution state, resource constraints, and detailed audit trails. It has been actively maintained since 2013 and currently supports modern Python versions.

Use it for:

• Build bioinformatics pipelines that chain alignment, variant calling, and annotation tools with automatic restartability.
• Monitor resource usage and enforce limits on long-running data processing workflows.
• Log and audit multi-step ETL pipelines with detailed execution records and file integrity checks.
• Orchestrate shell-based data analysis workflows in containerized environments.
• Implement checkpoint-based restarts for pipelines that fail partway through expensive computations.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Pypiper manages shell command execution within Python workflows, providing automatic restartability, file integrity checks, resource monitoring, and structured logging for multi-step pipelines.

Yes, if you're building multi-step shell-based workflows in Python and need restartability, resource monitoring, and structured logging. The active maintenance, permissive license, low install friction, and modern Python support make it a solid choice. No known security vulnerabilities. Not necessary for simple one-off scripts or if you prefer pure Python tooling over shell integration.

Install

piper on PyPI

pip install piper

uv add piper

poetry add piper

Installing piper

Before you install

Low install friction with a pure-Python wheel. Active maintenance with a recent commit on 2026-03-06 and ongoing support for current Python versions (3.10 through 3.14).

License in practice

BSD-2-Clause is permissive, allowing commercial and private use with minimal restrictions beyond attribution and liability disclaimers.

Quickstart

pip install piper

import piper

# Create and run a pipeline with shell command execution

Requires Python 3.10 or later.

Verify before relying

• Specific API and class names for creating and managing pipelines.
• Whether automatic restartability works across all shell environments or has platform-specific limitations.
• How file integrity protection is implemented and what formats it supports.
• Whether resource monitoring integrates with container orchestration systems.

Package facts

Evidence: piper-0.15.1-py3-none-any.whl

Keywords: command-runner, pipeline, restartable, shell, workflow

Tags

More Python Modules packages