pyfunceble-process-manager

What it is and what it does

This package is a process manager library extracted from the PyFunceble project to enable multiprocess work distribution without requiring the full PyFunceble installation. It provides a framework for spawning and managing worker processes that consume tasks from input queues and produce results to output queues, with built-in support for lifecycle hooks (poweron, preflight, inflight, postflight, poweroff checks) and reserved control messages for graceful or immediate shutdown.

You define custom workers by subclassing WorkerCore and implementing a target() method, then wrap them in a ProcessManagerCore subclass to control the number of workers, queue connections, and scaling behavior. The manager handles process lifecycle, inter-process messaging, and queue routing, allowing you to chain multiple managers together for multi-stage data pipelines.

Use it for:

• Build a data processing pipeline where one pool filters records and another pool writes results, with automatic worker scaling based on queue depth.
• Distribute CPU-bound validation tasks across multiple processes while collecting results in a shared output queue for downstream consumption.
• Implement a worker pool for I/O-bound operations (API calls, file reads) with graceful shutdown and preflight checks before each task.
• Chain multiple process managers to create a multi-stage ETL workflow with independent worker counts and dynamic up/down scaling.
• Process large datasets in parallel with custom filtering logic at each stage, using reserved control messages to coordinate shutdown across workers.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides a multiprocess task queue framework with worker pools, inter-process messaging, and dynamic scaling for distributing work across processes.

Yes. Low install friction, active maintenance, no known vulnerabilities, and a permissive license make it a safe choice. Install it if you need a structured multiprocess task queue with worker lifecycle hooks and dynamic scaling; the example code is clear and the API surface is straightforward. Skip it if you only need simple thread-based concurrency or if you prefer a heavier framework like Celery.

Install

pyfunceble-process-manager on PyPI

pip install pyfunceble-process-manager

uv add pyfunceble-process-manager

poetry add pyfunceble-process-manager

Installing pyfunceble-process-manager

Before you install

Low friction: pure Python wheel with only setuptools as a runtime dependency. Actively maintained with recent commits; no known vulnerabilities.

License in practice

Apache 2.0 permissive license allows commercial and private use with minimal restrictions; you must include a copy of the license and state significant changes.

Quickstart

pip install pyfunceble-process-manager

from PyFunceble.ext.process_manager import ProcessManagerCore, WorkerCore

class MyWorker(WorkerCore):
    def target(self, consumed):
        return consumed.upper()

class MyManager(ProcessManagerCore):
    STD_NAME = "my-manager"
    WORKER_CLASS = MyWorker

manager = MyManager(max_workers=4, generate_output_queue=True)
manager.start()
manager.push_to_input_queue("hello")
manager.terminate(mode="soft")

Requires Python 3.8 or later; multiprocessing on Windows may require wrapping manager code in `if __name__ == '__main__':`.

Verify before relying

• Performance characteristics and throughput limits under typical workloads
• Whether dynamic scaling works reliably with dependent managers in production
• Behavior and recovery guarantees during process crashes or queue saturation

Package facts

Evidence: pyfunceble_process_manager-1.0.11-py3-none-any.whl

Keywords: PyFunceble, process-manager

Tags

More Internet packages