multiprocessing-logging

What it is and what it does

multiprocessing-logging solves the problem of garbled or lost log messages when using Python's multiprocessing module. When subprocesses log independently, their output can interleave or be lost; this package installs a handler on the root logger that tunnels all log records from child processes back to the main process for centralized handling.

The module is simple to use: call `install_mp_handler()` after configuring logging but before spawning processes or creating a Pool. It relies on the fork start method for process creation and works on POSIX systems (Linux officially supported). The documentation notes a known limitation: fork-based process creation is unsafe when threads are also in use, creating a low probability of application hangs; the recommended workaround is to create a Pool once and reuse it rather than continuously spawning new processes.

Use it for:

• Aggregate logs from a multiprocessing.Pool into a single file or stream without interleaving.
• Centralize subprocess logging in a long-running daemon that spawns worker processes.
• Debug parallel workloads by collecting all subprocess output in order at the main process.
• Replace per-subprocess log files with a unified log when using fork-based multiprocessing.
• Ensure log records from child processes reach configured handlers (files, syslog, etc.) in the main process.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides a logging handler that tunnels log records from multiprocessing subprocesses to the main process, preventing garbled or lost log messages.

Yes, if you use multiprocessing on Linux and need reliable subprocess logging. The package is stable and has no security vulnerabilities, but maintenance is aging (last release 2023). Install before creating pools or spawning processes, and avoid continuous process creation to mitigate the known fork-threading hang risk. Not suitable for Windows.

Install

multiprocessing-logging on PyPI

pip install multiprocessing-logging

uv add multiprocessing-logging

poetry add multiprocessing-logging

Installing multiprocessing-logging

Before you install

Low install friction with no runtime dependencies. Maintenance is aging—last release was in 2023 and no commits since 2026-01-28—but the package is marked Production/Stable and remains archived-free.

License in practice

Licensed under LGPLv3, a copyleft license. You must disclose source code modifications and distribute under the same license if you redistribute the package or derivative works.

Quickstart

import logging
import multiprocessing_logging

logging.basicConfig()
multiprocessing_logging.install_mp_handler()
# Call install_mp_handler() before creating multiprocessing.Pool or spawning subprocesses

Only works on POSIX systems; Linux is officially supported. Does not work on Windows. PyPy3 is not recommended due to test hangs.

Verify before relying

• Whether the fork-based approach and threading interaction issue (low probability of hangs) is acceptable for your workload.
• Current compatibility with Python versions beyond 3.10, given the aging maintenance status.
• Whether PyPy (non-3) support is stable enough for production use in your environment.

Package facts

Evidence: multiprocessing_logging-0.3.4-py2.py3-none-any.whl

Keywords: multiprocessing, logging, logger, handler

Tags

More Python Modules packages