{"categories":[{"label":"Distributed Computing","url":"https://skillfed.io/packages/category/system-distributed-computing/2"}],"enrichment":{"capability":"Wraps Python's ProcessPoolExecutor and ThreadPoolExecutor to limit queued tasks, preventing memory exhaustion when submitting millions of jobs by only enqueuing new tasks as workers become available.","skillfed_tags":["concurrency","memory-bounded","backpressure"],"use_cases":["Processing millions of items on a memory-constrained system without exhausting available RAM.","Batch processing large datasets where task submission rate far exceeds worker completion rate.","Long-running services that accept unbounded work queues and need to apply backpressure to prevent memory issues.","Distributed crawling or scraping tasks where the task generator runs faster than workers can consume."],"what_it_does":"bounded-pool-executor provides drop-in replacements for concurrent.futures.ProcessPoolExecutor and ThreadPoolExecutor that apply backpressure to task submission. Instead of accepting all submitted tasks into an unbounded queue, it uses a BoundedSemaphore to block submit() calls until a worker finishes, ensuring the queue never grows beyond the number of active workers. This prevents the memory exhaustion that occurs when processing millions of items with standard executors on memory-constrained systems.\n\nThe package is a thin wrapper around the standard library's executor classes, requiring no external dependencies. It is designed as a direct replacement: you import BoundedProcessPoolExecutor or BoundedThreadPoolExecutor and use them identically to their standard counterparts, but with the guarantee that memory usage stays bounded by the number of workers rather than the number of submitted tasks.","worth_installing":"Yes, if you are processing large task volumes on memory-limited systems and can tolerate dormant maintenance. The package solves a real problem (memory exhaustion with standard executors) and has no dependencies, making it low-risk to add. However, verify compatibility with your Python version first, as the latest release was 2019-06-04 and no updates are planned."},"id":"bounded-pool-executor","links":{"html":"https://skillfed.io/packages/bounded-pool-executor","md":"https://skillfed.io/packages/bounded-pool-executor.md","pypi":"https://pypi.org/project/bounded-pool-executor/"},"maintenance":{"status":"dormant"},"meta":{"latest_release":"2019-06-04","license_spdx":null,"license_treatment":"permissive","name":"bounded-pool-executor","python_support":"unspecified","summary":"Bounded Process&Thread Pool Executor"},"popularity":{"monthly_downloads":417754,"position":6810,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.0.3"}