{"categories":[{"label":"Distributed Computing","url":"https://skillfed.io/packages/category/system-distributed-computing"}],"enrichment":{"capability":"Pypeln lets you build concurrent data pipelines using processes, threads, or asyncio tasks with a unified functional API, allowing fine-grained control over workers and memory at each stage.","skillfed_tags":["concurrent-pipelines","multiprocessing-abstraction","abandoned-but-stable"],"use_cases":["Process a stream of network requests or I/O-bound tasks using thread workers without managing thread pools manually.","Build a multi-stage ETL pipeline where different stages benefit from different concurrency models (e.g., asyncio for I/O, processes for CPU-bound work).","Debug or prototype concurrent logic using the sync module before switching to threads or processes for performance.","Parallelize CPU-bound transformations across multiple processes while controlling memory usage per stage with maxsize limits.","Chain functional operations (map, filter) on iterables with automatic worker distribution and queue management."],"what_it_does":"Pypeln is a Python library for building concurrent data pipelines using a functional API that abstracts over processes, threads, asyncio tasks, or synchronous generators. You define stages\u2014each with map, filter, or flat_map operations\u2014and specify how many workers should process data at each stage, with control over queue size to manage memory. Stages are composable and can be mixed (e.g., processes feeding into threads), and they integrate with regular Python code since stages are iterables.\n\nThe library targets medium-scale data tasks where parallelism is needed but frameworks like Spark or Dask feel overkill. It exposes a familiar functional programming interface and supports a pipe operator for readable pipeline construction. However, the project is abandoned\u2014no releases since January 2022 and last commit in July 2023\u2014so it receives no maintenance, security updates, or compatibility fixes.","worth_installing":"Yes, if you have a stable, medium-scale data pipeline task and can tolerate no maintenance. The library is feature-complete for its scope, has low install friction, and carries no known vulnerabilities. However, avoid it for new projects requiring long-term support or compatibility with future Python versions\u2014the abandoned status means no bug fixes or updates. Consider it only when the problem fits its design and you accept the maintenance risk."},"id":"pypeln","links":{"html":"https://skillfed.io/packages/pypeln","md":"https://skillfed.io/packages/pypeln.md","pypi":"https://pypi.org/project/pypeln/"},"maintenance":{"status":"abandoned"},"meta":{"latest_release":"2022-01-06","license_spdx":null,"license_treatment":"permissive","name":"pypeln","python_support":"supports_current","summary":null},"popularity":{"monthly_downloads":732003,"position":5203,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.4.9"}