{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/10"}],"enrichment":{"capability":"A Python SDK for writing serverless endpoints that run on Cozy's worker pool, handling model discovery, scheduling, download, device placement, cancellation, and streaming.","skillfed_tags":["serverless-inference","model-orchestration","gpu-management"],"use_cases":["Deploy a text-to-image model (e.g., Stable Diffusion XL) as a managed endpoint with automatic GPU placement and VRAM optimization.","Build a lightweight API proxy endpoint that routes requests to external services without needing GPU resources.","Create a model conversion pipeline that ingests checkpoints, quantizes them to fp8, and publishes them to the hub.","Develop a streaming inference endpoint (e.g., token-by-token LLM generation) using async generators.","Run multi-model endpoints where users select a checkpoint at request time via a ModelChoice enum."],"what_it_does":"gen-worker is a Python SDK for building serverless endpoints that run on Cozy's managed worker pool. You write a single decorated function or class, and the SDK handles the infrastructure: model discovery and download from HuggingFace, Civitai, or other sources; GPU device placement and low-VRAM offload; request scheduling and cancellation; file I/O and streaming; and reporting back to the control plane. It abstracts away device management entirely\u2014endpoint code never touches `.to(\"cuda\")` or offload configuration.\n\nThe package supports both simple stateless endpoints (a decorated function) and stateful ones (a class with setup() and handler methods). It includes bindings for popular model sources (HF, Hub, Civitai, ModelScope), optional support for PyTorch inference and training, media I/O for images, audio, and video, and streaming via async generators. Local development is supported through a CLI (gen-worker run, serve, invoke, prefetch) for testing before deployment. The SDK is actively maintained, requires Python 3.12+, and has no known vulnerabilities.","worth_installing":"Yes, if you are building endpoints for Cozy's platform or need a framework that abstracts GPU orchestration and model management. The SDK is actively maintained, has low install friction, permissive licensing, and zero known vulnerabilities. It is production-ready for inference and training workloads. Not suitable if you need to deploy to other serverless platforms or prefer manual device management."},"id":"gen-worker","links":{"html":"https://skillfed.io/packages/gen-worker","md":"https://skillfed.io/packages/gen-worker.md","pypi":"https://pypi.org/project/gen-worker/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-14","license_spdx":"MIT","license_treatment":"permissive","name":"gen-worker","python_support":"supports_current","summary":"A library used to build custom functions in Cozy Creator's serverless function platform."},"popularity":{"monthly_downloads":88489,"position":13725,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.116.0"}