---
id: gen-worker
version: "0.116.0"
license: MIT
license_treatment: permissive
maintenance: active
---

# gen-worker — A library used to build custom functions in Cozy Creator's serverless function platform.

License: permissive · Maintenance: active · Downloads: 88.5K/mo

## What it is and what it does

gen-worker is a Python SDK for building serverless endpoints that run on Cozy's managed worker pool. You write a single decorated function or class, and the SDK handles the infrastructure: model discovery and download from HuggingFace, Civitai, or other sources; GPU device placement and low-VRAM offload; request scheduling and cancellation; file I/O and streaming; and reporting back to the control plane. It abstracts away device management entirely—endpoint code never touches `.to("cuda")` or offload configuration.

The package supports both simple stateless endpoints (a decorated function) and stateful ones (a class with setup() and handler methods). It includes bindings for popular model sources (HF, Hub, Civitai, ModelScope), optional support for PyTorch inference and training, media I/O for images, audio, and video, and streaming via async generators. Local development is supported through a CLI (gen-worker run, serve, invoke, prefetch) for testing before deployment. The SDK is actively maintained, requires Python 3.12+, and has no known vulnerabilities.

Use it for:

- Deploy a text-to-image model (e.g., Stable Diffusion XL) as a managed endpoint with automatic GPU placement and VRAM optimization.
- Build a lightweight API proxy endpoint that routes requests to external services without needing GPU resources.
- Create a model conversion pipeline that ingests checkpoints, quantizes them to fp8, and publishes them to the hub.
- Develop a streaming inference endpoint (e.g., token-by-token LLM generation) using async generators.
- Run multi-model endpoints where users select a checkpoint at request time via a ModelChoice enum.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

A Python SDK for writing serverless endpoints that run on Cozy's worker pool, handling model discovery, scheduling, download, device placement, cancellation, and streaming.

Yes, if you are building endpoints for Cozy's platform or need a framework that abstracts GPU orchestration and model management. The SDK is actively maintained, has low install friction, permissive licensing, and zero known vulnerabilities. It is production-ready for inference and training workloads. Not suitable if you need to deploy to other serverless platforms or prefer manual device management.

## Install

pip install gen-worker
uv add gen-worker
poetry add gen-worker

## Installing gen-worker

Before you install: Low friction install with a pure-Python wheel. Active maintenance with a release on 2026-08-14. Requires Python 3.12 or later. Twelve runtime dependencies including grpcio, huggingface-hub, and protobuf suggest a mature, production-oriented codebase.

License in practice: MIT license (permissive) means you can use, modify, and distribute this package freely in commercial and private projects with minimal restrictions.

Quickstart:

pip install gen-worker

from gen_worker import RequestContext, endpoint
import msgspec

class Input(msgspec.Struct):
    prompt: str

@endpoint
def echo(ctx: RequestContext, payload: Input) -> str:
    return f"got: {payload.prompt}"

# Run locally: gen-worker run --payload '{"prompt": "hello"}'

Requires Python 3.12 or later. For model inference with PyTorch, install gen-worker[torch]. Local testing requires uv; never pip install globally as it shadows the working tree.

Verify before relying:

- Whether the platform's profiling gate for VRAM measurement works reliably across different GPU types and model sizes.
- How well the per-request state isolation (ctx.for_request) scales under high concurrency.
- Whether optional extras ([images], [audio], [video], [vision]) are fully documented and stable.

## Package facts

- License: MIT (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 88.5K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

serverless endpoint framework, worker pool sdk, model inference deployment, distributed function hosting, ml endpoint orchestration, cozy creator integration, async task scheduling, serverless-inference, model-orchestration, gpu-management

[View on SkillFed](https://skillfed.io/packages/gen-worker) · [View on PyPI](https://pypi.org/project/gen-worker/)