controlnet-aux

What it is and what it does

controlnet-aux wraps a collection of computer vision annotation models from lllyasviel's ControlNet repository and connects them to the Hugging Face Hub for model checkpoint hosting. It provides detector classes for edge detection, pose estimation, depth mapping, segmentation, and other tasks, each loadable via a unified Processor interface or individually. Models are downloaded on first use from the Hub and can be instantiated with optional device specification.

The package depends on torch, torchvision, scipy, numpy, opencv-python-headless, scikit-image, Pillow, einops, timm, filelock, and importlib_metadata. It is designed for image-to-image generation pipelines where you need to extract structural information from input images to guide model conditioning. Each detector accepts images and returns processed outputs.

Use it for:

• Extract pose skeletons from photos to guide pose-conditioned image generation
• Detect edges in images for edge-guided image synthesis or artistic processing
• Generate depth maps from single images for 3D-aware image editing
• Segment objects in images for object-aware inpainting or layout-guided generation
• Preprocess images for ControlNet-based diffusion models requiring structural annotations

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides computer vision annotation and detection models (edge detection, pose estimation, depth mapping, segmentation) packaged as PyPI-installable processors for use with ControlNet.

Yes, if you are building ControlNet-based image generation pipelines or need a convenient way to extract structural annotations from images. The package has low install friction, permissive licensing, and no known vulnerabilities. Caveat: maintenance is aging (no release in 463 days), so verify that model checkpoints and detector implementations match your use case before production deployment.

Install

controlnet-aux on PyPI

pip install controlnet-aux

uv add controlnet-aux

poetry add controlnet-aux

Installing controlnet-aux

Before you install

Low install friction with a pure Python wheel. Maintenance status is aging—last commit was 2025-05-08 with no release in 463 days—but the package remains in Production/Stable status with archived=false.

License in practice

Licensed under Apache (permissive), so you can use it freely in commercial and private projects without copyleft obligations.

Quickstart

pip install controlnet-aux

from controlnet_aux.processor import Processor

processor = Processor('openpose')
processed = processor(img, to_pil=True)

Requires torch and torchvision; model checkpoints are downloaded from Hugging Face Hub on first use (requires internet access and disk space).

Verify before relying

• Whether model checkpoints are cached locally or re-downloaded on each instantiation
• Memory footprint and GPU VRAM requirements for concurrent detector instances
• Performance characteristics (inference speed) for each detector type

Package facts

Evidence: controlnet_aux-0.0.10-py3-none-any.whl

Keywords: deep, learning

Tags

More Artificial Intelligence packages