cog

What it is and what it does

Cog is a CLI tool that abstracts away the complexity of containerizing machine learning models for production deployment. You define your model environment and inference logic in simple YAML and Python files, and Cog generates a Docker image with all the best practices baked in: correct CUDA/cuDNN/PyTorch/TensorFlow combinations, efficient layer caching, sensible environment defaults, and automatic OpenAPI schema generation from your Python type hints.

The generated container includes a high-performance HTTP inference server that exposes your model's inputs and outputs as a REST API. You can run it locally for testing, build it as a standalone Docker image for your own infrastructure, or deploy directly to Replicate. Cog handles the glue between research code and production deployment, eliminating the need for researchers to write Dockerfiles or coordinate with engineers on CUDA compatibility.

Use it for:

• Package a PyTorch or TensorFlow model with GPU support and deploy it as a REST API without writing a Dockerfile.
• Generate an OpenAPI schema and HTTP server from Python type annotations to serve inference requests.
• Resolve CUDA/cuDNN/framework version conflicts automatically instead of debugging dependency hell manually.
• Test a containerized model locally with `cog run` before building and shipping the Docker image.
• Deploy the same model container to your own Kubernetes cluster or to Replicate's hosted platform.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Cog packages machine learning models into production-ready Docker containers with automatic CUDA/dependency resolution, OpenAPI schema generation, and a built-in HTTP inference server.

Yes, if you are packaging ML models for production and want to avoid Dockerfile complexity and CUDA version mismatches. Cog is actively maintained, has low install friction, permissive licensing, and no known vulnerabilities. The main prerequisite is Docker; if you already have it, Cog is a straightforward way to standardize model containerization. Not necessary if you are building inference servers manually or using a higher-level ML platform that handles containerization for you.

Install

cog on PyPI

pip install cog

uv add cog

poetry add cog

Installing cog

Before you install

Low install friction with a pure-Python wheel. The package is actively maintained with a recent release and 9458 GitHub stars. It requires Docker as a system dependency, which is the primary prerequisite for use rather than a Python-level friction point.

License in practice

Apache License 2.0 is permissive; you can use, modify, and redistribute Cog and derivative works freely provided you include license notices and state changes. No commercial restrictions.

Quickstart

# Install Cog (requires Docker)
pip install cog

# Create cog.yaml and run.py, then:
from cog import BaseRunner, Input

class Runner(BaseRunner):
    def run(self, text: str = Input()) -> str:
        return f"Processed: {text}"

# Run locally
# cog run -i text="hello"

# Or build and serve
# cog build -t my-model
# cog serve

Docker must be installed and running. Cog is a CLI tool that orchestrates Docker; it does not run without it.

Verify before relying

• Whether coglet (a runtime dependency) is an internal Cog module or an external package with its own maintenance status.
• Performance characteristics of the auto-generated Rust/Axum HTTP server under load.
• Compatibility with GPU setups beyond NVIDIA (e.g., AMD ROCm, Apple Metal).

Package facts

Evidence: cog-0.22.0-py3-none-any.whl

Tags

More Artificial Intelligence packages