leptonai

What it is and what it does

Lepton AI is a Python framework that wraps research and modeling code into deployable AI services. You define a Photon class with handler methods, and the framework exposes them as HTTP endpoints with automatic documentation, client bindings, and built-in features like autobatching and background jobs. It includes shortcuts to launch popular models from HuggingFace (like GPT-2, Llama2, SDXL, Whisper) with a single command, and provides a Python client that lets you call your service as if it were a local function.

The package is built on fastapi and uvicorn for the service layer, ray for distributed compute, and integrates with huggingface_hub for model loading. It's designed for researchers and ML engineers who want to turn a notebook or script into a production-ready service without writing boilerplate HTTP code.

Use it for:

• Launch a HuggingFace model (e.g., GPT-2, Llama2) as a local HTTP service with one CLI command.
• Wrap custom Python ML code into a service by decorating handler methods with @Photon.handler.
• Build a text-to-image service using prebuilt examples like SDXL and access it via Python client or Gradio UI.
• Deploy background jobs and autobatch requests across multiple GPU workers using ray integration.
• Serve multiple models from a single Photon class and call them as native Python functions from a client.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Lepton AI is a Python framework for building and deploying AI services from research and modeling code, with built-in support for launching HuggingFace models, autobatching, background jobs, and a client for calling services as native Python functions.

Yes, if you need to quickly turn Python ML code or HuggingFace models into a service. The framework is actively maintained, has low install friction, and abstracts away HTTP boilerplate. Verify the license status before use in proprietary projects. Not suitable if you need deep control over HTTP routing or prefer a lower-level framework.

Install

leptonai on PyPI

pip install leptonai

uv add leptonai

poetry add leptonai

Installing leptonai

Before you install

Low install friction with a pure-Python wheel. Active maintenance with a release within the last 70 days. Depends on 20 runtime packages including fastapi, uvicorn, ray, and huggingface_hub, which are all standard ecosystem tools.

Quickstart

pip install -U leptonai

from leptonai.photon import Photon

class Echo(Photon):
    @Photon.handler
    def echo(self, inputs: str) -> str:
        return inputs

# Then run: lep photon runlocal -n echo -m my_photon.py
# Access with:
from leptonai.client import Client, local
c = Client(local(port=8080))
c.echo(inputs="hello world")

Verify before relying

• Whether the Apache 2.0 license mentioned in the description excerpt is the authoritative license (meta.license_spdx is null).
• Whether all HuggingFace models work or only standard pipelines, as the description notes custom-code models are not supported.

Package facts

Evidence: leptonai-0.27.3-py3-none-any.whl

Tags

More Artificial Intelligence packages