openlit

What it is and what it does

OpenLIT is an OpenTelemetry-native SDK that automatically instruments LLM applications, vector databases, and GPU operations to generate traces and metrics without code changes. It works by wrapping calls to LLM providers (OpenAI, Anthropic, Cohere, etc.), vector databases (ChromaDB, Pinecone, Qdrant, etc.), and AI frameworks (LangChain, LlamaIndex, Haystack, etc.), capturing request/response data, latency, token usage, and costs. The generated telemetry follows OpenTelemetry semantic conventions and can be exported to any OTLP-compatible backend.

The package is designed for developers building GenAI applications who need observability without modifying their application logic. It handles the complexity of instrumenting multiple LLM providers and frameworks through a single initialization call. The 26 runtime dependencies include the OpenTelemetry SDK, instrumentation modules for common web frameworks (FastAPI, Flask, Django, etc.), and client SDKs for major LLM and cloud providers (OpenAI, Anthropic, boto3). This breadth of dependencies means the package pulls in significant infrastructure but also ensures broad compatibility out of the box.

Use it for

• Monitor token usage and costs across multiple LLM providers in a single application to track GenAI spending.
• Trace LLM request latency and errors through LangChain or LlamaIndex pipelines to debug slow or failing AI workflows.
• Instrument vector database queries (ChromaDB, Pinecone) to correlate retrieval performance with LLM response quality.
• Export GPU metrics (NVIDIA, AMD) alongside LLM traces to correlate compute resource usage with inference performance.
• Send LLM observability data to existing monitoring tools for unified monitoring of AI and traditional infrastructure.
• Track custom and fine-tuned model costs using custom pricing files for accurate budget forecasting.

Worth the install?

AI-flagged interpretation of the facts on this page. Verify before relying on it.

Install

openlit on PyPI

Before you install

Low friction installation with a pure Python wheel. Active maintenance (last commit 2026-08-14) and 2688 GitHub stars indicate ongoing development. The package carries 26 runtime dependencies spanning OpenTelemetry components, LLM SDKs, and web framework instrumentation, which are installed automatically.

Requires an OpenTelemetry collector endpoint to send traces and metrics; without one, instrumentation runs but data is not exported.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions, making it suitable for production deployments in most organizational contexts.

Quickstart

pip install openlit

import openlit
openlit.init()

# Your LLM code now auto-instrumented
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(model="gpt-4", messages=[...])

Package facts

Evidence: openlit-1.45.0-py3-none-any.whl

Tags

More Monitoring packages

See also aliyun-trace · opentelemetry-instrumentation-vertexai · opentelemetry-instrumentation-llamaindex · opentelemetry-instrumentation-openai · opentelemetry-semantic-conventions-ai · opentelemetry-instrumentation-litellm · opentelemetry-instrumentation-chromadb · opentelemetry-instrumentation-groq · agentops · mlflow-skinny