trulens-core

What it is and what it does

TruLens is an instrumentation and evaluation framework for LLM-based applications built on OpenTelemetry. It decorates functions to capture structured traces of every step—retrieval, LLM calls, tool invocations—recording latency, tokens, and cost per operation. Once traces are collected, the package evaluates them using LLM judges that score and explain their reasoning, enabling you to identify where applications fail, compare versions, and measure quality tradeoffs against cost.

The package ships with seven agentic evaluators (LogicalConsistency, ExecutionEfficiency, PlanAdherence, ToolSelection, and others) and supports both inline evaluation as your app runs and batch evaluation over pre-collected datasets. Because tracing is OpenTelemetry-native, traces are portable to any OTLP-compatible backend—Jaeger, Grafana Tempo, Datadog—making it interoperable with existing observability infrastructure. Runtime dependencies include SQLAlchemy for storage, Pydantic for validation, and OpenTelemetry libraries for instrumentation.

Use it for:

• Instrument a RAG pipeline to trace retrieval, ranking, and generation steps, then score groundedness and context relevance per query.
• Compare two versions of an agent by running both against a dataset and measuring latency, token cost, and evaluation scores side-by-side.
• Debug agent failures by examining structured traces that show which tool was called, what arguments were passed, and where reasoning broke down.
• Evaluate agentic systems for plan quality and tool selection by running purpose-built judges that flag redundant steps or incorrect tool choices.
• Export traces to an existing observability backend (Datadog, Grafana) while running evaluations in TruLens without duplicating instrumentation.
• Batch-evaluate a historical dataset of app runs using the Run API to compute metrics offline and identify systematic quality issues.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Instruments LLM applications with OpenTelemetry-native tracing and evaluates them with LLM-based judges that explain their reasoning, capturing latency, cost, and quality per step.

Yes. TruLens is actively maintained, has no known vulnerabilities, and solves a real problem for LLM application teams: making agent behavior traceable and measurable. The OpenTelemetry foundation ensures portability, and the built-in evaluators are benchmarked against human annotations. Install it if you need to instrument and evaluate LLM apps; skip it if you only need basic logging.

Install

trulens-core on PyPI

pip install trulens-core

uv add trulens-core

poetry add trulens-core

Installing trulens-core

Before you install

Low friction: pure Python wheel with 19 runtime dependencies including well-established libraries like SQLAlchemy, Pydantic, and OpenTelemetry. Active maintenance with release on 2026-08-14 and 3508 GitHub stars.

License in practice

MIT license permits commercial and private use with minimal restrictions; suitable for most projects.

Quickstart

pip install trulens-core

from trulens.core.otel.instrument import instrument
from trulens.otel.semconv.trace import SpanAttributes

class MyRAG:
    @instrument(
        span_type=SpanAttributes.SpanType.RETRIEVAL,
        attributes={
            SpanAttributes.RETRIEVAL.QUERY_TEXT: "query",
            SpanAttributes.RETRIEVAL.RETRIEVED_CONTEXTS: "return",
        },
    )
    def retrieve(self, query: str) -> list:
        pass

Requires Python 3.9 or later; LLM provider packages (e.g., trulens-providers-openai) must be installed separately for feedback evaluation.

Verify before relying

• Whether the package works with Python 3.14 in practice, given it is marked as Alpha status.
• Performance characteristics when evaluating large-scale batch runs with the Run API.
• Compatibility with OTLP backends beyond those mentioned (Jaeger, Grafana Tempo, Datadog).

Package facts

Evidence: trulens_core-2.13.0-py3-none-any.whl

Tags

More Quality Assurance packages