promptflow-tracing

What it is and what it does

promptflow-tracing is a tracing library designed to instrument LLM applications built with Prompt Flow, Flex Flow, or other frameworks like langchain and semantic kernel. It integrates with OpenTelemetry to capture detailed execution traces—showing how data flows through your LLM application, where time is spent, and what inputs and outputs occur at each step. This is useful for debugging, monitoring, and understanding the behavior of complex LLM pipelines.

The package sits on top of three core dependencies: openai for LLM calls, opentelemetry-sdk for the tracing infrastructure, and tiktoken for token counting. It is actively maintained by Microsoft, supports Python 3.9 through 3.13, and carries no known security vulnerabilities. Installation is straightforward and the maintenance signal is strong.

Use it for:

• Debug LLM application behavior by tracing execution flow and identifying where failures or unexpected outputs occur.
• Monitor production LLM applications to measure latency, token usage, and API call patterns across Flex Flow or DAG Flow pipelines.
• Integrate tracing into langchain or semantic kernel applications to gain visibility into agent decision-making and tool calls.
• Export traces to an observability backend (e.g., Jaeger, Datadog) for centralized monitoring and analysis of LLM workflows.
• Profile and optimize LLM application performance by identifying bottlenecks in multi-step flows.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Captures and visualizes the internal execution of LLM application flows (DAG and Flex) with OpenTelemetry-compatible tracing for observability.

Yes. If you are building LLM applications with Prompt Flow, langchain, or similar frameworks and need observability, this package provides a low-friction, actively maintained way to add OpenTelemetry-compatible tracing. No known vulnerabilities, permissive license, and straightforward dependencies make it a safe choice.

Install

promptflow-tracing on PyPI

pip install promptflow-tracing

uv add promptflow-tracing

poetry add promptflow-tracing

Installing promptflow-tracing

Before you install

Low friction install with three straightforward runtime dependencies (openai, opentelemetry-sdk, tiktoken). Active maintenance with recent commits and no known vulnerabilities.

License in practice

MIT license permits commercial and private use with minimal restrictions—suitable for most projects.

Quickstart

pip install promptflow-tracing

from promptflow_tracing import trace
from opentelemetry.sdk.trace import TracerProvider

@trace
def my_llm_function():
    pass

Requires Python 3.9 or later; OpenTelemetry SDK must be configured for tracing to be exported.

Verify before relying

• Whether the package works with all listed frameworks (langchain, semantic kernel, OpenAI, agents) equally well or if some require additional setup.
• How to configure and export traces to a backend observability system beyond the OpenTelemetry SDK.
• Performance overhead of tracing on production LLM applications.

Package facts

Evidence: promptflow_tracing-1.18.5-py3-none-any.whl

Keywords: telemetry

Tags

More Monitoring packages

Further reading