{"categories":[{"label":"Testing","url":"https://skillfed.io/packages/category/software-development-testing/6"}],"enrichment":{"capability":"Langwatch-scenario is an agent testing framework that simulates multi-turn conversations between your agent and LLM-powered user simulators, with optional judge agents to evaluate behavior against criteria.","skillfed_tags":["agent-testing","llm-evaluation","multi-turn-simulation"],"use_cases":["Test that a customer support agent asks follow-up questions and provides accurate information before resolving tickets.","Validate a recipe recommendation agent generates vegetarian options and includes ingredient lists and cooking instructions.","Verify a weather agent calls the correct tool and handles edge cases like missing location data.","Evaluate a coding assistant's ability to explain code changes and respond to clarification requests.","Benchmark agent behavior across different LLM models to compare response quality and tool usage."],"what_it_does":"Langwatch-scenario is an agent testing framework designed to validate LLM-based agent behavior through automated simulation. It runs multi-turn conversations between your agent implementation and simulated users (powered by LLMs), optionally with judge agents that evaluate outcomes against predefined criteria. You integrate your agent by implementing a single `call()` method, then define scenarios as pytest tests or standalone scripts that describe the context and expected behavior.\n\nThe framework handles the conversation loop, message passing, and evaluation. It supports both scripted control (where you define exact turn sequences) and autopilot mode (where the user simulator drives the conversation until success or max turns). Runtime dependencies include pytest, litellm, openai, pydantic, fastapi, and opentelemetry-sdk, reflecting its design for LLM integration, async execution, and observability. The package is actively maintained and available in Python, TypeScript, and Go.","worth_installing":"Yes. The package is actively maintained, has no known vulnerabilities, uses a permissive Apache-2.0 license, and solves a real problem\u2014testing agent behavior in realistic multi-turn scenarios. Low install friction and a large dependency set (28 runtime packages) are typical for LLM testing frameworks. Install if you need to validate agent behavior beyond unit tests or if you're building agents that must handle complex, multi-turn interactions reliably."},"id":"langwatch-scenario","links":{"html":"https://skillfed.io/packages/langwatch-scenario","md":"https://skillfed.io/packages/langwatch-scenario.md","pypi":"https://pypi.org/project/langwatch-scenario/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-13","license_spdx":null,"license_treatment":"permissive","name":"langwatch-scenario","python_support":"supports_current","summary":"The end-to-end agent testing library"},"popularity":{"monthly_downloads":97271,"position":13159,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"1.1.1"}