{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/10"}],"enrichment":{"capability":"Unitxt provides a unified framework for evaluating AI model performance across text, tables, vision, speech, and code using a modular catalog of benchmarks and datasets.","skillfed_tags":["model-evaluation","benchmarking","dataset-catalog"],"use_cases":["Evaluate a custom LLM against standard benchmarks like MMLU Pro or GPQA without writing dataset loaders","Run multi-task evaluation across different domains in a single command-line call","Load and format datasets in chat API format for any model without manual preprocessing","Build reproducible evaluation pipelines with modular, shareable task and template components","Compare model performance across text, code, and structured data tasks using unified metrics"],"what_it_does":"Unitxt is a Python library designed for evaluating AI model performance in a unified, modular way. It provides a large catalog of pre-built benchmarks and datasets that can be loaded and formatted for any model, supporting text, tables, vision, speech, and code evaluation. The library works model-agnostically with HuggingFace, OpenAI, WatsonX, and custom inference engines, and emphasizes reproducibility through shareable, composable components.\n\nYou use Unitxt either through its command-line interface for quick benchmark runs or programmatically by loading datasets, defining tasks with metrics, and running inference and evaluation. It handles data preparation, formatting for chat APIs, and metric computation in one workflow, reducing the boilerplate needed to benchmark models at scale.","worth_installing":"Yes. Unitxt is actively maintained, has no known vulnerabilities, and offers low-friction installation. It solves a real problem\u2014standardizing AI model evaluation across diverse data types and models\u2014with a permissive license suitable for enterprise use. Install it if you need to benchmark models against established datasets or build reproducible evaluation workflows."},"id":"unitxt","links":{"html":"https://skillfed.io/packages/unitxt","md":"https://skillfed.io/packages/unitxt.md","pypi":"https://pypi.org/project/unitxt/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-05-27","license_spdx":null,"license_treatment":"permissive","name":"unitxt","python_support":"supports_current","summary":"Load any mixture of text to text data in one line of code"},"popularity":{"monthly_downloads":92135,"position":13476,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"1.26.10"}