codeflash-benchmark
Pytest benchmarking plugin for codeflash.ai - automatic code performance optimization
Decision gist · record as of 2026-08-14
Yes, if you are using CodeFlash's optimization service and need to measure performance impact in pytest. The low install friction and active maintenance make it a reasonable choice for that specific workflow. However, verify the license terms first, and confirm that the plugin's benchmarking output meets your performance measurement needs before committing to it in production.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.9 or later; pytest must be installed as a runtime dependency.
- Low install friction with a single runtime dependency on pytest.
- The package is actively maintained with recent releases, though repository metadata is not publicly available.
License · maintenance · safety
(unclear) — License status is unclear—no SPDX identifier or raw license text is declared. Verify the actual license terms before using in proprietary or restricted contexts.
last release 2026-02-19 (176 days)
0 known vulnerabilities (OSV.dev, 2026-08-14) · 151,172 downloads/mo, #10,941 on PyPI
Alternatives
Verify before relying
pip install codeflash-benchmark
import pytest
# Use in pytest tests; plugin integrates automatically with pytest- What license does this package actually use, and are there any restrictions on commercial or derivative use?
- Does the plugin require CodeFlash account credentials or API keys to function?
- What specific benchmarking metrics or output formats does the plugin provide?
What it is and what it does
codeflash-benchmark is a pytest plugin designed to integrate benchmarking into CodeFlash's automatic code optimization workflow. It extends pytest with performance measurement capabilities that CodeFlash uses to evaluate and validate code optimizations. The plugin depends only on pytest and installs with minimal friction.
The package is positioned as part of CodeFlash's optimization pipeline rather than as a standalone benchmarking tool. It is actively maintained and supports current Python versions (3.9+). However, the license terms are not declared, and the exact benchmarking features and metrics it provides are not detailed in the available documentation.
Use it for
- Measure performance of code before and after CodeFlash optimizations in an automated pipeline.
- Integrate performance testing into pytest suites used by CodeFlash's optimization service.
- Validate that code changes improve performance as part of continuous integration workflows.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are using CodeFlash's optimization service and need to measure performance impact in pytest.
The low install friction and active maintenance make it a reasonable choice for that specific workflow. However, verify the license terms first, and confirm that the plugin's benchmarking output meets your performance measurement needs before committing to it in production.
Install
codeflash-benchmark on PyPI
Before you install
Low install friction with a single runtime dependency on pytest. The package is actively maintained with recent releases, though repository metadata is not publicly available.
Requires Python 3.9 or later; pytest must be installed as a runtime dependency.
License in practice
License status is unclear—no SPDX identifier or raw license text is declared. Verify the actual license terms before using in proprietary or restricted contexts.
Quickstart
pip install codeflash-benchmark
import pytest
# Use in pytest tests; plugin integrates automatically with pytestVerify before relying
- What license does this package actually use, and are there any restrictions on commercial or derivative use?
- Does the plugin require CodeFlash account credentials or API keys to function?
- What specific benchmarking metrics or output formats does the plugin provide?
Package facts
| License | Not declared unclear |
| Python support | Supports the current Python release >=3.9 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 1 packagepytest |
| Maintenance | Actively maintained 176 days since the last release |
| First released | |
| Downloads | 151,172 / month, #10,941 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
Evidence: codeflash_benchmark-0.3.0-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “pytest benchmarking plugin”
- codeflash-benchmarkA pytest plugin that adds benchmarking capabilities for CodeFlash's…
- pytest-codspeedA pytest plugin that integrates performance benchmarking into your…
- pytest-benchmarkA pytest fixture that benchmarks Python functions by running them in…
Give your agent the search over MCP, or paste the wish link into any chat.
More Quality Assurance packages
Coverage.py measures which lines of Python code are executed during test runs, reporting coverage percentages and identifying untested code paths.
Install it if you want to measure test completeness or enforce coverage thresholds in your project.
Ruff is a Python linter and code formatter written in Rust that combines linting, formatting, and code fixing into a single tool, replacing Flake8, Black, isort, and related utilities.
Pexpect spawns and controls interactive console applications by sending input and matching output patterns, automating tasks that would otherwise require manual interaction.
Black reformats Python source code to a consistent style by parsing entire files and rewriting them according to an opinionated, deterministic set of rules, eliminating manual formatting decisions.
pytest-xdist distributes pytest tests across multiple CPU cores or machines to speed up test execution, with the simplest usage being `pytest -n auto` to spawn workers equal to available CPUs.
Install it if your test suite takes long enough that parallelization would save meaningful time.
Validates AWS CloudFormation templates in YAML or JSON format against resource provider schemas and best practices, checking property values and configuration correctness.
Install it if you work with CloudFormation templates.
See also codeflash · pytest-benchmark · asv-runner · asv · google-benchmark · msbench-utils · pytest-codspeed · pytest-harvest · pyperf · harbor