flaky

What it is and what it does

Flaky is a pytest plugin that automatically reruns tests that fail, allowing you to handle tests that depend on unreliable external components—such as flaky network services or timing-sensitive fixtures—without skipping them or removing them from your suite. Instead of treating intermittent failures as permanent, you decorate a test with @flaky and specify how many times it should run and how many times it must pass to be considered successful.

The plugin works by intercepting test failures and retrying them according to your configuration. You can apply the decorator to individual test functions or entire test classes, override behavior per-test, and even provide custom filter functions to decide which failures warrant a retry and which should fail immediately. It integrates seamlessly with pytest and can be disabled via command-line flags if needed.

Use it for:

• Mark tests that depend on external APIs or services as flaky to avoid false test failures from transient network issues.
• Retry tests with timing-sensitive assertions that occasionally fail due to system load or scheduling variance.
• Apply flaky behavior to an entire test class while overriding retry counts for specific test methods that need different handling.
• Use a custom rerun_filter to skip retries for certain error types (e.g., application crashes) while retrying for others.
• Add delays between retries using a rerun_filter to allow external services time to recover before the next attempt.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A pytest plugin that automatically reruns failing tests up to a configurable number of times, allowing tests that depend on unreliable components to pass without being skipped or removed.

Yes, if you have tests that depend on unreliable components and want to reduce false failures without removing or skipping tests. The plugin is mature, widely used, and has no security vulnerabilities. The aging maintenance status is not a blocker for stable functionality, but verify compatibility with your current pytest version before adopting.

Install

flaky on PyPI

pip install flaky

uv add flaky

poetry add flaky

Installing flaky

Before you install

Low friction installation with no runtime dependencies. Maintenance status is aging—last release was in March 2024—but the repository remains active and the package is widely used (4+ million monthly downloads). Suitable for established projects where the plugin's core functionality is stable.

License in practice

Licensed under Apache Software License, Version 2.0. You may use, modify, and distribute the package freely in commercial and private projects, provided you include a copy of the license and do not hold the authors liable.

Quickstart

pip install flaky

from flaky import flaky

@flaky(max_runs=3, min_passes=2)
def test_something():
    # test code here
    pass

Requires pytest to be installed and used as the test runner. Doctests cannot be marked flaky.

Verify before relying

• Current compatibility with pytest versions released after March 2024 is not documented in the fact sheet.
• Whether the package works with modern pytest plugins and extensions beyond pytest-xdist is unclear.
• Support for Python versions beyond 3.8 (the latest listed in classifiers) needs confirmation.

Package facts

Evidence: flaky-3.8.1-py2.py3-none-any.whl

Keywords: pytest, plugin, flaky, tests, rerun, retry

Tags

More Testing packages