buildkite-test-collector

What it is and what it does

Buildkite Test Collector is a pytest plugin that instruments your test runs and sends execution data to Buildkite Test Engine. It acts as the official Python adapter for the Buildkite platform, capturing test metadata and optionally tracing slow operations (HTTP calls, database queries, sleep intervals, and custom annotations) to help you identify performance bottlenecks and flaky tests.

The package integrates with pytest through a plugin mechanism and works across multiple CI systems—Buildkite, GitHub Actions, CircleCI, and others—via environment variables. You mark tests with execution tags, filter runs by those tags, and optionally instrument code to capture span data; the collector then transmits everything to your Test Engine dashboard. Span capture is currently manual (unlike the Ruby collector), so you write your own instrumentation or use the provided SpanCollector fixture.

Use it for:

• Monitor test performance and identify slow tests in your CI pipeline by collecting execution spans.
• Filter and run subsets of tests by execution tags (e.g., run only 'color:red' tests) for faster feedback loops.
• Centralize test analytics across multiple CI systems using a single Buildkite Test Engine dashboard.
• Trace HTTP requests, database queries, and sleep calls within tests to diagnose performance issues.
• Integrate test results into Buildkite's platform for historical analysis and trend detection.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Collects test execution data from pytest and sends it to Buildkite Test Engine for analysis, supporting test filtering by tags and optional span tracing for performance insights.

Yes, if you use pytest and Buildkite Test Engine. The package is actively maintained, has low install friction, carries a permissive MIT license, and no known vulnerabilities. It requires Python >=3.10 and an API token from Buildkite, but once configured it integrates transparently. Span tracing is powerful but requires manual instrumentation; if you only need basic test collection and filtering, this is a straightforward addition to your CI workflow.

Install

buildkite-test-collector on PyPI

pip install buildkite-test-collector

uv add buildkite-test-collector

poetry add buildkite-test-collector

Installing buildkite-test-collector

Before you install

Low friction install with four common runtime dependencies. Actively maintained with a release 4 days ago; repo is not archived and receives regular commits.

License in practice

MIT license permits free use, modification, and distribution with minimal restrictions—suitable for both open-source and commercial projects.

Quickstart

pip install buildkite-test-collector

import pytest

@pytest.mark.execution_tag("color", "red")
def test_example():
    assert True

# Set BUILDKITE_ANALYTICS_TOKEN env var, then run:
# pytest --tag-filters "color:red"

Requires Python >=3.10 and BUILDKITE_ANALYTICS_TOKEN environment variable set in your CI system.

Verify before relying

• Whether automatic span capture (http, sql, sleep) is planned or requires manual instrumentation for all use cases
• Support for test frameworks beyond pytest
• Performance overhead when collecting and transmitting test data at scale

Package facts

Evidence: buildkite_test_collector-1.8.0-py3-none-any.whl

Tags

More Testing packages