Automated testing skills for AI agents

Ask an agent to add tests and it will add tests. They will be green on the first run. That is the problem, not the reward — a suite that has never failed has never been shown to work, and a model writing tests from its own general knowledge produces exactly that shape: arrange, act, assert, mock the database, chase a coverage percentage, done. Any of those tests can pass over code that is broken. A mock with no interface behind it happily accepts a method that was renamed last week. An assertion inside a promise nobody awaited never runs. A hardcoded return value satisfies the single case that got written.

The split across the testing skills available today is sharper than it looks from the titles. Most of them teach an agent how to write a test. A smaller set teaches it how a passing test lies, and hands it a switch that turns the lie into a red run instead. That second set is what to install, and the switches are specific enough to name.

Top picks

Triangulation: the step that forces real logic instead of a lucky pass

tdd from the Prowler repository admits what the green phase actually produces: its Phase 2 says to write the minimum code to pass, then shows the honest version of that, a function that returns a constant, commented "FAKE IT - hardcoded is valid for first test". Phase 3, which it labels CRITICAL, is a stage the familiar three-step loop does not have: "One test allows faking. Multiple tests FORCE real logic." It gives a required-scenario table — happy path, zero and empty values, boundary values, different valid inputs, error conditions — and marks "Different valid inputs" as the row that breaks the fake.

Compare the TDD section in python-testing, a clean RED-GREEN-REFACTOR loop followed straight by a coverage requirement of 80% overall and 100% on critical paths: nothing in that loop makes the agent break its own first assertion, and coverage will not catch the hardcoded return, which is fully covered. The caveat is that the Prowler skill is written for one codebase — it detects a stack by directory (ui/, prowler/, api/), runs pnpm test:coverage and uv run pytest, and declares Apache-2.0 in its front matter. The triangulation loop is portable; the paths are not.

pytest: the configuration that makes silence fail

pytest from cc-foundry routes fixtures, parametrize, monkeypatch and plugins out to four reference files, and spends the page itself on the parts that decide whether a test can lie. Its mocking rules are explicit about the trap: "Always use spec=" on MagicMock because it catches attribute typos at test time, autospec=True because it recursively specs method signatures from the real object, and "Never mock the thing you're testing", because needing to mock part of the system under test means the unit has too many responsibilities. Its configuration sets --strict-markers so a mistyped marker fails instead of being silently ignored, strict=True on xfail so a known failure that quietly starts passing fails the run, and a filterwarnings list that turns warnings into errors, deprecations from dependencies aside — and its plugin list carries pytest-randomly for one stated purpose: randomising test order to catch hidden dependencies.

pytest from claude-mpm-skills is twice the length and covers far more ground — FastAPI, Django, DRF, async fixtures, a full pytest.ini template — and its stated rule is to mock external dependencies. But its worked mocking example instantiates a service and then patches a method on that same object before calling another method on it, and its mocks are bare mocker.Mock() with no spec anywhere in the document; the breadth is real, and the discipline that makes a mock fail when the interface moves is not there. pytest-skill is a third shape again: a one-screen cheat sheet of fixtures, parametrize, markers and commands, with its depth in a reference/playbook.md beside it.

React: make the unmocked request fail the run

react-testing is where to start for an agent working on components, and the reason is a single configuration line with a justification attached. It sets up Mock Service Worker at the network layer with onUnhandledRequest: "error", on the stated grounds that any unmocked request should fail the test loudly because "silent passes are worse than red". Its anti-pattern list ends on a check you can run today: "Tests that pass with it.skip() removed — your test does not actually assert what you think".

It also draws the boundary honestly: JSDOM cannot render real layout, run native browser animation, or handle scrolling, drag-and-drop, iframes and cross-origin flows, so those belong in Playwright rather than being faked in a component test. react-testing-library is the better desk reference — its query-type table sets out what each variant does on no match, one match and many matches, and it notes that React 19 requires v16.1.0+ — but it stops at the DOM: no network layer, no coverage guidance, and a prohibition list mostly about query and event style rather than about tests that pass for the wrong reason.

Generating tests: the generator that pushes back

test-harness generates pytest suites, and what makes it worth installing over a plain "write tests for this" prompt is that it refuses jobs. It lists when not to generate tests at all: auto-generated code such as protobuf or ORM models, where the generator or the schema is the thing to test; UI and end-to-end requests, which are out of scope; code with no clear behaviour, such as pure configuration; and third-party library code, where your usage is the target rather than the library.

Its mock phase separates what to mock from what never to mock — the function under test, pure functions called by the target, data structures and value objects — and then connects mocking to the false pass: "Mocks without assertions are coverage holes." Its rationalisations table answers "100% coverage means the code is correct" with the correction that coverage measures execution, not correctness, and that a test which runs code without meaningful assertions adds no value. Coverage returns as a gate in its verification checklist — 80% overall, 90% on new code, 95% on critical paths — but as one item among seven that end on mocks used only at boundaries. The thresholds themselves, like its fixture design and mock strategy, sit in reference files the workflow loads when it needs them.

Spring Boot: the false pass named by annotation

spring-boot-test-patterns is the Java pick, and its distinguishing feature is that its warnings are about the Spring context rather than about testing in general. It attaches time budgets to test types — under 50ms for plain Mockito unit tests, under 100ms for slice tests such as @DataJpaTest and @WebMvcTest, under 500ms for full @SpringBootTest integration runs — which turns a preference for slices from advice into something an agent can check. Its constraints section is where the value concentrates: context caching is invalidated by differing @MockBean configurations, @TestPropertySource creates separate contexts, @DirtiesContext forces a rebuild, and @SpringBootTest is the wrong tool for a unit test.

Its sibling unit-test-application-events then names an exact false pass: @Async requires @EnableAsync, so a test using Thread.sleep may still pass even when the async proxy was never wired — the fix it gives is to verify against a mock instead. The rivals are narrower by design: junit-5-skill is a strong JUnit 5 and Mockito API surface with a four-row anti-pattern table, and test-quality is the better of the two on what not to test, saying plainly to skip trivial getters and setters, POJOs and generated code.

A name is a topic, not an identity

Skill names are not namespaced, so the name on a testing skill tells you the subject and nothing else. A skill called pytest can be a compact conventions document whose configuration section is the point, or a long framework manual with Django, DRF and FastAPI chapters. Add a suffix and call it pytest-skill, and you get a one-screen cheat sheet that keeps its material in a playbook file beside it. Three different jobs; the names barely separate them.

It gets stranger when the names are more specific. A skill called python-testing-patterns can open with a word-for-word identical description and the same "When to Use This Skill" list as another skill of that name and still be a different document underneath. The python-testing-patterns under LibreUIUX-Claude-Code runs ten numbered patterns inline, then sections on testing database code, CI/CD integration and worked configuration files. The python-testing-patterns under agents keeps a navigation tier and sends its detail to a references/details.md, and carries material the longer document does not have — retry-behaviour tests driven by mock side effects, and freezegun time-freezing for token-expiry logic. The python-testing-patterns under Claude-Cortex is built around a lookup table mapping tasks to reference files, with a short numbered workflow and a list of common mistakes either side of it. No front matter among them declares an origin, source or basis, so nothing marks one as the canonical version — treat them as independent documents that happen to collide, and identify the one you are holding by its body.

The check takes about a minute. Open the body, read the front matter before the prose, and answer three questions. Where is the substance — inline, or behind references/ files you may not be installing? What does it assume about your repository — does it name directories, package managers or sibling skills? And what does it actually change about the tests your agent writes, as opposed to what it describes? A skill that reads well and moves nothing is the expensive kind of wrong here, because its output is green either way.

House rules travel badly

Several of the skills above were written for one repository and then shared, which is fine as long as you notice. The Prowler TDD skill works out which stack you are in from the directory you are working in and calls pnpm test:coverage and uv run pytest; its vitest sibling ships the same pnpm scripts, mandates Given/When/Then structure and vi.spyOn over vi.mock, and hands end-to-end work to another skill in the same repository. The claude-mpm-skills pytest document carries a section of pytest settings collected from its author's own projects, naming repositories you do not have. These are not defects. They do mean the commands are illustrations rather than instructions, and an agent that follows them literally will try to run a script your package.json does not define.

Two front-matter checks are worth making a habit. First, invocation flags: the claude-mpm-skills pytest document sets user-invocable: false and disable-model-invocation: true, which means installing it is not the same as it firing — something in your setup has to route to it deliberately. A skill that never triggers is indistinguishable from a skill that does not exist. Second, licence: read the skill's own front matter and the repository record together, because they do not always say the same thing. The Prowler testing skills declare Apache-2.0 in their front matter and the repository agrees; the LambdaTest pytest and JUnit skills declare MIT and name TestMu AI as author, and the repository agrees there too. Where a skill declares nothing and the repository record is empty, treat it as no licence declared rather than as permissive by default, and decide before you vendor it into a codebase you ship.

What you have now

The failure this started from is quiet by construction: an agent that writes tests will hand you green, and green from a test that cannot fail looks exactly like green from a test that can. Nothing in a coverage percentage separates them, which is why picking on coverage numbers alone is picking on the one signal that is blind to the problem.

The five above each close a different part of it. Prowler's TDD skill closes the loop itself, forcing a second and third input until a hardcoded answer stops working. The cc-foundry pytest conventions close the Python side, where specced mocks fail when an interface moves, warnings become errors, unregistered markers stop being silently ignored, and randomised ordering exposes tests that only pass in sequence. The ECC React skill closes the network, where an unmocked request errors instead of quietly returning nothing. The armory test-harness closes the generation step, declining the jobs where a generated test would be theatre and insisting that every mock carries an assertion. The developer-kit Spring documents close the framework layer, where the trap is an annotation that was never wired and the test sleeps through its own failure.

Install the ones matching your stack, and read whatever each one calls its constraints, warnings or pitfalls before you read its examples. What you have avoided is the suite that goes green because nothing in it was ever able to go red.