JevHarness bets that separating LLM authoring from runtime judgment cuts agent latency

JevHarness is built on a specific bet about where latency and cost actually come from in agent loops: not the decision itself, but the full reasoning chain a large model runs to reach it. The solution here is to separate authoring from execution. A capable LLM writes a Python harness once - encoding its reasoning strategy into explicit feature computation, control flow, and structured questions - and then a lightweight judgment model called Jev handles every runtime decision without the authoring model present.

Jev answers three question types: choice, score, and noul. The harness constructs the inputs - a task-specific state object, named questions, answer types, and the instructions and criteria for each option - and Jev returns calibrated probabilities alongside its answer. The Pokémon battle example makes this concrete: the harness computes damage race estimates and legal action metadata, then asks Jev to pick one of four actions. Jev returns a choice with per-option probabilities, and the harness acts on it. The full decision takes a median of 568 ms in the archived evaluation run, with individual Jev requests at 269 ms median. Those numbers come from 12 evaluation games with cache hits excluded, so they reflect real network calls.

The improvement story is worth reading carefully. After five reflection rounds, the selected harness raised win rate from 25% to 75% on the evaluation set - but the README is explicit that this is performance on the selection eval, not on held-out games. The search kept the round-3 candidate as its best, not the round-5 one, which is the kind of detail that matters when you're deciding whether to trust a reported number.

Reflection works by feeding complete execution traces back to the authoring LLM: every decision, observation, Jev question and answer, memory state, and failure. The GEPA integration handles candidate selection from a frontier, compares parent and proposal on the same training batch, and records rejected proposals alongside accepted ones. The boundary between harness and task adapter is deliberately strict - the harness cannot rewrite its own reward or read hidden task state.

The runtime has real constraints. Functional Python nodes require a macOS native sandbox; if that isolation is unavailable, execution fails closed rather than running unsandboxed. Version 2 expression and Jev flows skip those Python workers entirely. The package and import names are auto-jev and auto_jev, which differs from the repository name - worth knowing before you start wiring dependencies.

Installation targets both Codex and Claude Code via a skill system, with a manual fallback that copies files into .agents/skills/ or .claude/skills/. The skill itself handles task elicitation: it asks about inputs, legal actions, examples, success criteria, and budget before generating any harness code. That scaffolding is where most of the practical value lives for someone building a new task from scratch.