Realtime-Venus: A full-duplex interaction system with asynchronous delegation

Realtime-Venus splits a conversational AI system into two concurrent loops that never have to pause for each other. The foreground loop handles live perception and speech; the background loop executes tool calls and reasoning tasks. When the frontend emits a private <delegate> span, the Harness captures a snapshot of the evidence available at that exact moment, runs the task asynchronously, and returns a polished reply through a private <backend> channel. The frontend then decides when to speak that reply—without ever having stopped listening or talking.

The two frontend models, Realtime-Venus-Omni and Realtime-Venus-Audio, are separately trained 9B models built on MiniCPM-o 4.5. Both use a one-second chunk cadence and predict a <|listen|> or <|speak|> token at each boundary, which is how the system distinguishes a backchannel (preserve the current response) from a genuine interruption (emit <|turn_eos|> and revise). That distinction is trained, not rule-based, which matters: acoustically similar overlaps can require opposite responses depending on conversational function.

On video benchmarks, Realtime-Venus-Omni leads the compared online models on six of eight tasks, including StreamingBench at 70.2% and Daily-Omni at 81.3%. The audio frontend leads on MMAU (78.0%) and MMAU-Pro (63.2%) among compared models. On Full-Duplex-Bench v1.5, Realtime-Venus-Audio responds to 75% of interruptions while maintaining continuation rates of 97%, 88%, and 86% under backchannels, other-directed speech, and background speech—exceeding Gemini 3.1 Live and GPT-4o on all three continuation metrics. The trade-off is real: its interruption-response rate trails Joy-Duplex, GPT-4o, and Gemini 3.1 Live.

Tool use tells a more complicated story. On Full-Duplex-Bench v3, Realtime-Venus-Omni reaches 86.0% tool selection F1 but only 43.0% Pass@1—the metric requiring correct tool selection, no extra calls, and accurate arguments throughout. GPT-Realtime leads at 60.0% Pass@1. Argument grounding under natural speech, with its fillers and false starts, remains the hard part.

The delegation decision benchmark reveals a structural asymmetry: Realtime-Venus-Audio recalls 92% of requests that genuinely need external capabilities but delegates far too many routine ones (non-delegation specificity of 39%). Realtime-Venus-Omni is the opposite—it avoids unnecessary delegation reliably (84% specificity) but misses a third of cases that actually need it. Neither model has solved the routing problem cleanly.

The training corpus exceeds 2.8 million samples, with offline understanding at roughly 56%, proactive duplex interaction at 37%, and delegation workflows at 6%. The long-video memory module—training-free, motion-gated, and MMR-inspired for retrieval—adds up to 5.88 percentage points on the 60–90-minute LVOmniBench bin without touching the fine-tuned policy.