GameWAM's best contribution is naming a failure mode it can't yet fix

Two problems have long kept game agents from being truly autonomous: control models that pick actions without modeling what those actions will look like visually, and world models that predict visual futures without picking any actions at all. GameWAM collapses that gap by jointly generating future frames and executable keyboard-mouse trajectories through parallel Video and Action Diffusion Transformers trained with flow matching.

The action side is harder than it sounds. Minecraft's native interface uses the same physical mouse for camera rotation during gameplay and cursor movement inside crafting GUIs - identical hardware, completely different statistical distributions. GameWAM handles this with a per-timestep router that selects between gameplay- and GUI-specific prediction branches, each with its own normalization, while keeping a single shared action vector. The router can switch mode from one action step to the next, so a single trajectory can move through inventory management and then back into combat without handing off to a separate controller.

The temporal design is equally deliberate. Each planning unit predicts a longer action horizon than it commits to executing - only a short prefix is actually sent to the environment before the model replans from the new observation. This predict-long/execute-short structure gives longer look-ahead without delaying feedback. Across interaction cycles, a hierarchical memory keeps recent executed segments explicitly addressable while compressing older context into fixed-size long-term slots at two timescales, so memory cost stays bounded regardless of episode length.

On the Minecraft Universe benchmark, GameWAM achieves the highest average success rates on both the 30-task Mini set and the full 800-plus task set, while using fewer executed native actions than the compared agents across all task categories. On ViZDoom's four maps it is competitive or leading against the multimodal agents tested.

The paper's most distinctive contribution may be the failure mode it names rather than solves. Low-Frequency Action Source Imprinting (LASI) describes what happens when the random noise vector that seeds flow-based action generation is reused across replanning steps: low-frequency components of that source coherently steer coarse camera motion, and the bias compounds with each cycle until the agent spins in place and completes almost nothing. Controlled DCT interventions show that replacing only the low-frequency source coefficients with a donor's values causes the output to follow the donor in roughly 95 percent of trials, and zeroing those same coefficients removes over 99 percent of source-induced variance in yaw DCT mode zero. The fix used in evaluation - resample the source at every replanning step - breaks the accumulation but leaves the underlying sensitivity intact. The authors treat mechanism-level removal as an open problem, which is the honest position given that their training-time interventions all involved trade-offs against task performance.