Optical flow doesn't need correlation volumes, warping, or iterative refinement

Optical flow has accumulated a specific set of architectural habits over the years: correlation volumes, feature warping, iterative refinement, convex upsampling. These components work, but they also lock models into task-specific machinery that resists scaling and repurposing. FreeFlow asks whether all of it is actually necessary.

The answer, demonstrated across three major benchmarks, is no. FreeFlow is a hierarchical transformer encoder-decoder that uses no flow-specific components whatsoever. Its architecture combines three attention variants in a fixed per-layer order: window attention for local processing, shifted-window attention to pass information across tile boundaries, and a global attention block operating on a spatially downsampled map to keep compute tractable. That's the whole model. A lightweight three-layer prediction head maps the final decoded patch tokens directly to flow and uncertainty outputs, with no iterative stages and no specialized upsampling.

The benchmark numbers are hard to dismiss. On Sintel, FreeFlow-L reaches 0.68 EPE on Clean and 1.48 on Final, improving over the previous best on Clean by 14% relative. On KITTI-2015 it achieves 3.23 Fl-all, outperforming all non-stereo, non-multiframe methods. On Spring it sets the best EPE and Fl among two-frame methods. These results come from a model that runs native 1080p inference without tiling, which matters: the paper's comparison against late-feature-fusion alternatives shows that fusing only at the decoder stage leaves global coherence weak, particularly when large displacements span tile boundaries.

The hierarchical attention design is the key engineering choice. Window and shifted-window blocks handle local detail and cross-tile communication throughout the network rather than only at the end. The global block, applied after a stride-2 downsampling, injects long-range context at a cost comparable to the windowed blocks. This dense feature exchange throughout the network is what separates FreeFlow from tiling-based predecessors like CroCo-Flow, where tiles interact only through late averaging.

Scaling behavior is clean. Moving from small to large variants yields consistent accuracy gains without architectural changes, a property the paper attributes directly to the uniform encoder-decoder structure. The ablations also show that the masking ratio used during cross-view completion pretraining interacts with the attention configuration: at a 0.95 masking ratio (higher than CroCo's default 0.9), the global attention block becomes important, while at 0.9 its removal has little effect.

One honest caveat from the paper itself: zero-shot transfer does not improve monotonically with model size. Without broad motion-diverse pretraining data, larger capacity alone doesn't help. Methods pretrained on large video datasets like Kinetics-400 still lead on zero-shot Sintel. FreeFlow's strength is in the fine-tuned regime, where its architectural simplicity and high-resolution processing combine effectively.