Smarter reasoning models need per-problem token budgets, not blanket compression

The core problem here is misallocation, not excess. Large reasoning models don't simply think too much — they think too much on easy problems and not enough on hard ones. Applying a uniform token budget or a global length penalty to fix this just shifts the damage: you save tokens on simple arithmetic and lose accuracy on olympiad problems. The paper calls this the efficiency tax, and it's a more precise diagnosis than the usual complaint about "overthinking."

When2Think addresses this by treating reasoning depth as an instance-level resource allocation problem. The mechanism is called Instance-level Difficulty-Aware Control (IDAC). Before each training epoch, a reference policy samples trajectories for every problem in the dataset and caches two statistics per instance: empirical accuracy (a proxy for difficulty) and average trajectory length (a token budget). During training, a correctness-gated efficiency bonus is scaled by how far the current trajectory deviates from that cached reference length — harder problems get more slack, easier ones get penalized for verbosity. No learned reward model, no online reference-model queries during policy updates.

Two supporting mechanisms matter. Batch-Wise Standardization (BWS) normalizes advantages across the full mini-batch rather than within each problem's rollouts. This preserves cross-instance difficulty structure: a long but successful trajectory on a hard problem can still receive a positive gradient signal, whereas GRPO-style within-problem normalization would fragment that signal into horizontal bands that obscure difficulty. Importance sampling enforces balanced Think/NoThink exploration during data collection, preventing the policy from collapsing into one mode early in training.

The ablations are unusually clean about what each component contributes. Removing IDAC while keeping the other two pieces produces a model that can switch between modes but cannot calibrate how deeply to reason once it enters Think mode — accuracy on hard benchmarks drops substantially. Removing importance sampling while keeping IDAC and BWS leaves accuracy largely intact but shifts the policy toward more verbose behavior on easy problems, because the NoThink mode gets less training exposure.

On AIME24, the full system raises Pass@3 by ten percentage points over the base model while cutting token usage by roughly 28%. On MATH-500 Level 1 problems, Think trajectories drop from the majority to a small fraction, while Level 5 problems retain near-complete Think coverage. The fraction of Think trajectories increases monotonically across all five difficulty levels — a clean behavioral signature that none of the compression or discrete-routing baselines reproduce.

The honest limitation: this only works where verifiable rewards exist. Mathematical reasoning has ground-truth answers; open-ended tasks don't, and extending IDAC there would require additional reward infrastructure the paper doesn't provide. The authors also note that difficulty estimation from offline statistics can be unreliable when problem difficulty is genuinely ambiguous. Both are real constraints on the scope of the result.