Large Discovery Models: Empirically-grounded Model-Based Open-Ended Search

Scientific discovery breaks the cheap-verifier assumption that powers modern LLM reasoning. You can't run a wet-lab assay a thousand times to check a candidate's score. The Large Discovery Model (LDM) is built around that constraint: it couples an LLM proposal process with a Gaussian process surrogate and an acquisition function, then runs them in a closed loop where each expensive evaluation updates the surrogate and redirects subsequent generation.

The key conceptual move is a three-way partition of search operations. Exploitation works within already-modelled, high-confidence regions. Exploration evaluates uncertain candidates already reachable by the current search. Discovery is different: it changes what the search can express at all, moving candidates from unknown unknowns into the modelled space. Standard Bayesian optimisation only does the first two; LDM adds the third by letting the LLM reshape the active search domain when the surrogate detects stagnation.

The acquisition function here is not a post-generation filter. It governs how inference-time compute is allocated—which candidate branches to expand, which to score, which to send for costly evaluation. The paper derives a closed-form acquisition-tilted optimal policy via the Gibbs variational principle, showing that LLM-only search and classical BO are the two limiting cases of a single framework, with LDM occupying the regime between them.

Across three domains—neural-network training program search, antibody CDRH3 sequence design, and multi-objective small-molecule discovery targeting KRAS G12D—LDM consistently outperforms both LLM-only and BO-only baselines. The ablations are the most instructive part. Removing the discovery mechanism (keeping the surrogate but fixing the feature set) causes premature convergence. Replacing the acquisition function with the posterior mean alone degrades performance once local optima are reached. Scaling the candidate pool helps only when the added candidates are filtered by a calibrated value model, not by language plausibility.

The fine-tuning experiments push further. A smaller student model trained on acquisition-weighted traces from a teacher's high-budget search transfers its policy to unseen targets and, more strikingly, to entirely unseen tasks. A model fine-tuned only on program-search and molecular trajectories, with no protein data at all, matches the in-distribution antibody model on four of five antigens. The one failure—antigen 1H0D_C—is instructive: its optimum requires de novo motif design, not just candidate selection, which demands in-domain sequence knowledge the task-level model never acquired.

Limitations are stated plainly. The GP surrogate scales cubically with observation count. Kernel functions still require manual customisation per domain. And the framework is bounded by the LLM's pretrained coverage—in domains with weak priors, the gains over specialised methods are smaller, as the antibody results show.

What the paper establishes is that the bottleneck in LLM-driven research loops is not proposal expressivity but value. A pure LLM agent with reflection and a full experiment log can do real sequential research—the ablation traces show genuine multi-phase hypothesis formation—but it plateaus because it has no calibrated estimate of where uncertainty remains high or when a boundary is genuinely exhausted. The surrogate and acquisition supply exactly that missing object.