FunSearch lets an LLM mutate one isolated priority function with zero problem context, while an islands-model genetic algorithm and automated evaluator drive discovery.
lane: program loop · Romera-Paredes et al., "Mathematical discoveries from program search with large language models," Nature 625:468-475 (2023), published online 2023-12-14 — Google DeepMind
Gives. Proposals from a cheap model that cannot recall the answer: it never sees the problem, only two scored versions of one function.
Needs. A human skeleton with one open function and an exact scorer under 20 CPU-minutes.
Breaks. Shallow by design; a stronger reasoner has nothing to reason about (Davis).
FunSearch never tells the LLM what problem it is solving. The human supplies a solve(n, priority) skeleton, such as a greedy algorithm for cap sets, and an evaluate(solution) scorer. FunSearch shows the LLM two scored instances of the single mutable function, named "priority function 1" and "priority function 2," and asks only for "priority function 3." The LLM never sees the skeleton, the evaluator, or a description of the cap-set problem (Davis review, Sec. 3). Every other LLM-in-the-loop program-search method in this hub gives the model some problem framing. FunSearch strips that down to bare code-continuation, and the genetic-algorithm superstructure, not the LLM, carries all problem-specific guidance.
1. Propose: FunSearch prompts the LLM, Codey, a PaLM-family code model in the original paper, with two high-scoring priority functions sampled from an islands population, and asks it to generate a third in the same style. This runs millions of times (Davis review Sec. 3, pseudocode reproduced from the paper). 2. Fit/Judge: FunSearch splices the candidate priority function into the fixed skeleton, runs it, and scores it with the user-supplied evaluate function. For cap sets this is simply the size of the constructed set; for bin packing it is average performance across a benchmark collection (Davis Sec. 3). No numeric optimizer touches the output; the LLM's text output is the whole candidate. 3. Keep: FunSearch adds scored (priority-function, value) pairs to a population. The paper's full method, not shown in Davis's simplified pseudocode, runs this with an islands model for diversity and "a strategy to improve idea diversity," executing in parallel (DeepMind blog). The GitHub implementation directory holds the evolutionary algorithm and code-manipulation routines, but it explicitly ships without the LLM, the sandbox for untrusted code execution, or the distributed infrastructure used in the paper (repo README). 4. FunSearch returns the best-scoring program in the population on demand. A human, in the one documented case mathematician Jordan Ellenberg, can read the output code and feed insight back in.
There is no continuous/gradient component anywhere in the loop. The entire search is discrete: LLM text generation proposes code, the fixed-skeleton interpreter executes it, and a scalar scorer ranks it. The only signal crossing any boundary is that scalar evaluation score, which determines population membership and future prompt sampling — never a gradient, residual vector, or natural-language critique. This is the most purely discrete member of the program-loop lane in this hub.
FunSearch needs, per the authors' own statement, quoted in Davis Sec. 4.4: "a) availability of an efficient evaluator; b) a 'rich' scoring feedback quantifying the improvements (as opposed to a binary signal); c) ability to provide a skeleton with an isolated part to be evolved." Davis adds a fourth implicit condition: the problem must be solvable by the specified algorithm structure, for example greedy, applied to some attainable priority function, a condition one can rarely verify in advance. Evaluation must be cheap enough to run millions of times. The paper notes evaluation can take minutes per call for problems of interest, which caps parallel sampler and evaluator throughput (WebSearch synthesis of paper limitations table). This confines FunSearch to problems with a fast, graded, automatable scorer and a known good algorithmic skeleton, not to open-ended scientific problems that lack such structure.
From the paper's own results, as reported and cross-checked in Davis's review (Sec. 2):
1. Ernest Davis's review (cs.nyu.edu, 2024-01-07) argues the shallowness of the LLM's role is total: "the LLM in FunSearch is remarkable for the shallowness of the mathematical understanding that it seems to exhibit... the LLM is not told either what problem is being addressed or what the overall program is." He argues an AGI-level math reasoner dropped into the same architecture would have no opportunity to use its abilities, since it is never told what it is working on (Sec. 4.3). 2. Narrow applicability, confirmed by the authors: FunSearch "works best" only under the three conditions above; Davis notes that "works best" should probably read "works at all" (Sec. 4.4). That is a hard restriction, limited to scoring-function problems with a cheap evaluator and a correct algorithmic skeleton, ruling out most open scientific discovery tasks. 3. DeepMind's own RL comparison attempt failed to generalize: corresponding author Alhussein Fawzi told Davis by email, 2023-12-18, that an initial reinforcement-learning approach searching directly in the construction space "didn't look promising — it didn't scale in particular to large problem sizes." DeepMind never tried RL in the space of functions, which is closer to what FunSearch does (Davis Sec. 4.9). 4. Cherry-picking left unaddressed: Davis notes that the published paper does not disclose what other problems, if any, FunSearch was tried on and failed at (Sec. 4.10). That is an open generality question the authors did not answer in the paper. 5. "Kolmogorov complexity" framing challenged: the authors claim FunSearch's advantage comes from an implicit bias toward low-Kolmogorov-complexity, short, programs. Davis calls that a "red herring." He shows that a roughly 2000-bit brute-force program could in principle compute the same quantity exactly, which undercuts the compression argument as an explanation for why FunSearch scales (Sec. 4.7).
LLM-SR, covered in llm-sr.md, took up FunSearch's genetic-programming-with-LLM-mutation structure and islands-model population diversity strategy directly, and explicitly cites FunSearch's islands model (Romera-Paredes et al. 2024) for its own population buffer. AlphaEvolve, covered in alphaevolve.md, is DeepMind's direct successor. It generalizes FunSearch's single-function-skeleton restriction to whole-file and whole-codebase evolution, and swaps the single LLM for a Flash/Pro ensemble with an evaluation cascade. FunSearch itself borrows from classical genetic programming, mutation and selection over program populations, but is the first in this lineage to use an LLM as the mutation operator rather than random syntactic edits.
Verified: WebFetch on DeepMind blog post, GitHub repo README, Wikipedia FunSearch page; full WebFetch+pdftotext read of Ernest Davis's review PDF (cs.nyu.edu/~davise/papers/FunSearch.pdf, dated 2024-01-07); WebSearch standard + extended for numbers and critiques. Nature article itself paywalled, not opened directly — numbers cross-checked against Davis's review, which quotes the paper's tables directly. · code: https://github.com/google-deepmind/funsearch (Apache 2.0 for code, CC-BY for other materials; partial release — no LLM, no sandbox, no distributed infra included)