A decision table, not a feature list. Each row names the need, the approach, the condition that must hold, and the number that sets the expectation.
| You need | Use | Only if | Expect |
|---|---|---|---|
| A verifiable program for a problem with an exact scorer: combinatorics, kernels, scheduling | AlphaEvolve; FunSearch when one subroutine is the whole problem | A cheap automated evaluator exists. No evaluator, no method. | Real new results: 48-multiplication 4×4 complex matmul; OpenEvolve reproduced circle packing at 2.634 vs 2.635. |
| A closed-form law from small noisy data, library unknown | LLM-SR | Low-dimensional data, a physical prior the LLM can write as Python. | 31.5% symbolic accuracy on LLM-SRBench. Treat output as a hypothesis to check, not a result. |
| A closed-form law, library known | SINDy | True terms are in the library; derivatives are estimable; noise is low. | Exact recovery on Lorenz. Nothing outside the library, ever. |
| A coding agent that improves itself on a benchmark | Darwin Gödel Machine | You can afford the compute and sandbox the agent. Keep the archive; without open-ended exploration it ends at 23.0%. | SWE-bench 20.0% → 50.0%. Documented reward hacking of its own safeguards along the way. |
| A better agent design without touching weights | ADAS | Single-step QA-style tasks; one objective. | Gains over hand-built baselines; an independent re-run found them subtle and inconsistent. |
| A tuned LLM pipeline, no training | GEPA; DSPy MIPROv2 for demonstrations | More than a few dozen training examples. With 30, GEPA regressed 28.4 points on GSM8K-Hard. | +6% average over GRPO at up to 35x fewer rollouts; loses to GRPO on AIME-2025. |
| Textual feedback through a multi-call pipeline | TextGrad | Each node can be critiqued in words. | GPT-4o GPQA 51% → 55%. The gradient is a paragraph. |
| New facts held in the weights | MEMIT, then retrieval | Fewer than ~1,000 edits on GPT-J; ~4,000 on GPT2-XL. | Beyond that, GLUE collapses to zero. No readable diff of what changed. |
| The model to write its own training data | SEAL | You can pay ~6 hours on 2×2 H100 per RL round and accept forgetting across a stream of edits. | Self-written data beats GPT-4.1-written data on their tasks; GRPO/PPO were unstable, ReST-EM used instead. |
| Per-task adaptation at inference | Test-time training, LoRA | Compute per query is acceptable; the benchmark allows weight updates. | ARC public 53%, semi-private 47.5%. Some ARC Prize tracks disallow it. |
| Memory that keeps learning during inference | Titans | Long-context recall is the task. | Scales past 2M tokens. Surprise-gated weight updates, continuous, no audit trail. |
| A simulator fitted to data | Neural ODE / JAX-MD | Dynamics are not stiff. For dz/dt = −10⁴z the adjoint cannot be reversed in double precision. | Constant-memory gradients through the solver. Slow training; ANODE documents the instability. |
| A new network architecture | ASI-Arch | ~20,000 GPU-hours and acceptance that validation stopped at 1.3B parameters. | 105 or 106 architectures, depending on which version of the paper you read. |
| A change you can audit after a failure | Any program loop | Never ROME, MEMIT, SEAL or Titans for this need. | A source diff. The weight loop gives a norm shift. |
What no row covers. A loop where the LLM writes a mechanistic model as code, an optimizer fits its rate constants against measured biology, and the residual returns to the LLM as text. Each half exists on this hub. No paper has closed it. See the mechanistic biology page for the closest attempts.