Gradient boundary

Where the continuous part ends and the discrete part begins, approach by approach. What crosses the line: a score, a residual, a paragraph.

Rule

Put the boundary where the optimizer can do its job and the proposer cannot. Constants, kinetic rates and adapter weights go below the line. Form, topology and control flow stay above it. The signal that crosses decides how fast the loop learns.

The spectrum

Approach	Continuous, fitted	Discrete, searched	What crosses
FunSearch	nothing	one subroutine's text	one scalar score
AlphaEvolve	nothing in the loop; integers snapped after the fact	whole files	score vector from an evaluation cascade
ADAS	nothing	agent `forward()` code	scalar score plus the agent's own code
DGM	nothing; the LLM stays frozen	the agent's own source files	pass/fail per task plus free-text logs
GEPA, TextGrad	nothing	prompt and pipeline text	a paragraph of critique; the gradient is a string
LLM-SR	`params` by BFGS or Adam on MSE	the skeleton `f(x, params)`	negative MSE as a scalar into the next prompt
SINDy	coefficients by least squares	which terms survive the hard threshold λ	nothing; the library is fixed by a human
ASI-Arch	the full weight tensor of each candidate	operators, gating, recurrence form	loss and benchmark numbers plus Analyst text
Neural ODE / JAX-MD	the right-hand side or energy function, by adjoint	solver scheme and step control	nothing; no search step
FLeCS, CLAMP	kinetic and edge parameters	topology, fixed from priors	nothing; no LLM
DreamCoder	recognition network weights	primitives and programs	solved programs into the library
Scallop	neural predicates	hand-written logic rules	gradients through provenance semirings
SEAL	LoRA adapters, rank 64 or 128	the self-edit text	downstream accuracy after the update
TTT, Titans	LoRA per task; the memory MLP per token	task selection and augmentations, by hand	loss; surprise as a gradient norm
ROME / MEMIT	one MLP matrix, closed-form least squares	which layer, chosen once by causal tracing	nothing; the edit is committed unchecked

Two ways to cross

A scalar is safe and slow: FunSearch learns from a number. A trace is fast and leaky: GEPA learns from the whole log and needs 35x fewer rollouts, then overfits a 30-example set by 28.4 points. The mechanistic-biology papers cross nothing because nothing proposes; that is the gap this hub keeps pointing at.

Where the continuous side fails on its own

The adjoint is not free. ANODE shows that reversing dz/dt = −10⁴z cannot be done in double precision and that a ReLU ODE accumulates 1% reconstruction error after 11 adaptive steps. SINDy's one discrete operation, the threshold λ, is hand-tuned. The weight side has the same shape of problem: ROME's edited layer drifts in norm until it is incompatible with the rest of the network.

What follows

The boundary is a design choice, and it is documented in every paper as a sentence about what the optimizer touches. Read that sentence first. It predicts the failure section.