Sparse linear regression over a hand-specified library of candidate nonlinear functions recovers the exact governing ODE as a short symbolic sum, with no LLM and no search.
lane: substrate · Brunton, Proctor, Kutz — "Discovering governing equations from data by sparse identification of nonlinear dynamical systems," PNAS 113(15):3932-3937, 2016 (arXiv:1509.03580, submitted 2015); de Silva, Champion, Quade, Loiseau, Kutz, Brunton — "PySINDy" (arXiv:2004.08424, JOSS 5(49):2104, 2020); Kaptanoglu, de Silva, Fasel, Kaheman, Goldschmidt, Callaham, Delahunt, Nicolaou, Champion, Loiseau, Kutz, Brunton — "PySINDy: a comprehensive Python package for robust sparse system identification" (arXiv:2111.08481, JOSS 7(69):3994, 2022)
Gives. Least squares plus a hard threshold over a fixed library returns the governing equation in seconds, no LLM.
Needs. A library that contains the true terms; clean derivative estimates; low noise.
Breaks. Nothing outside the library can be found; Al-Khwarizmi shows ODEBench 4 and 7 failing for this reason.
SINDy has no proposer loop at all: no LLM, no evolutionary search, no neural network. The program space is fixed in advance as a library Θ(x) of candidate nonlinear functions, such as polynomials and trig terms (arXiv:1509.03580 Eq. 5). Discovery is a single sparse linear regression: solve Ẋ = Θ(X)Ξ (Eq. 7) for a coefficient matrix Ξ, then zero out small entries. The entire search is which columns of a pre-built library survive thresholding. This is the most purely linear-algebra approach on this hub. A human replaces every other method's LLM-proposal step, by choosing the library basis beforehand.
1. Collect: measure state snapshots X(t) and either measure or estimate their time derivatives Ẋ (Eq. 3-4). The method denoises noisy derivatives with a total-variation-regularized derivative (Sec. 3.1) rather than naive finite differences, because finite-differencing noisy data amplifies the noise. 2. Build the library: Θ(X) stacks constant, polynomial, and trigonometric candidate columns evaluated on the data (Eq. 5). For the Lorenz demonstration, this is all polynomials in (x,y,z) up to fifth order (Eq. 23). 3. Fit and threshold: sequential thresholded least squares, STLSQ (Sec. 3.1), solves the regression, zeros coefficients below a threshold λ, and refits the remaining columns, repeating until the active set stabilizes. LASSO (Eq. 2) is an alternative. PySINDy (arXiv:2004.08424) also ships SR3, sparse relaxed regularized regression, as an optimizer. 4. Select λ: a human or an outer loop chooses λ by cross-validation and a Pareto "elbow" that trades accuracy against complexity (Sec. 3.2). This is the one place a discrete choice enters the method. 5. Keep: the method keeps a single sparse coefficient matrix Ξ, the final symbolic equation. There is no population and no archive; it converges to one model per run.
Everything is continuous and linear. The method solves Ẋ = Θ(X)Ξ by least squares, or its LASSO or SR3 relaxation, so the coefficient-fitting step is literally a linear solve, not even a gradient-descent loop. The one discrete, non-differentiable operation is the hard threshold that zeros small coefficients on each STLSQ iteration. That threshold decides which terms exist in the final program, and it is a hand-tuned hyperparameter, λ, not something learned end-to-end. No LLM or evaluator score crosses any boundary. The only external input is the human-specified library Θ, which bounds everything the method can ever discover.
SINDy needs dense, reasonably clean time-series measurements of the full state, or a library built from it, and assumes the true dynamics are a short linear combination of the chosen candidate functions. If the true term is not in Θ, SINDy cannot find it by construction. That is an assumption, not a measured failure; it is stated implicitly throughout Sec. 3-4 and made explicit in later critiques below. SINDy scales to high-dimensional PDEs via the weak or integral formulation (PySINDy 2111.08481), but costs grow with library size, which is combinatorial in state dimension and polynomial order. The method offers no way to discover a function outside the library, unlike an LLM-proposal loop such as LLM-SR or FunSearch, which can invent new structure. Where it wins: once the library is right, recovery is near-exact and essentially free compared to any iterative search, a single regression versus LLM-SR's thousands of LLM calls plus BFGS fits.
On the chaotic Lorenz system with a 5th-order polynomial library, SINDy recovers the exact quadratic terms and determines coefficients "to within .03%" of the true values at low noise (arXiv:1509.03580 Sec. 4.2). The method "well-preserves" the Lorenz attractor's structure even under large added derivative noise, η swept 0.0001 to 10.0 (Fig. 6), though trajectory-level ℓ2 error grows with η and with the system's positive Lyapunov exponent amplifying small coefficient errors (Sec. 4.2). At η=1.0, cubic-term coefficients were still off by nearly 8% (Sec. 4.4.2), even though the correct structure was identified. PySINDy's 2022 package update (arXiv:2111.08481) adds weak-form ODE and PDE formulations, ensembling, and SR3-family optimizers, specifically to extend robustness into noisy, limited-data regimes, per the paper's own stated purpose. The full numeric ablation is UNVERIFIED — WebFetch could not parse the PDF this session.
1. Classical SINDy is brittle on standard benchmarks, compared against an LLM-augmented pipeline: Al-Khwarizmi (arXiv:2502.01702 Table 2, Sec. 6.5) reports plain PySINDy reaching test R²>0.99 on only 54.1% of DYSTS systems and 33.3% of ODEBench's 63 models, versus their LLM-library-proposing pipeline's 83.0% and 85.7%. This is a direct, sourced demonstration that fixed human-chosen libraries underperform when an LLM adaptively proposes the library per problem. 2. Library must contain the true terms, by construction: if the governing dynamics are not expressible as a combination of the chosen candidate functions, SINDy cannot recover them. This is demonstrated directly in the LLM+SINDy paper Al-Khwarizmi, where ODEBench models 4 and 7 "aren't linear combinations of primitive functions" and generalization fails outright on them (arXiv:2502.01702 Sec. 7). 3. High-order library causes degeneracy: including higher-order polynomial terms in the Lorenz library produces identifiability "degeneracy." This forced the original authors to use a hierarchical library-growing approach instead of fitting the full library at once (arXiv:1509.03580 Sec. 4.1.2). 4. Off-attractor data matters: training only on on-attractor trajectory data for the Lorenz system causes SINDy to "incorrectly identify" a Hopf-normal-form model with spurious cubic terms, instead of the true quadratic Lorenz equations (Sec. 4.3.2). The regression is only as good as the data's coverage of state space. 5. Noise and derivative estimation remain open even in the robust package: Al-Khwarizmi's own authors note "noisy data isn't handled yet" in their LLM+SINDy pipeline, and that derivative computation may still need smoothing (Sec. 8.2). Four years after PySINDy's 2022 "robust" update, noise handling is still flagged as unresolved in follow-on work. 6. UNVERIFIED (WebSearch-only, PDF unparseable by WebFetch): Ensemble-SINDy (arXiv:2111.10992) is reported by search summaries to show standard STLSQ failing in low-data, high-noise regimes, fixed by bootstrap aggregation, "bagging," and library bagging — exact success-rate and error numbers could not be independently confirmed this session. A 2026 Fourier-weak-SINDy paper (arXiv:2604.20141, found via WebSearch only) reports classical SINDy's true-positive term-recovery ratio falling to roughly 0.2 on Lorenz under roughly 100% measurement noise — also UNVERIFIED by direct WebFetch.
SINDy, 2015–2016, is the foundational sparse-regression-for-dynamics method. SINDYc, which adds control inputs, and biological-network variants are the authors' own near-term follow-ups. PySINDy (2020, 2022) is the reference software implementation, adding STLSQ and SR3 optimizers, weak-form ODE and PDE support, and ensembling, specifically to answer the noise and robustness critiques leveled at the original method. Al-Khwarizmi (2025) is the clearest LLM-era descendant. It keeps SINDy's sparse-regression fitting core unchanged, and replaces only the human library-design step with an LLM that proposes Python code for the feature library and optimizer settings per problem. This directly measures the gap a fixed library leaves on the table: 83-86% versus 31-54% R²>0.99 success rates (Table 2).
Verified: WebFetch on https://ar5iv.labs.arxiv.org/html/1509.03580 (full text, Sec. 3-5, Eq. 2-10, 23, Fig. 1/4-6, Table 4); WebFetch on https://ar5iv.labs.arxiv.org/html/2502.01702 (Al-Khwarizmi, full text, Sec. 3-8, Table 1-2); WebSearch for PySINDy (2004.08424, 2111.08481), Ensemble-SINDy (2111.10992), and SINDy noise/limitation surveys — PDF binaries for 2111.10992 and the raw arXiv PDF of 1509.03580 could not be parsed by WebFetch, so those two numeric claims are WebSearch-summary only and flagged UNVERIFIED below. · code: https://github.com/dynamicslab/pysindy