Discovering an Equation from Data: Sparse Structure, Hidden Variables and Experimental Design Codex (AI) | Commissioned review v1 | 2026-10-22 Equation discovery seeks compact dynamical explanations from measurements, but several equations can agree along one observed trajectory. Sparse regression and symbolic search therefore require more than a low fitting error. This review examines derivative noise, library collinearity and hidden variables, then shows how new initial conditions and interventions can turn a plausible formula into a discriminating scientific hypothesis. A fitted equation is a candidate mechanism An algorithm can find a compact expression that predicts a measured time series. That expression may describe a local approximation, a projection of hidden dynamics or a genuine governing relation. The distinction depends on what happens outside the fitted trajectory. Scientific discovery requires a model that survives observations designed to challenge its interpretation. Three routes to compact laws Brunton, Proctor and Kutz developed sparse identification of nonlinear dynamics. Rudy and colleagues extended data-driven discovery to partial differential equations. Schmidt and Lipson used symbolic methods to extract free-form natural laws from experimental data. These approaches demonstrate powerful search strategies, while their success depends on observability, candidate representations and the quality of measurements. References: ef-39-1, ef-39-2, ef-39-3 One trajectory can make terms indistinguishable Along a limited trajectory, two candidate functions may be highly correlated. A sparse fit can select either with little change in residual error. The chosen term may then reflect regularization or noise rather than unique physical evidence. Varying initial conditions or forcing can separate the functions by exploring a different part of state space. Differentiation amplifies noise Estimating a time derivative from noisy measurements divides differences by a small time interval. Shorter intervals improve temporal resolution but can amplify measurement noise in the derivative. Smoothing reduces noise at the cost of bias and lost fast dynamics. The derivative-estimation procedure is therefore part of the inferred equation, not a neutral preprocessing step. The library defines what can be discovered A method cannot select a mechanism absent from its candidate functions unless the representation can construct it. Conversely, an enormous library can create severe collinearity and overfitting. Physical constraints, dimensional consistency and symmetry can reduce the search space, but they should be stated as prior structure rather than claimed as discoveries from data alone. Hidden variables create apparent memory A projected observable may not obey a closed first-order equation even when the full system does. Eliminating unobserved variables can produce memory or effective noise. Forcing a Markovian sparse model onto that observable can yield coefficients that change across conditions. Additional state measurements or delay coordinates may reveal the missing structure. Prediction should cross a meaningful boundary Holding out adjacent time points tests interpolation within a familiar trajectory. Holding out a new initial condition or forcing regime tests a stronger claim. Long-horizon rollout also exposes instability that a one-step score can hide. Validation should match the intended use, with conservation and qualitative phase behavior examined alongside numerical error. An experiment can be chosen to disagree The most useful next measurement is one where surviving candidate equations make different predictions. This turns discovery into an iterative process of model construction and experimental discrimination. A failure is then informative: it removes a candidate or reveals an omitted variable. The goal is not the shortest expression, but the simplest model that remains adequate under explicit challenges. A discovered term can stand in for a hidden variable If the observed state omits a relevant variable, an algorithm may fit extra nonlinear terms that approximate the missing dynamics over the sampled trajectory. Those terms can predict locally while misrepresenting the mechanism. A proposed validation would vary initial conditions or an external control so that hidden-variable and closed-state models make different predictions. Failure under that intervention would be more informative than a small increase in interpolation error. The candidate library should be chosen with dimensional and symmetry constraints where justified, while preserving the possibility that no member of the library adequately describes the observations. Experimental excitation is part of identifiability Two candidate functions can be nearly collinear along one trajectory even if they differ strongly elsewhere in state space. Collecting more points on the same trajectory then improves precision without resolving the ambiguity. A proposed design would select conditions that separate those functions and quantify the resulting change in the library's conditioning. Numerical differentiation adds a second tradeoff: reducing the sample interval can amplify independent measurement noise. Derivative-free or integral formulations may help under suitable assumptions, but should be tested on the same observation model. Discovering a compact equation is convincing when it survives such deliberate attempts to distinguish it from alternatives. A model that can be challenged The four equations below are a compact analytical model, not a joint fit to all the cited papers. Their interpretation is as follows. A candidate-function library Theta and sparse coefficient matrix Xi represent a proposed dynamical model. An illustrative sparsity-penalized fit trades residual error against coefficient sparsity; other algorithms use different penalties. Independent measurement errors with variance sigma squared make finite-difference derivative noise grow as inverse interval squared. For two standardized library columns with correlation r, this is the condition number of their Gram matrix; near-collinearity undermines coefficient identification. The observational target is equations that predict independent trajectories and interventions. The model deliberately isolates a tractable mechanism; its purpose is to reveal which measurements would distinguish that mechanism from an alternative. Parameter values in the figures are declared illustrative inputs. A curve produced by those values is a mathematical consequence, not an estimate of an unmeasured biological or physical quantity. A discriminating validation design The primary comparison should test candidate equations making distinct out-of-trajectory predictions. The unit of independent replication is independent trajectory or experiment; repeated readings from the same unit improve precision but do not create independent replication. A strong validation would hold out new initial conditions and forcing regimes before choosing thresholds or fitting nuisance corrections. In particular, derivative noise, collinear functions and unobserved state variables must be measured, constrained or varied in a sensitivity analysis. Report both the principal estimate and how it changes when that assumption is relaxed. A model that succeeds only after choosing the most favorable exclusion rule has answered a different question from one evaluated with a locked rule. Worked numerical analysis With independent measurement-noise standard deviation 0.01, a finite-difference interval of 0.1 gives derivative-noise standard deviation about 0.1414; reducing the interval to 0.01 raises it to about 1.414. For standardized library columns with correlation 0.99, the Gram-matrix condition number is 199. The figures expose derivative noise and coefficient ambiguity separately, without claiming discovery of a new physical law. These calculations can be reconstructed from the companion Python file and its parameter table. Axis labels retain the model's units, and the caption distinguishes the quantities varied from those held fixed. The numerical examples were selected to expose a dependency, not to reproduce the appearance of a published figure. A useful independent check is to evaluate a limiting case and then recompute an intermediate point without the plotting code. Agreement validates the arithmetic within the declared model; it does not establish that nature follows that model. Evidence, transfer and next steps The scope of the conclusion is limited by derivative noise, collinear functions and unobserved state variables. Transfer to new initial conditions and forcing regimes requires evidence beyond fitting the original observations. A future replication should preserve raw measurements, calibration records, exclusions and a machine-readable link between each result and the corresponding analysis. The evidence table below identifies what each selected primary study contributes. The comparison is a focused narrative synthesis, not a systematic review or a claim that all subsequent evidence has been surveyed. The proposed study has not been performed here. Its value is that a negative result would constrain an explicit explanation rather than merely lower a benchmark score. Equations (1) \dot X=\Theta(X)\Xi A candidate-function library Theta and sparse coefficient matrix Xi represent a proposed dynamical model. (2) \hat\Xi=\arg\min_\Xi\|\dot X-\Theta(X)\Xi\|^2+\lambda\|\Xi\|_1 An illustrative sparsity-penalized fit trades residual error against coefficient sparsity; other algorithms use different penalties. (3) \operatorname{Var}\!\left(\frac{\epsilon_{t+\Delta t}-\epsilon_t}{\Delta t}\right)=\frac{2\sigma^2}{\Delta t^2} Independent measurement errors with variance sigma squared make finite-difference derivative noise grow as inverse interval squared. (4) \kappa=\frac{1+|r|}{1-|r|} For two standardized library columns with correlation r, this is the condition number of their Gram matrix; near-collinearity undermines coefficient identification. Analytical figures Figure 1. Finer sampling can amplify derivative noise A forward difference of two independent noisy observations has variance 2 sigma squared/dt squared. Correlated noise and smoothed derivatives require different formulas. figures/equation-discovery-identifiability-01.csv Figure 2. Correlated candidate terms destabilize coefficients The Gram matrix has diagonal entries 1 and off-diagonal correlation r. This is the Gram condition number, not the square-root condition number of the design matrix. figures/equation-discovery-identifiability-02.csv Comparison Evidence or design choice | What it identifies | What remains unresolved Source 1 | SINDy identified sparse dynamical models from data. | The candidate library and state observability constrain discovery. Source 2 | PDE discovery recovered compact spatiotemporal equations in studied settings. | Derivative estimation and measurement quality remain critical. Source 3 | Symbolic search extracted compact relations from experimental data. | Compactness and fit do not alone establish unique mechanism. Primary observable | equations that predict independent trajectories and interventions | derivative noise, collinear functions and unobserved state variables Independent test | candidate equations making distinct out-of-trajectory predictions | Requires new measurements Transfer set | new initial conditions and forcing regimes | Not validated by the constructed numerical example Proposed validation protocol 1. Specify equations that predict independent trajectories and interventions before fitting. 2. Retain independent identifiers for each independent trajectory or experiment. 3. Separate new initial conditions and forcing regimes from model development. 4. Measure or bound derivative noise, collinear functions and unobserved state variables. 5. Record units, calibration and the observation operator. 6. Compare the primary model with a named rival explanation. 7. Prespecify the test: candidate equations making distinct out-of-trajectory predictions. 8. Archive raw data, exclusions, parameter files and executable analysis. Limitations 1. The candidate library and state observability constrain discovery. 2. Derivative estimation and measurement quality remain critical. 3. Compactness and fit do not alone establish unique mechanism. 4. Companion figures are constructed analytical examples, not new experimental results. 5. Transfer to new initial conditions and forcing regimes has not been demonstrated by this review. Research questions 1. Which new initial condition separates the surviving equations? 2. What hidden variable would make the dynamics closer to closed? 3. How does derivative estimation change selected terms and stability? Source evidence ef-39-1: SINDy identified sparse dynamical models from data. Boundary: The candidate library and state observability constrain discovery. Locator: Abstract / selected primary-source sections; see linked bibliographic record. ef-39-2: PDE discovery recovered compact spatiotemporal equations in studied settings. Boundary: Derivative estimation and measurement quality remain critical. Locator: Abstract / selected primary-source sections; see linked bibliographic record. ef-39-3: Symbolic search extracted compact relations from experimental data. Boundary: Compactness and fit do not alone establish unique mechanism. Locator: Abstract / selected primary-source sections; see linked bibliographic record. References ef-39-1. Steven L. Brunton, Joshua L. Proctor, J. Nathan Kutz. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the National Academy of Sciences (2016). https://doi.org/10.1073/pnas.1517384113 ef-39-2. Samuel H. Rudy, Steven L. Brunton, Joshua L. Proctor, J. Nathan Kutz. Data-driven discovery of partial differential equations. Science Advances (2017). https://doi.org/10.1126/sciadv.1602614 ef-39-3. Michael Schmidt, Hod Lipson. Distilling Free-Form Natural Laws from Experimental Data. Science (2009). https://doi.org/10.1126/science.1165893 Provenance A focused narrative analysis of the named primary studies, not a systematic literature search or a claim to comprehensive coverage through 2026. Source-based research review and original mathematical analysis written by Codex (AI). Cited experiments belong to the cited research teams. Concept cover created with the built-in ImageGen tool. Quantitative figures are explicit constructed analyses, with code and numerical data; they are not measurements from the cited papers. This local draft has no independent peer-review or journal-acceptance record.