{"id":"0e729c79-2656-4b21-bf1e-cabb3cb5e54f","arxiv_id":"2508.19487","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"EQUATE fine-tunes a pre-trained symbolic regression transformer with symbolic-numeric alignment and evaluator-guided embedding search, outperforming its backbone and most baselines on Feynman, Strogatz, and black-box benchmarks.","lead":"This paper introduces EQUATE, a method that fine-tunes a pre-trained equation-discovery foundation model on small domain-specific datasets by aligning symbolic and numeric representations and using an evaluator to guide a search for better equations. If the results hold, it offers a practical path to extracting interpretable equations from small data, which matters for scientific discovery in data-scarce fields.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4.1 never states which data split is used for the R2 that stops the search; if it is the test split, the reported gains over E2E and SRBench baselines could be test-set selection artifacts.","rationale":"The reader's weakest assumption, evaluator miscalibration, is a legitimate concern, but it is not the most load-bearing one. The search loop is already gated by an explicit R2 threshold; if that threshold is evaluated on training data, a miscalibrated evaluator mostly degrades search efficiency rather than the final accuracy of the selected equation. The stopping-rule ambiguity in Section 4.1 is more fundamental because it determines whether the reported test numbers are unbiased estimates of the method's performance or best-of-N selections made with access to the test labels. The text never specifies which split is used for the R2 in \"the search stops once an equation with R2 > 0.99 is found,\" and the Table 1 caption defines the reported metric on the test set. The printed R2 formula in Equation 3 and Section 4.1 is also non-standard, using the equation-estimated labels in the denominator, so the paper does not even clearly fix the quantity being optimized. This is a fixable but critical experimental-protocol issue. If the authors can confirm the stopping R2 is computed only on training or validation data, and re-run the evaluation with an explicit protocol, the conditional acceptance would be justified; if the stopping R2 used test labels, the headline comparison is unsupported as reported. I therefore keep the reader's CONDITIONAL verdict but identify a different primary concern, which is why my agreement with the reader's weakest assumption is only partial.","tokens_in":18196,"tokens_out":9364,"duration_ms":89382,"concrete_test":"Ask the authors to supply a runnable evaluation script in which the R2 used for early stopping and candidate selection is computed exclusively on the training split, never on the test split, and re-run Table 1 and Figure 3 under that protocol. If the reported R2 > 0.99 and R2 values decrease materially, for example if the Strogatz R2 > 0.99 advantage over E2E shrinks by more than 0.05, the headline improvement is inflated by test-set selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that EQUATE consistently outperforms E2E and SRBench baselines. The reported metrics are test-set R2 and the proportion of equations with test R2 > 0.99, as stated in the Table 1 caption. However, Section 4.1 says: \"The search stops once an equation with R2 > 0.99 is found or the maximum number of candidates reaches 100,\" without specifying whether this R2 is computed on the training split, a validation split, or the test split. If the stopping check is performed on the test split, EQUATE is selecting the best of up to 100 candidates directly against the labels used for evaluation, while the E2E baselines decode without this test feedback. The large improvements in Table 1, such as Strogatz R2 > 0.99 rising from 0.357 to 0.786, could then be largely an artifact of test-set selection rather than evidence for the embedding-search mechanism. This concern is more load-bearing than evaluator miscalibration: even a poorly calibrated evaluator can appear effective if every candidate is filtered by true test R2 before reporting. The ambiguity is compounded because the printed R2 formula in Equation 3 and Section 4.1 is not the standard R2, since its denominator uses the equation-estimated labels instead of the gold labels, so it is unclear what quantity is actually optimized and thresholded.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EQUATE, a fine-tuning framework for the end-to-end symbolic regression transformer E2E. EQUATE constructs a task-specific fine-tuning set by sampling subsets of the given dataset and generating candidate equations with the pretrained backbone, then learns a shared symbolic-numeric embedding space with a data encoder, an LSTM equation encoder, an attention-based fusion module, and an MLP evaluator that predicts a fitness-and-simplicity score. At inference, it performs gradient ascent in the fused embedding space under evaluator guidance, decodes the optimized embedding into an equation, and stops when a candidate with R2 > 0.99 is found or 100 candidates are generated. Experiments on Feynman, Strogatz, and black-box benchmarks report improved R2 and R2 > 0.99 over the E2E backbone and several SRBench baselines, together with ablations, a noise-robustness study, and an analysis of the accuracy-simplicity trade-off.","tokens_in":18456,"tokens_out":5499,"duration_ms":51356,"significance":"If the reported results withstand scrutiny, the paper makes a useful practical contribution: it shows a lightweight way to adapt a pretrained equation-discovery foundation model to small domain-specific datasets, combining symbolic and numeric modalities in a shared embedding space and using a learned evaluator to steer a continuous search. The framing of equation discovery as embedding search rather than pure token decoding is well motivated, and the paper explicitly evaluates complexity, inference time, and noise robustness, which are all relevant to practical symbolic regression. The main strengths are the clear problem statement, the use of publicly available E2E checkpoints and SRBench benchmarks, and the component ablations that isolate the equation encoder and embedding-space optimization. However, two evaluation-protocol issues currently prevent full support of the central claim: the data split used for the stopping criterion is unspecified, and the printed R2 formula is non-standard. These issues must be resolved before the performance comparisons can be taken as evidence for the proposed mechanism.","major_comments":[{"comment":"The paper never states which data split is used for the R2 that triggers early stopping: 'The search stops once an equation with R2 > 0.99 is found or the maximum number of candidates reaches 100.' If this R2 is computed on the test split, EQUATE is selecting the best of up to 100 candidates using the exact labels used for evaluation, whereas the E2E baselines decode without this test feedback, so the large improvements in Table 1 (e.g., Strogatz R2 > 0.99 rising from 0.357 to 0.786) could be test-set selection artifacts rather than evidence for the embedding-search mechanism. The authors must specify the split explicitly and, if test data are used for stopping, rerun the comparison with stopping decided on training or validation data only, or give all baselines the same candidate-selection protocol.","section":"Section 4.1"},{"comment":"The R2 formula is written with the equation-estimated labels in the denominator: r = 1 - Σ(\\tilde{y}_i - y_i)^2 / Σ(\\tilde{y}_i - \\bar{y})^2, where \\bar{y} is the average true label. The standard coefficient of determination uses Σ(y_i - \\bar{y})^2 in the denominator. As printed, the quantity used for evaluator training, the early-stopping criterion, and the reported metric is not standard R2 and can differ substantially from the SRBench baselines' definition, especially for poorly fitted equations. Please replace the denominator with Σ(y_i - \\bar{y})^2 (if that was intended) and confirm that all R2 values in Tables 1-2 and Figures 3-6 are computed with the standard definition; otherwise, the reported comparisons are not on the same scale as the baselines.","section":"Equation (3) and Section 4.1"},{"comment":"The gradient-ascent search moves embeddings according to the evaluator's predicted fitness, but the paper reports no validation that the evaluator's predictions correlate with actual R2 in the embedding region visited by the search. Without such calibration, the improvement attributed to 'evaluator-guided embedding space optimization' could instead arise from the candidate-selection or early-stopping rule. Please report a calibration analysis (e.g., predicted versus true fitness on held-out data-equation pairs, and actual R2 along the search trajectory) and, if needed, re-evaluate the ablation in Section 4.5 with this evidence.","section":"Section 3.4 and Section 4.5"}],"minor_comments":[{"comment":"The phrase 'the search step size to a maximum of 20 steps' conflates the gradient step size with the number of gradient steps; please specify the numerical value of the step size η and the number of steps separately.","section":"Appendix A.6"},{"comment":"The noise-robustness experiment does not state the noise distribution, whether noise is added only to training labels, how many repetitions were averaged, or whether error bars are shown; please provide these details.","section":"Section 4.4"},{"comment":"The Black-box column for the frozen row lists a dash for R2 > 0.99; please explain why this metric is omitted for that condition.","section":"Table 2"},{"comment":"Figure 3(b) labels the baseline as 'SINP', while the text and baseline list use 'SNIP'; please correct the label.","section":"Figure 3"},{"comment":"The ablation figure reports only point estimates without numerical values or error bars; please report the exact metrics and their variance across runs.","section":"Section 4.5"},{"comment":"The claim that EQUATE maintains complexity 'either lower than or comparable' to the backbone is not supported for Feynman, where EQUATE-Sampling complexity is 57.76 versus 53.61 for E2E-Sampling; please qualify this statement.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the experimental setup is largely standard, but the stopping-rule ambiguity and the non-standard R2 formula are load-bearing for the central claim. If the authors can clarify the split used for early stopping and recompute the metrics with a standard R2 definition, the contribution may be publishable after a careful revision. I would also urge the authors to share code or detailed hyperparameters, since several search parameters (η, number of initial points, selection of top-performance instances) are currently underspecified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — you should know two things up front: EQUATE is a genuinely new way to adapt a symbolic regression foundation model to small datasets, and its main empirical claim is currently undercut by a stopped-rule ambiguity that has to be resolved before you can trust the numbers.\n\nWhat is new: the dual-encoder design (frozen transformer data encoder plus LSTM equation encoder plus attention fusion), a learned evaluator for fitness and complexity in embedding space, and gradient search in that shared space. That combination is not in E2E, TPSR, or SNIP. Distilling the same pretrained backbone's own outputs into a task-specific model is a reasonable idea, and the ablations in Figure 5 show both the equation encoder and the embedding-space optimization contribute. The benchmarks are standard and broad, and the Strogatz gains are large.\n\nThe soft spots are real. The biggest is Section 4.1: \"The search stops once an equation with R2 > 0.99 is found\" with no statement of whether that R2 is computed on the training split or the test split. The Table 1 caption reports test-set R2. If the stopping check is on test labels, EQUATE is selecting up to 100 candidates directly against the evaluation labels while the E2E baselines decode without that feedback, and the Strogatz jump from 0.357 to 0.786 could be an artifact. That is not a minor typo; it decides whether the central comparison is fair. The printed R2 formula in Equation 3 and in Section 4.1 is also nonstandard, with the equation-estimated labels rather than the gold labels in the denominator, so I cannot tell what quantity is actually being thresholded. There are no error bars or multiple seeds, no code release, and the \"preserved complexity\" claim is contradicted on Feynman (57.76 vs 53.61 in Table 1). Evaluator miscalibration is a legitimate secondary concern, but the split ambiguity matters more.\n\nCredit where earned: the framework is coherent, the experiments use standard public benchmarks, and the appendix traces one gradient-search run improving a single function. That is not enough to validate the headline, but it is enough to justify a careful review.\n\nThis paper is for researchers working on foundation models for symbolic regression. It deserves a serious referee, not a desk reject. My recommendation: send it to peer review with an explicit request to report which split drives the stopping rule, correct the R2 definition, add seed variance, and release code. If the stopping check turns out to be on test labels, the main comparison needs to be rerun without test feedback.","headline":"Novel fine-tuning framework for symbolic-regression foundation models with promising gains on standard benchmarks, but a load-bearing ambiguity about whether the search stops on test labels makes the headline results unverifiable as reported.","tokens_in":19031,"tokens_out":2692,"would_cite":false,"duration_ms":26307,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EQUATE turns discrete equation search into continuous embedding-space optimization, guided by a learned fitness evaluator, and reports better equations across three benchmarks at low complexity.","keywords":["symbolic regression","equation discovery","foundation model distillation","embedding space optimization","evaluator-guided search","transfer learning","data-efficient learning","transformers"],"falsifier":"Log the evaluator's predicted fitness and the actual $R^2$ of the decoded equation at every gradient step on a fixed benchmark equation; systematic divergence between predicted and actual scores as the search moves away from its starting embedding would show the evaluator is miscalibrated. A complementary check is to replace the evaluator with the true $R^2$ computed on the same sampled subset and require EQUATE to match that oracle and beat a random walk in the same embedding space by roughly the claimed margin.","tokens_in":17957,"feed_emoji":"🧪","tokens_out":12891,"duration_ms":109594,"temperature":0.7,"pith_summary":"This paper tries to establish that a generic pre-trained equation-discovery transformer, which by itself often transfers poorly to a single small domain-specific dataset, can be adapted to that dataset through a distillation-style fine-tuning loop. The proposed framework, EQUATE, samples many small subsets of the target data, uses the backbone to generate candidate equations, labels each data–equation pair by a $R^2$-score plus a simplicity term, and learns a shared embedding space for numerical data and symbolic equations. At inference it replaces beam search or sampling with gradient ascent in that embedding space, guided by a learned evaluator, then decodes the optimized embedding into the final equation. On the Feynman, Strogatz, and black-box benchmarks the paper reports consistent gains over the backbone and most comparison methods, with the largest gains on Strogatz where the backbone is weakest. If correct, this is a practical route to data-efficient symbolic regression: small annotated datasets become enough to specialize a large generative model without retraining it from scratch.","feed_headline":"Foundation-model distillation finds better equations from small data","feed_subtitle":"EQUATE turns discrete equation search into continuous embedding search and beats the backbone on three benchmarks.","key_machinery":"The load-bearing object is the fused embedding of a data–equation pair, $E_f = E_n + \\text{context}$, built from the pre-trained transformer's data embedding $E_n$ and an LSTM equation embedding $E_s$ through scaled-dot-product attention. The evaluator is a small MLP that regresses the quantity $r = (1-\\sum_i(\\tilde{y}_i - y_i)^2/\\sum_i(\\tilde{y}_i-\\bar{y})^2) + \\lambda\\exp(-l(\\tilde{f})/L)$, combining $R^2$ fitness with a simplicity penalty; its gradient with respect to $E_f$ supplies the search update $E_f^+ = E_f + \\eta\\,\\partial r/\\partial E_f$. During training, the reconstruction loss of the partially frozen decoder and the evaluator's mean-square error are jointly minimized, so the embedding space is shaped to make fitness locally smooth and predictable. At inference, the optimized embedding is decoded autoregressively into a prefix-notation equation. The mechanism's point is to supply a task-specific gradient direction, replacing token-level likelihood, which rewards equations that look like the pretraining data, with a continuous signal that rewards equations that fit this dataset.","core_discovery":"EQUATE's central claim is that discrete equation search can be reformulated as continuous optimization in a shared symbolic-numeric embedding space, and that this reformulation cures the negative-transfer problem of foundation models in low-data symbolic regression. The paper argues that fine-tuning a partially frozen transformer encoder-decoder alongside an LSTM equation encoder and an evaluator produces a space in which fitness is locally predictable, so that gradient steps on the evaluator's prediction with respect to the fused embedding move toward regions whose decoded equations fit the target data. The reported results support the claim that this beats the backbone's own decoding strategies as well as most genetic-programming and deep baselines, improves resilience to label noise, and does not inflate equation complexity or inference cost. The paper also claims that the best fine-tuning recipe keeps most pretrained weights frozen, updating only the equation encoder and the final decoder layer, and that both the symbolic alignment and the evaluator-guided search are necessary for the full gain.","pith_inferences":["The authors leave implicit that the same alignment-and-evaluator loop is a general template: any pre-trained sequence model that emits structured discrete objects, such as feature transformations or program sketches, could be adapted to a new dataset whenever a cheap numeric fitness exists.","Because 90 percent of training candidates are generated by the backbone itself, the search is confined by the backbone's candidate distribution; increasing the random share or adding genetic-programming candidates would test whether the evaluator can guide the search beyond that distribution.","The paper does not compare evaluator-guided gradient ascent with simply retrieving the best-known observed embedding by fitness; that comparison would separate the contribution of continuous search from the contribution of the aligned embedding space itself."],"forward_implications":["On Feynman, EQUATE with sampling raises the proportion of equations reaching test $R^2 > 0.99$ from 0.815 to 0.874 and average $R^2$ from 0.9730 to 0.9833 relative to the same decoding strategy of the backbone.","On Strogatz, the same variant raises $R^2 > 0.99$ from 0.357 to 0.786 and average $R^2$ from 0.8156 to 0.9818, showing that the method can deliver its largest gain exactly where the backbone transfers poorly.","The frozen fine-tuning recipe outperforms full fine-tuning, LoRA, and random initialization on all three benchmarks, supporting the paper's claim that preserving pretrained priors is central to the gain.","Ablations show that removing the equation encoder hurts the high-accuracy $R^2 > 0.99$ metric more than average $R^2$, while removing embedding-space optimization keeps average $R^2$ largely intact but loses high-accuracy cases; both components are needed for the best results."],"supporting_citations":[{"why":"Supplies the pre-trained transformer backbone whose encoder and decoder EQUATE reuses and fine-tunes.","marker":"[23]"},{"why":"Establishes the transformer pretraining paradigm for symbolic regression that the backbone extends.","marker":"[5]"},{"why":"Provides the Feynman benchmark equations used for evaluation.","marker":"[39]"},{"why":"Provides the Strogatz dynamical-system benchmark equations used for evaluation.","marker":"[25]"},{"why":"Defines the public benchmark suite and the baseline protocol used in the experiments.","marker":"[8]"},{"why":"Provides a transformer-with-MCTS baseline that the evaluator-guided search is compared against.","marker":"[36]"},{"why":"Provides a symbolic-numeric pretraining baseline that motivates the alignment design.","marker":"[30]"}],"fun_headline_variants":["Search for equations becomes continuous embedding optimization","EQUATE distills foundation models for low-data symbolic regression","Continuous embedding search beats discrete equation search","Small-data equation discovery via evaluator-guided embedding search"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes that the learned evaluator predicts true data–equation fitness accurately in exactly the region of embedding space that gradient ascent visits; if its predictions are miscalibrated there, the search can be steered toward embeddings whose decoded equations are worse than the backbone's own outputs.","fun_headline_variants_meta":{"raw":{"variants":["Search for equations becomes continuous embedding optimization","EQUATE distills foundation models for low-data symbolic regression","Continuous embedding search beats discrete equation search","Small-data equation discovery via evaluator-guided embedding search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000522,"raw_usage":{"total_tokens":2520,"prompt_tokens":934,"completion_tokens":1586,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":1528}},"tokens_in":550,"tokens_out":1586,"duration_ms":10924,"temperature":1.0,"reasoning_tokens":1528,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:52:49.574054+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log the evaluator's predicted fitness and the actual $R^2$ of the decoded equation at every gradient step on a fixed benchmark equation; systematic divergence between predicted and actual scores as the search moves away from its starting embedding would show the evaluator is miscalibrated. A complementary check is to replace the evaluator with the true $R^2$ computed on the same sampled subset and require EQUATE to match that oracle and beat a random walk in the same embedding space by roughly the claimed margin.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the transformer pretraining paradigm for symbolic regression that the backbone extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Strogatz dynamical-system benchmark equations used for evaluation."}],"review_version":2}