{"id":"ee244ff2-9b6d-4462-a090-d075fdaf770a","arxiv_id":"2505.04067","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A neural network reading the first 100 orbital harmonics of a light curve recognizes known heartbeat stars with about 86% agreement and yields four new candidates in the Kepler catalog.","lead":"Astronomers trained recurrent neural networks to recognize heartbeat stars from the amplitudes of the first 100 orbital harmonics in their light curves. The method automates a discovery process that has mostly relied on visual inspection, and it reports four new candidate heartbeat stars in the Kepler catalog.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 86% accuracy is eccentricity-regression consistency on pre-selected positives, not a detection statistic; the four new HBSs lack independent confirmation, so the practical detection claim is not yet demonstrated.","rationale":"The reader's weakest assumption concerned synthetic-to-real generalization, particularly missing TEOs. The paper's real-data tests partially address that concern, but the metric used there is weak: 86% 'prediction consistency' is a regression tolerance on preselected HBSs, not a detection statistic. My concern is therefore the next link in the chain: even if the RNN predicts eccentricity well, the paper does not demonstrate that this translates into reliable HBS recognition. The four claimed new HBSs are the only direct evidence of practical detection, and they are not independently confirmed. This supports the reader's CONDITIONAL verdict without changing it; the condition should be explicit: report proper detection statistics on a labeled mixed sample and obtain independent confirmation of the new candidates.","tokens_in":12278,"tokens_out":5370,"duration_ms":56275,"concrete_test":"Take the full Kirk et al. (2016) catalog with known labels (HBS, EB, other), apply the complete Sect. 3.2 pipeline (orbital frequency from Sect. 2.4, harmonic features, RNN ensemble, e>0.1, eclipse-count classification), and compute precision, recall, and ROC/AUC for HBS detection against those labels, including the five weak Kepler HBSs and without hand-excluding wrong-period systems. If precision/recall are not substantially above the low HBS prior in the catalog, the 'practical detection' claim is unsupported; additionally, check the four candidates against independent TESS photometry or radial-velocity data to confirm they are genuine heartbeat stars.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 reports '86% accuracy prediction consistency (with a deviation of less than 0.15)' on four real data sets. This is not an HBS detection rate; it is the fraction of already-known HBSs and eccentric binaries whose predicted eccentricity falls within ±0.15 of the literature value. The test set is filtered: five Kepler HBSs with weak heartbeat signals are excluded, and OGLE orbital frequencies are taken from the literature rather than derived. Section 3.2 then converts eccentricity predictions into discoveries via a multi-stage pipeline (e>0.1, visual period check, eclipse counting) with no reported precision, recall, or false-positive rate on a labeled sample. The four new HBSs emerge from this pipeline; only KIC 6794131 received a K95+ model fit, while the other three are classified by visual inspection of phase-folded light curves. If the RNN's eccentricity is accurate but the eclipse-count criterion misclassifies single-eclipse eccentric binaries as HBSs, the discovery claim fails even though the regression claim holds. This is the load-bearing weak point: the paper's central practical claim rests on an unquantified detection step and unconfirmed candidates.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a machine-learning method for recognizing heartbeat stars (HBSs). Light curves are first transformed into Fourier spectra, from which the amplitudes of the first 100 orbital harmonics are extracted and normalized into a 100-dimensional feature vector. Two recurrent neural networks (GRU and LSTM) are trained on 52,000 synthetic light curves generated with ELLC to regress orbital eccentricity. The trained networks are then applied to real HBSs from OGLE, Kepler, and TESS, and to a catalog of eclipsing binaries from Kirk et al. The authors report 95% accuracy on a synthetic test set, 86% 'accuracy prediction consistency' on real data, and the identification of four new HBS candidates, one of which is fitted with a K95+ model. The paper emphasizes that the harmonic-feature approach reduces computational cost and can be extended to other periodic variables.","tokens_in":12560,"tokens_out":5157,"duration_ms":53160,"significance":"If the claims hold, the work is a useful step toward automated HBS discovery: the features are physically motivated and compact, the training is entirely synthetic so there is no circularity in the labels, the code is public, and the method is tested on external real-data catalogs. The realistic significance is, however, lower than the abstract suggests, because the headline 86% figure is a regression-consistency fraction, not a detection accuracy, and the four new candidates are not independently confirmed. The central architecture is sound and reproducible, and the main gaps are quantifiable and fixable within the scope of the manuscript.","major_comments":[{"comment":"The headline '86% accuracy' is not an HBS detection accuracy: it is the fraction of known HBSs and eccentric binaries whose predicted eccentricity lies within ±0.15 of the literature value. In addition, five Kepler HBSs with weak heartbeat signals were excluded from the test set. The abstract and summary currently call this 'average detection accuracy', which overstates what is demonstrated. Please relabel the metric (e.g., regression consistency within 0.15), report per-dataset sizes and per-dataset fractions, and add actual binary classification metrics (precision/recall/F1 for the e>0.1 threshold) on the labeled test sets, ideally with a sensitivity analysis to the excluded weak-signal systems.","section":"Sect. 3.1, abstract, conclusions"},{"comment":"The discovery pipeline is not validated as a detector. The paragraph in Sect. 3.2 states that systems with no eclipses or one eclipse are classified as HBSs, but also that this strategy is 'not entirely rigorous'; it then converts roughly 900 candidates into four HBSs without reporting precision, recall, or false-positive rates on any labeled sample. A grazing-eclipsing eccentric binary can show a single eclipse and would be misclassified by this rule. Please run the full pipeline on a labeled subset of the Kirk catalog (known HBSs, known eccentric binaries, circular EBs) and report a confusion matrix. In addition, only KIC 6794131 receives a K95+ fit; the other three candidates need independent confirmation (e.g., model fits to all four, additional photometry or radial velocities) before the paper can claim they are 'newly identified HBSs'.","section":"Sect. 3.2"},{"comment":"The training set contains no tidally excited oscillations, yet Sect. 4 asserts robustness to TEOs based on the aggregate real-data performance. This generalization claim is plausible but unquantified: TEOs add harmonic power at specific frequencies and can change the normalized harmonic vector. Please report the consistency metric separately for known TEO-bearing and non-TEO HBSs in Fig. 4, and/or test the trained networks on synthetic light curves with injected TEO-like harmonic amplitudes.","section":"Sect. 2.1 vs Sect. 4"}],"minor_comments":[{"comment":"The first sentence contains a grammar issue: 'Since the variety of their light curve morphologies' should be 'Because of the variety of their light curve morphologies' (or similar).","section":"Abstract"},{"comment":"There is a duplicated word in 'the orbital orbital frequency'; it should be 'orbital frequency'.","section":"Sect. 2.4"},{"comment":"The feature vector is defined in Eq. (2) as (a1,...,an), but the network uses the first 100 harmonics; please clarify how vectors with fewer detected harmonics are handled and whether zero-padding or truncation is applied.","section":"Sect. 2.2 and 2.3"},{"comment":"The 0.02 deviation threshold used to define training accuracy and the 0.15 tolerance used for real-data consistency should be stated explicitly as the metric definitions; currently the term 'accuracy' is used in both places without a formal definition.","section":"Sect. 2.3 and 3.1"},{"comment":"The claim that 100 input units were chosen after 'extensive testing' would be more convincing with a small figure showing accuracy versus the number of harmonics used.","section":"Sect. 2.3"},{"comment":"The sentence 'we suggest classifying them as either type' for systems with both two eclipses and heartbeat signals is ambiguous; please specify the intended classification convention.","section":"Sect. 3.1"},{"comment":"The title contains an unusual space in 'V alidation'; this appears to be a formatting artifact and should be corrected.","section":"Title"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central mechanism is sound and within the scope of the journal, and the synthetic-only training design is a genuine strength that avoids label circularity. The main risk is that the discovery claim and the '86% accuracy' headline overreach the evidence; both are fixable with additional validation analyses. The harmonic feature idea is closely tied to the authors' prior work (Li et al. 2024d) and should be positioned as an incremental extension rather than a fully novel concept. The four new candidates, if confirmed, would be a modest but real contribution. I recommend requiring the detection-validation analysis before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read it. The actual contribution is the feature representation: normalized amplitudes of the first 100 orbital harmonics fed as a 100-dimensional vector into a two-layer GRU or LSTM that predicts eccentricity. That is new, far cheaper than CNN classifiers on phase-folded light curves, and the code plus trained networks are on GitHub. As an eccentricity regressor trained on synthetic ELLC light curves, it works decently: about 86% of known HBSs from OGLE, Kepler, and TESS, plus 45 published eccentric binaries, get predicted eccentricities within ±0.15 of the literature values.\n\nThat 86%, however, is not a detection accuracy, and the abstract and summary present it as one. It is the fraction of pre-selected known positives that land inside a fairly loose tolerance; the real-data tests contain no true negatives, so the number cannot carry the \"recognition\" framing. The actual detection step—Section 3.2 on the Kirk et al. catalog—is a multi-stage pipeline with no precision, recall, or false-positive rate reported. Roughly 1,200 systems pass the e>0.1 screen, a visual period check removes ~280, and an eclipse count separates HBSs (zero or one eclipse) from eccentric binaries (two). The authors admit this rule is \"not entirely rigorous.\" Of the four new HBSs, only KIC 6794131 received a K95+ model fit; the other three are visual classifications from phased light curves. Plausible candidates, but an unquantified, unconfirmed discovery claim.\n\nThe remaining soft spots are minor. The synthetic training set has no tidally excited oscillations; the normalization argument is plausible and the known TEO systems in the real test set do land inside tolerance, but that is indirect evidence, not a controlled test. Excluding the five weak-signal Kepler HBSs is honest but narrows the claim. There is no circularity—synthetic labels for training, external catalogs for testing—and the self-citation to Li et al. 2024d is legitimate groundwork.\n\nBottom line: a solid methods paper with an inflated evaluation. For anyone doing automated variable-star classification in TESS-era data, the harmonic compression idea is directly usable. It deserves a serious referee, not a desk reject; the referee should ask for the regression scatter labeled as regression, a detection test on a labeled sample with a false-positive rate, and confirmation for at least the four candidates. I would cite the feature method in my own work.","headline":"A genuinely cheap and novel harmonic-feature RNN for eccentricity regression, but the '86% accuracy' is a regression consistency score on known positives, not a detection rate, and the discovery claim rests on an unvalidated eclipse-count step.","tokens_in":13105,"tokens_out":5742,"would_cite":true,"duration_ms":54777,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Heartbeat stars can be found automatically by feeding the first 100 orbital harmonics of a light curve into a small recurrent network, which predicts eccentricity and works on real survey data at 86% accuracy.","keywords":["heartbeat stars","recurrent neural networks","orbital harmonics","light curve classification","eccentric binaries","tidally excited oscillations","synthetic light curves","Kepler eclipsing binaries"],"falsifier":"Run the two trained networks on all 173 Kepler heartbeat stars listed in the Kirk et al. catalog and compare every predicted eccentricity with the published value; if significantly fewer than 86% of predictions land within 0.15, the claimed synthetic-to-real generalization would fail.","tokens_in":12129,"feed_emoji":"💓","tokens_out":8525,"duration_ms":80659,"temperature":0.7,"pith_summary":"Heartbeat stars are binaries whose eccentric orbits produce a distinctive heartbeat-like pulse in their light curves near periastron, and most known examples were found by eye. This paper argues that they can be recognized automatically: take the Fourier spectrum of a light curve, keep the amplitudes of the first 100 orbital harmonics, normalize them, and feed the resulting vector into a small recurrent network that predicts orbital eccentricity. Trained on 52,000 synthetic light curves generated with the ELLC code, the GRU and LSTM networks reach 95% accuracy on synthetic tests and agree with published eccentricities within 0.15 for 86% of real heartbeat stars from OGLE, Kepler, and TESS. Applied to the Kepler eclipsing-binary catalog of Kirk et al., the method surfaced four new heartbeat stars, one of which has a model-fitted eccentricity matching the network prediction. If correct, this makes large-scale automated searches for heartbeat stars practical and could extend to other periodic variables.","feed_headline":"Recurrent net finds heartbeat stars at 86% accuracy","feed_subtitle":"Trained only on synthetic light curves, the networks also flag four new heartbeat stars in Kepler data.","key_machinery":"The load-bearing object is the normalized orbital-harmonic feature vector: after locating the orbital frequency, the first 100 harmonic amplitudes extracted by FNPEAKS are divided by their Euclidean norm, so each light curve becomes a 100-number sequence. This vector is what the recurrent network sees, and it matters because it compresses a roughly 30,000-point light curve into a small ordered set, discards the orbital period and absolute flux scale, and lets a two-layer GRU or LSTM rather than a deep network predict eccentricity. The orbital-frequency finder is a second piece of machinery, since the network cannot be applied to real data until the harmonic grid is known; it works by testing which integer multiple of the strongest peak has the most harmonic matches, or, for pulsating systems, by finding the most common spacing between adjacent Fourier peaks.","core_discovery":"The paper's central claim is that the morphology of a heartbeat-star light curve is encoded, for classification purposes, in the amplitudes of the first 100 orbital harmonics of its Fourier spectrum, and that a recurrent network reading only those amplitudes can recover the orbital eccentricity. The authors construct 52,000 synthetic light curves with ELLC by randomizing eccentricity, inclination, argument of periastron, mass ratio, surface brightness ratio, and radii, plus 2,000 zero-eccentricity close binaries to teach the network what is not a heartbeat star. They define a harmonic as a Fourier peak with signal-to-noise at least 4 that lies within 0.05 of an integer multiple of the orbital frequency, normalize the amplitude vector by its Euclidean norm, and train two-layer GRU and LSTM networks to output eccentricity. On held-out synthetic data the test accuracy is 95%; on real heartbeat stars from OGLE, Kepler, and TESS the networks agree with literature eccentricities to within 0.15 in 86% of cases; and within the Kirk et al. eclipsing-binary catalog the method finds four new heartbeat stars (KIC 4940438, 6794131, 7601633, 9243795), with KIC 6794131's model-fitted eccentricity of 0.179 close to the predicted 0.171.","pith_inferences":["The reported 86% is an agreement rate with published eccentricities on known heartbeat stars, not a completeness or purity measurement against a labeled background; a realistic survey deployment would need a false-positive estimate on non-heartbeat stars.","If the harmonic envelope is the main carrier of information, a natural test is whether a single interpretable statistic, such as the slope of harmonic amplitude versus harmonic number, correlates with eccentricity; if it does, the recurrent network could be distilled into a simpler explainable detector.","Injecting tidally excited oscillations into the synthetic training set is an obvious robustness upgrade; the paper shows the network tolerates their absence in training, which suggests including them could improve high-eccentricity cases where these oscillations are strong.","The method's period independence suggests the same trained networks could be applied to future surveys with different cadences, provided the Fourier spectrum resolves the orbital harmonics; that extension is untested and would be a useful external validation."],"forward_implications":["Survey archives can be pre-filtered automatically: reducing each light curve to 100 harmonic amplitudes lets a two-layer recurrent network rank large numbers of candidates by predicted eccentricity before any human inspection.","The same networks recognize eccentric binaries as well as heartbeat stars, and the two classes can be roughly separated by counting eclipses in the phase-folded light curve, so one pipeline can serve both searches.","Because the orbital frequency is computed automatically, the method does not need a known period to extract features, allowing application to any periodic variable whose orbital harmonics are resolved.","Visual inspection effort drops from hundreds of thousands of objects to a few hundred candidates, as demonstrated by finding about 1,200 high-eccentricity candidates and then four new heartbeat stars inside the Kirk et al. catalog."],"supporting_citations":[{"why":"Supplies the ELLC synthesis code used to generate all 52,000 training light curves with controlled orbital parameters and noise.","marker":"P. F. L. Maxted 2016"},{"why":"Defines the Kepler heartbeat-star class and provides the 2,878-system eclipsing-binary catalog in which the four new heartbeat stars are found.","marker":"B. Kirk et al. (2016)"},{"why":"Supplies the 991-object OGLE heartbeat-star catalog used as the largest real-data test set.","marker":"M. Wrona et al. (2022b)"},{"why":"Provides TESS heartbeat stars included in the real-data test set.","marker":"P. A. Ko laczek-Szymański et al. (2021)"},{"why":"Prior harmonic analysis of heartbeat stars with tidally excited oscillations that motivates using orbital harmonics as feature vectors.","marker":"M.-Y. Li et al. (2024d)"},{"why":"Provides the Kepler heartbeat-star sample used as a test set and the K95+ model-fitting method used to confirm KIC 6794131.","marker":"M.-Y. Li et al. (2023)"}],"fun_headline_variants":["Neural net spots 4 new heartbeat stars via orbital harmonics","Synthetic light curves train RNN to spot heartbeat stars","GRU/LSTM reads orbital harmonics, finds 4 heartbeat stars","Recurrent nets trained on synthetic data find real heartbeat stars","AI finds 4 new heartbeat stars using light curve harmonics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that 52,000 synthetic ELLC light curves, generated with Gaussian noise but without tidally excited oscillations, are representative enough of real survey light curves that a network trained only on them will recognize real heartbeat stars.","fun_headline_variants_meta":{"raw":{"variants":["Neural net spots 4 new heartbeat stars via orbital harmonics","Synthetic light curves train RNN to spot heartbeat stars","GRU/LSTM reads orbital harmonics, finds 4 heartbeat stars","Recurrent nets trained on synthetic data find real heartbeat stars","AI finds 4 new heartbeat stars using light curve harmonics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001286,"raw_usage":{"total_tokens":5332,"prompt_tokens":1102,"completion_tokens":4230,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":718,"completion_tokens_details":{"reasoning_tokens":4146}},"tokens_in":718,"tokens_out":4230,"duration_ms":29699,"temperature":1.0,"reasoning_tokens":4146,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:37:52.658045+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the two trained networks on all 173 Kepler heartbeat stars listed in the Kirk et al. catalog and compare every predicted eccentricity with the published value; if significantly fewer than 86% of predictions land within 0.15, the claimed synthetic-to-real generalization would fail.","supporting_citations":[],"review_version":1}