REVIEW 4 major objections 7 minor 56 references
Mining double-line spectroscopic candidates in the LAMOST medium-resolution spectroscopic survey using human-AI hybrid method
T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A hybrid cross-correlation plus deep-learning pipeline extracts 7,096 double-line (SB2) and 1,903 triple-line (SB3) spectroscopic binary candidates from LAMOST-MRS DR9, with 70.1% and 89.6% newly identified.
desk verdict A genuinely useful SB2/SB3 candidate catalog from LAMOST-MRS DR9, but the ML precision gains are computed without measuring real-data recall, so completeness is unknown. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The object that carries the argument is the cross-correlation function (CCF) between each observed blue-arm spectrum and one of three synthetic template spectra (hot dwarf, cool dwarf, cool giant), computed over radial velocities from -500 to +500 km/s. The CCF converts the spectrum into a smooth curve whose peaks mark stellar components; a derivative-based procedure following the method of Merle et al. (2017), using the third derivative and Gaussian smoothing, finds even heavily blended peaks. Four deep-neural-network classifiers (C1-C4), each trained on 6,000 samples built from synthetic ATLAS-model spectra plus observational CCFs categorized as L0-L3, are combined by majority voting with normalized-probability thresholds of 95% for double-line and 99% for triple-line spectra. This ensemble selects candidates for the final human visual inspection, and the CCF representation is what lets the synthetic training set be applied to real data.
What would settle it
Re-examine the 69 double-line and 8,780 triple-line CCFs that the ensemble selected but inspection rejected; if any of them are confirmed as real multi-line systems using independent data (higher S/N coadded spectra of the same targets or Gaia non-single-star astrometry), the claimed 99.7% SB2 precision and the underlying transfer assumption would be falsified.
Extended reading notes
Core claim
The central discovery claimed is that the combination of conventional CCF analysis, four DNN classifiers used in an ensemble, and final human-eye verification extracts 27,164 double-line and 3,124 triple-line spectra from 6,565,721 selected blue-arm spectra, corresponding to 7,096 SB2 and 1,903 SB3 candidates. The authors present these as roughly 1% of the selection dataset, with 70.1% of SB2 and 89.6% of SB3 candidates not listed in previous catalogs. Using the visually confirmed spectra as ground truth, the ML stage raises SB2 precision from 23.0% (CCF alone) to 99.7%, while SB3 precision rises only from 7.2% to 26.3%; the authors state that the triple-line training data do not fully reflect real L3 samples and that some true SB2s may still be filtered out.
Load-bearing premise
The classifiers are trained almost entirely on synthetic binary spectra whose radial-velocity separations are fixed between 60 and 250 km/s and whose flux ratios sit mostly between 1/3 and 3, and the paper assumes these CCFs transfer to real LAMOST spectra without systematically discarding true binaries — recall on real data is never measured.
Editorial extensions
If this is right
- The published catalog gives the community 7,096 SB2 and 1,903 SB3 candidates from one homogeneous pipeline, the largest such LAMOST-MRS sample to date.
- About 3,650 SB2 and 1,312 SB3 candidates have at least six exposures, enough to attempt orbital solutions and mass estimates.
- Because 70.1% of SB2 and 89.6% of SB3 candidates are absent from earlier catalogs, previous searches were substantially incomplete, not just smaller.
- Re-running the same CCF-plus-ensemble pipeline on later LAMOST releases should extend the sample with comparatively little new human effort.
- Triple-line systems remain the bottleneck: with 26.3% precision after ML, SB3 candidates still consume extra human review and will need better training data.
Reading between the lines
- The fixed training ranges (RV difference 60–250 km/s, flux ratio about 1/3 to 3) imply the catalog is incomplete for low-amplitude and extreme-ratio binaries; that incompleteness is an inference from the training setup, not a claim the paper makes.
- Taking the 99.7% SB2 precision at face value, only about 70 of the 27,233 ML-selected double-line spectra should be spurious, so the human inspection stage acts as a residual cleaner rather than the main filter.
- Cross-matching the output against known eclipsing or astrometric binaries in the same fields would measure recall and produce a completeness function, which the paper does not provide.
- The factor-of-four time saving compares candidate counts before and after ML; a full cost accounting would add the effort of generating synthetic training spectra and tuning thresholds.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents a hybrid pipeline for finding double-line and triple-line spectroscopic binary candidates in LAMOST-MRS DR9. The pipeline first uses a conventional cross-correlation function (CCF) technique on 6,565,721 blue-arm spectra with S/N ≥ 5, then applies an ensemble of four deep neural network classifiers to the CCFs, and finally performs visual inspection of the ML-selected candidates. The authors report 27,164 confirmed double-line spectra and 3,124 confirmed triple-line spectra, corresponding to 7,096 SB2 and 1,903 SB3 candidates, of which 70.1% and 89.6% are newly identified. They claim that the ML stage improves SB2 precision from 23.0% to 99.7% and reduces visual-inspection workload by a factor of four.
Significance. If the completeness and precision claims hold, this would be the largest homogeneous SB2 and SB3 candidate catalog from LAMOST-MRS to date, providing a useful sample for binary population studies and follow-up radial-velocity monitoring. The paper's strengths include the final visual inspection of the selected spectra, cross-matching against ten external binary and stellar catalogs, Monte Carlo radial-velocity uncertainties, and a clearly described synthetic training design. The central quantitative claims, however, rest on an unmeasured real-data recall, so the significance is conditional on an additional validation step.
major comments (4)
- [Section 5] The headline precision gain from 23.0% to 99.7% is a conditional precision on the ML-selected subset, not a global precision. The 23.0% figure is 27,164/118,274 for all CCF-positive spectra, while the 99.7% figure is 27,164/27,233 for the ML-selected subset; this comparison implicitly assumes that all 91,041 CCF-positive spectra rejected by the ensemble are false positives. Real-data recall on the rejected set is never measured, so the catalog completeness and the 'factor of four' savings in visual inspection are not established. Please quantify the false-negative rate, for example by visually inspecting a random sample of the rejected spectra or by testing the ensemble on known SB2 systems not used in the cross-match.
- [Section 3.2.1 and Section 5] The ML classifiers are trained and 10-fold cross-validated on simulated SB2 and SB3 CCFs with radial-velocity differences restricted to 60–250 km/s and flux ratios mostly between 1/3 and 3, and the reported >99% precision, recall, and F1 scores in Section 3.2.2 measure performance on that same simulation distribution. Real LAMOST-MRS CCFs include lower S/N, line blending, asymmetric peaks, and flux ratios outside the simulated range. The paper itself concedes in Section 5 that 'true SB2 candidates may still be included in the spectra that are filtered out,' but it does not estimate how many. A transfer-validation experiment on real spectra with known multiplicity labels is needed before the efficiency and precision claims can be taken as representative of real survey performance.
- [Section 4 and Section 5] For SB3 candidates, the paper reports that only 3,124 of 11,904 ML-selected spectra (26.3%) pass visual inspection, and it attributes the losses to the training data not fully reflecting the real L3 distribution. Given this acknowledged mismatch, the reported SB3 candidate count of 1,903 should be presented as a lower limit with a quantitative completeness estimate, or the abstract and conclusion should explicitly state that the SB3 sample is heavily incomplete. Without such a caveat, the '89.6% newly identified' statistic for SB3 candidates could be misleading because it refers only to the subset that survives the ML and visual filters.
- [Section 4] Visual inspection is the de facto ground truth for the final catalog, but the paper does not report how many inspectors were involved, whether there was independent double-checking, or any inter-inspector agreement statistic. The criterion 'the double-line or triple-line signal in the peak area must be significantly stronger than that in the wing part' is qualitative, which makes the ground-truth labels non-auditable. Please provide a quantitative rejection criterion or an inter-rater agreement metric, at least for a randomly chosen subsample, so that readers can assess the reliability of the final catalog.
minor comments (7)
- [Section 3.1.1] In the sentence 'We generate three spectral template using the stellar spectral synthesis program SPECTRUM,' the word 'template' should be plural, and the sentence should be rephrased for clarity.
- [Figure 4 caption] The caption says 'the R V1, R V2 and R V1 in SB3 classification,' but the third quantity should be R V3, not a duplicate R V1.
- [Table 2] The column heading 'R V calculation classification' is ambiguous; the rows labeled C1, C2, C3, and C4 should be described more clearly in the table caption or in Section 3.2.2.
- [Section 4.1] In the sentence 'Taking into account of all the cross match results, 2121 SB2 and 197 candidates identified in this work have been included in other catalogs or studies,' the number 197 should be labeled as SB3 candidates to avoid ambiguity.
- [Section 4] The sentence 'The radius is determined from the the diameters of the fiber of LAMOST' contains a duplicated 'the'.
- [Abstract and Section 5] The phrase 'about 1% of the selection dataset' is ambiguous because the paper refers to both 6,565,721 spectra and 930,783 stars; specifying 'about 1% of the selected stars' would make the statistic unambiguous.
- [General] The paper does not state where the machine-readable catalog and the code for the CCF and ML pipeline will be made available; for a catalog paper of this type, a data-availability statement is important for reproducibility.
Circularity Check
No significant circularity: the ML screening is validated against human visual inspection, and the catalog is cross-checked against external surveys.
full rationale
The paper's derivation chain is: (1) CCF peak detection selects 118,274 double-line and 43,519 triple-line spectra; (2) ensemble DNN classifiers trained on synthetic binary/triple spectra (Section 3.2.1) reduce these to 27,233 and 11,904; (3) human visual inspection confirms 27,164 and 3,124. The claimed precision gain (23.0% to 99.7% for SB2) is computed with the same visually confirmed numerator over the pre- and post-ML denominators, which is a standard precision comparison rather than a prediction forced by fitted inputs. The ML classifiers were not trained on the visual labels, so the confirmation step is independent of the training loop. The acknowledged limitation that recall on the 91,041 rejected CCF-positive spectra is unmeasured, and the paper's own statement that 'true SB2 candidates may still be included in the spectra that are filtered out,' are completeness risks, not circularity. Self-citations to Li et al. (2021) supply an empirical RV-difference upper bound (250 km/s) and an MC uncertainty recipe; these constrain the training domain but do not by themselves produce the final catalog, and 2,121 candidates are matched to external catalogs (KEBC, TESS-EBs, APOGEE, Gaia NSS, GALAH, SB9, etc.), providing independent anchoring. No self-definitional, fitted-input-as-prediction, or uniqueness-imported-by-authors step appears.
Assumptions & free parameters
free parameters (5)
- Gaussian smoothing sigma for CCF derivatives =
initial 13 km/s, increment 1 km/s up to 100 km/s
- CCF peak selection thresholds =
CCF > 60%, second derivative < 40%
- ML probability thresholds for final selection =
P > 95% for L2, P > 99% for L3
- RV difference range for synthetic training binaries =
60 to 250 km/s
- S/N selection threshold =
S/N > 5
assumptions (4)
- domain assumption ATLAS stellar atmosphere models with SPECTRUM provide realistic synthetic spectra for LAMOST-MRS wavelengths and resolution.
- domain assumption CCF derivative peak detection following Merle et al. (2017) reliably finds RV components for binaries with delta-RV above roughly 60 km/s.
- domain assumption The human visual criterion on CCF peak shape is a valid ground truth for SB2/SB3 classification.
- ad hoc to paper The synthetic training distribution, with RV differences 60-250 km/s and flux ratios roughly 1/3 to 3, transfers to real LAMOST-MRS CCFs without systematic loss of true binaries.
Cite this review
Pith. "Pith review of Mining double-line spectroscopic candidates in the LAMOST medium-resolution spectroscopic survey using human-AI hybrid method." pith.science (2026). https://pith.science/paper/R27G3XLL
@misc{pith2026241114714,
author = {Pith},
title = {Pith review of: Mining double-line spectroscopic candidates in the LAMOST medium-resolution spectroscopic survey using human-AI hybrid method},
year = {2026},
howpublished = {\url{https://pith.science/paper/R27G3XLL}},
note = {Machine review of arXiv:2411.14714}
}
read the original abstract
We utilize a hybrid approach that integrates the traditional cross-correlation function (CCF) and machine learning to detect spectroscopic multi-systems, specifically focusing on double-line spectroscopic binary (SB2). Based on the ninth data release (DR9) of the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST), which includes a medium-resolution survey (MRS) containing 29,920,588 spectra, we identify 27,164 double-line and 3124 triple-line spectra, corresponding to 7096 SB2 candidates and 1903 triple-line spectroscopic binary (SB3) candidates, respectively, representing about 1% of the selection dataset from LAMOST-MRS DR9. Notably, 70.1% of the SB2 candidates and 89.6% of the SB3 candidates are newly identified. Compared to using only the traditional CCF technique, our method significantly improves the efficiency of detecting SB2, saves time on visual inspections by a factor of four.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
/L Ãhx z 3 ^So w ajs]2 h:J\#ƝZ UN3\6 xU @Լ Z & emFPĊ Ic YUdBG 4 ?j<G6u# e#Z
thebibliography [1] 20pt to REFERENCES 6pt =0pt 10pt plus 3pt =0pt =0pt =1pt plus 1pt =0pt =0pt -12pt =13pt plus 1pt =20pt =13pt plus 1pt \@M =10000 =-1.0em =0pt =0pt 0pt =0pt =1.0em @enumiv\@empty 10000 10000 `\.\@m \@noitemerr \@latex@warning Empty `thebibliography' environment \@ifnextchar \@reference \@latexerr Missing key on reference command Each re...
2019
-
[2]
2016, arXiv e-prints, arXiv:1603.04467, 10.48550/arXiv.1603.04467
Abadi , M., Agarwal , A., Barham , P., et al. 2016, arXiv e-prints, arXiv:1603.04467, 10.48550/arXiv.1603.04467
-
[3]
Allende Prieto , C., Majewski , S. R., Schiavon , R., et al. 2008, Astronomische Nachrichten, 329, 1018, 10.1002/asna.200811080
-
[4]
Birko , D., Zwitter , T., Grebel , E. K., et al. 2019, , 158, 155, 10.3847/1538-3881/ab3cc1
-
[5]
2005, Information Fusion, 6, 5, https://doi.org/10.1016/j.inffus.2004.04.004
Brown, G., Wyatt, J., Harris, R., & Yao, X. 2005, Information Fusion, 6, 5, https://doi.org/10.1016/j.inffus.2004.04.004
-
[6]
Castelli , F., & Kurucz , R. L. 2003, in Modelling of Stellar Atmospheres, ed. N. Piskunov , W. W. Weiss , & D. F. Gray , Vol. 210, A20. astro-ph/0405087
arXiv 2003
-
[7]
2012, Research in Astronomy and Astrophysics, 12, 1197, 10.1088/1674-4527/12/9/003
Cui , X.-Q., Zhao , Y.-H., Chu , Y.-Q., et al. 2012, Research in Astronomy and Astrophysics, 12, 1197, 10.1088/1674-4527/12/9/003
-
[8]
De Silva , G. M., Freeman , K. C., Bland-Hawthorn , J., et al. 2015, , 449, 2604, 10.1093/mnras/stv327
Show all 56 references
-
[9]
Dietterich, T. G. 2000, in Multiple Classifier Systems (Berlin, Heidelberg: Springer Berlin Heidelberg), 1--15
2000
-
[10]
G., Mahabal , A
Djorgovski , S. G., Mahabal , A. A., Graham , M. J., Polsterer , K., & Krone-Martins , A. 2022, arXiv e-prints, arXiv:2212.01493. 2212.01493
2022 arXiv
-
[11]
2013, , 51, 269, 10.1146/annurev-astro-081710-102602
Duch \^e ne , G., & Kraus , A. 2013, , 51, 269, 10.1146/annurev-astro-081710-102602
2013 doi
-
[12]
2021, Wide binaries from Gaia eDR3, Zenodo, 10.5281/zenodo.4435257
El-Badry, K. 2021, Wide binaries from Gaia eDR3, Zenodo, 10.5281/zenodo.4435257
2021 doi
-
[13]
2018, , 476, 528, 10.1093/mnras/sty240
El-Badry , K., Ting , Y.-S., Rix , H.-W., et al. 2018, , 476, 528, 10.1093/mnras/sty240
2018 doi
-
[14]
A., Covey , K
Fernandez , M. A., Covey , K. R., De Lee , N., et al. 2017, , 129, 084201, 10.1088/1538-3873/aa77e0
2017 doi
-
[15]
2022, VizieR Online Data Catalog: Gaia DR3 Part 3
Gaia Collaboration . 2022, VizieR Online Data Catalog: Gaia DR3 Part 3. Non-single stars (Gaia Collaboration, 2022) , VizieR On-line Data Catalog: I/357. Originally published in: Astron. Astrophys., in prep. (2022). https://ui.adsabs.harvard.edu/abs/2022yCat.1357....0G
2022
-
[16]
Gaia Collaboration , Vallenari , A., Brown , A. G. A., et al. 2023, , 674, A1, 10.1051/0004-6361/202243940
2023 doi
-
[17]
C., et al
Gilmore , G., Randich , S., Worley , C. C., et al. 2022, , 666, A120, 10.1051/0004-6361/202243134
2022 doi
- [18]
-
[19]
Gubner, J. A. 2006, Probability and random processes for electrical and computer engineers (Cambridge University Press)
2006
-
[20]
2020, Research in Astronomy and Astrophysics, 20, 161, 10.1088/1674-4527/20/10/161
Han, Z.-W., Ge, H.-W., Chen, X.-F., & Chen, H.-L. 2020, Research in Astronomy and Astrophysics, 20, 161, 10.1088/1674-4527/20/10/161
2020 doi
-
[21]
He, H., & Garcia, E. A. 2009, IEEE Transactions on Knowledge and Data Engineering, 21, 1263, 10.1109/TKDE.2008.239
2009 doi
- [22]
-
[23]
2016, , 151, 68, 10.3847/0004-6256/151/3/68
Kirk , B., Conroy , K., Pr s a , A., et al. 2016, , 151, 68, 10.3847/0004-6256/151/3/68
2016 doi
-
[24]
R., Stassun , K
Kounkel , M., Covey , K. R., Stassun , K. G., et al. 2021, , 162, 184, 10.3847/1538-3881/ac1798
2021 doi
-
[25]
2022, , 517, 356, 10.1093/mnras/stac2513
Kovalev , M., Chen , X., & Han , Z. 2022, , 517, 356, 10.1093/mnras/stac2513
2022 doi
-
[26]
2021, , 256, 31, 10.3847/1538-4365/ac22a8
Li , C.-q., Shi , J.-r., Yan , H.-l., et al. 2021, , 256, 31, 10.3847/1538-4365/ac22a8
2021 doi
-
[27]
2020, arXiv e-prints, arXiv:2005.07210
Liu , C., Fu , J., Shi , J., et al. 2020, arXiv e-prints, arXiv:2005.07210. 2005.07210
2020 arXiv
-
[28]
L., Zhao , Y.-H., Zhao , G., et al
Luo , A. L., Zhao , Y.-H., Zhao , G., et al. 2015, Research in Astronomy and Astrophysics, 15, 1095, 10.1088/1674-4527/15/8/002
2015 doi
-
[29]
D., Wycoff , G
Mason , B. D., Wycoff , G. L., Hartkopf , W. I., Douglass , G. G., & Worley , C. E. 2001, , 122, 3466, 10.1086/323920
2001 doi
-
[30]
2010, , 140, 184, 10.1088/0004-6256/140/1/184
Matijevi c , G., Zwitter , T., Munari , U., et al. 2010, , 140, 184, 10.1088/0004-6256/140/1/184
2010 doi
-
[31]
2017, , 608, A95, 10.1051/0004-6361/201730442
Merle , T., Van Eck , S., Jorissen , A., et al. 2017, , 608, A95, 10.1051/0004-6361/201730442
2017 doi
-
[32]
2020, , 635, A155, 10.1051/0004-6361/201935819
Merle , T., Van der Swaelmen , M., Van Eck , S., et al. 2020, , 635, A155, 10.1051/0004-6361/201935819
2020 doi
- [33]
-
[34]
Nair, V., & Hinton, G. E. 2010, in International Conference on Machine Learning
2010
-
[35]
Pedregosa, F., Varoquaux, G., Gramfort, A., et al. 2011, J. Mach. Learn. Res., 12, 2825–2830
2011
-
[36]
A., Batten , A
Pourbaix , D., Tokovinin , A. A., Batten , A. H., et al. 2004, , 424, 727, 10.1051/0004-6361:20041213
2004 doi
-
[37]
M., Hogg , D
Price-Whelan , A. M., Hogg , D. W., Rix , H.-W., et al. 2018, , 156, 18, 10.3847/1538-3881/aac387
2018 doi
- [38]
-
[39]
E., et al
Pr s a , A., Kochoska , A., Conroy , K. E., et al. 2022, , 258, 16, 10.3847/1538-4365/ac324a
2022 doi
-
[40]
2019, Research in Astronomy and Astrophysics, 19, 064, 10.1088/1674-4527/19/5/64
Qian , S.-B., Shi , X.-D., Zhu , L.-Y., et al. 2019, Research in Astronomy and Astrophysics, 19, 064, 10.1088/1674-4527/19/5/64
2019 doi
-
[41]
A., Henry , T
Raghavan , D., McAlister , H. A., Henry , T. J., et al. 2010, , 190, 1, 10.1088/0067-0049/190/1/1
2010 doi
-
[42]
2022, , 666, A121, 10.1051/0004-6361/202243141
Randich , S., Gilmore , G., Magrini , L., et al. 2022, , 666, A121, 10.1051/0004-6361/202243141
2022 doi
-
[43]
2011, , 416, 817, 10.1111/j.1365-2966.2011.18698.x
Sana , H., James , G., & Gosset , E. 2011, , 416, 817, 10.1111/j.1365-2966.2011.18698.x
2011
-
[44]
J., & Geach , J
Smith , M. J., & Geach , J. E. 2023, Royal Society Open Science, 10, 221454, 10.1098/rsos.221454
2023 doi
- [45]
-
[46]
2006, , 132, 1645, 10.1086/506564
Steinmetz , M., Zwitter , T., Siebert , A., et al. 2006, , 132, 1645, 10.1086/506564
2006 doi
-
[47]
2020, , 249, 22, 10.3847/1538-4365/ab9904
Tian , Z., Liu , X., Yuan , H., et al. 2020, , 249, 22, 10.3847/1538-4365/ab9904
2020 doi
-
[48]
2020, , 638, A145, 10.1051/0004-6361/202037484
Traven , G., Feltzing , S., Merle , T., et al. 2020, , 638, A145, 10.1051/0004-6361/202037484
2020 doi
- [49]
-
[50]
E., et al
Virtanen, P., Gommers, R., Oliphant, T. E., et al. 2020, Nature Methods, 17, 261, 10.1038/s41592-019-0686-2
2020 doi
-
[51]
2020, , 643, A122, 10.1051/0004-6361/201936090
S koda , P., Podsztavek , O., & Tvrd \' k , P. 2020, , 643, A122, 10.1051/0004-6361/201936090
2020 doi
-
[52]
S., Liu , X
Xiang , M. S., Liu , X. W., Yuan , H. B., et al. 2015, , 448, 822, 10.1093/mnras/stu2692
2015 doi
-
[53]
G., Adelman , J., Anderson , John E., J., et al
York , D. G., Adelman , J., Anderson , John E., J., et al. 2000, , 120, 1579, 10.1086/301513
2000 doi
-
[54]
2022, , 258, 26, 10.3847/1538-4365/ac42d1
Zhang , B., Jing , Y.-J., Yang , F., et al. 2022, , 258, 26, 10.3847/1538-4365/ac42d1
2022 doi
-
[55]
2015, Data Science Journal, 10.5334/dsj-2015-011
Zhang, Y., & Zhao, Y. 2015, Data Science Journal, 10.5334/dsj-2015-011
2015 doi
-
[56]
Zverko , J., Z i z n ovsk \'y , J., Mikul \'a s ek , Z., & Iliev , I. K. 2007, Contributions of the Astronomical Observatory Skalnate Pleso, 37, 49
2007
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.