{"id":"3f798d55-513e-4318-a879-5bef86517f13","arxiv_id":"2506.11637","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Fusing embedding representations from multiple photometric bands improves transformer-based variable star classification by roughly 10% in F1, with only minor sensitivity to how asynchronously the bands are sampled.","lead":"This paper extends the Astromer transformer model so it can use light curves from multiple telescope filters at once, not just one. The multiband versions improve variable-star classification F1 by about 10% over single-band versions, at the cost of much longer pre-training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FMA results for the best fusion variants (TF, CXA, LSTM) come from runs that exceeded the 72-hour cluster limit in every or most runs; the paper never clarifies whether these are truncated checkpoints, so the FMA-vs-SMA comparison may be a comparison of stopping times rather than architectures.","rationale":"The reader's weakest assumption correctly identifies the 72-hour limit as the most load-bearing concern. The paper's headline claims rest on FMA results, and Table 5 shows that the FMA variants highlighted as best (TF, CXA, LSTM) systematically exceeded the stated cluster limit. If those runs were killed at 72h and their final checkpoints used for evaluation, then the FMA numbers correspond to an uncontrolled stopping time. This would not only make the FMA-vs-SMA comparison ambiguous but also call into question whether the abstract's 'jointly pre-trained multiband encoders further improving performance' is a genuine architectural benefit or an artifact of how much training each model received. The concern is addressable: a rerun to convergence or training curves would resolve it. I do not think it warrants rejection outright, because the multiband-vs-single-band comparison on Alcock is also supported by SMA, which is unaffected by the FMA pre-training issue; and the FMA results, even if early-stopped, may still show an effect. But the paper's current presentation is materially incomplete. I agree with the reader's conditional verdict and the identified weakest assumption. Other issues (missing ATLAS single-band baselines, no error bars, no code/data) are real but secondary; the 72h issue is the one that most directly threatens the central claim.","tokens_in":19055,"tokens_out":8750,"duration_ms":77627,"concrete_test":"Reproduce the FMA-TF and FMA-LSTM pre-training runs (same 600k MACHO subset, same hyperparameters) without the 72h limit, continuing until early stopping on validation RMSE triggers; then fine-tune on Alcock (synchronous and pseudo-asynchronous) and ATLAS at 500 samples per class and on the full dataset, and compare F1/RMSE to Tables 9–11. If the converged numbers match within ~0.02 F1, the 72h entries are harmless; if they shift materially, the published FMA results are stopping-time artifacts and the FMA-vs-SMA conclusion is unsupported. Alternatively, supply the training-loss curves showing that the models had converged before the 72h cutoff, which would also settle the issue.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 5 reports pre-training wall-clock times. For the FMA variants that the paper identifies as best — CXA, LSTM, and TF mixing — a majority of the five runs are listed as '>72' (TF and CXA in all five runs, LSTM in three). The text in Sec. 5.1 states these models 'often exceeded the cluster's 72-hour limit' but never states whether the results in Figs. 6–8 and Tables 6–11 come from the 72-hour checkpoint, from resumed runs, or from fully converged models. If the runs were terminated at 72h and the last checkpoint used for evaluation, then the FMA models are early-stopped at an arbitrary wall-clock time. The SMA and single-band models mostly completed within 20–65h, so the comparison between FMA and the baselines is a comparison of training effort, not solely architecture. The abstract's claims — both the ~10% F1 gain over single-band and the FMA-over-SMA improvement — are anchored in these FMA numbers. This is a load-bearing omission. Additionally, the paper does not report single-band baselines for the ATLAS classification tables (Tables 8, 10, 11), so the advertised 10% gain is only directly evidenced on Alcock; but the truncated-run issue is more fundamental because it affects the internal FMA-vs-SMA comparison on both datasets.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the single-band transformer-based foundational model Astromer to multiband light curves by adding fusion mechanisms that combine per-band encoder embeddings. Two architectural paradigms are compared: a Simple Multiband Architecture (SMA), in which independently pretrained single-band encoders are fused only after pretraining, and a Full Multiband Architecture (FMA), in which encoders and fusion are jointly pretrained. Six mixing strategies (AVG, LF, CXA, MHA, LSTM, TF) are evaluated. The models are pretrained on 600k MACHO light curves, fine-tuned on the Alcock and ATLAS catalogs, and evaluated with masked-reconstruction RMSE and downstream classification F1-score. The central claims are that multiband models outperform single-band models by roughly 10% in F1-score, that jointly pretrained FMA models further improve over SMA models, and that synchronous versus asynchronous band sampling matters little for multiband performance.","tokens_in":19388,"tokens_out":5800,"duration_ms":52002,"significance":"If the empirical claims hold, the paper delivers a useful, reasonably efficient recipe for extending a single-band transformer foundation model to multiband light curves, with a systematic ablation of six fusion strategies across two pretraining paradigms and several data budget sizes. The comparison of mixing-layer parameter counts, wall-clock pretraining times, and classification performance at 20-500 labeled samples per class is a practical contribution for the transient/variable-star community. The paper also documents the compute trade-off between joint and independent pretraining, which is relevant for deployment. The main potential value is as a design template for future multiband foundation models. However, the headline results rest on two pieces of evidence that are not fully in place: the FMA runs that produced the best results are reported only through entries marked '>72' hours in the pretraining table, and the ATLAS classification tables lack single-band baselines.","major_comments":[{"comment":"The paper reports FMA pretraining times of '>72' hours for all five runs of the TF and CXA variants and for four of five LSTM runs, yet Tables 6-11 report final test results for those exact models. The text in Sec. 5.1 says these models 'often exceeded the cluster's 72-hour limit' but never states whether the reported numbers come from the 72-hour checkpoint, from resumed runs, or from fully converged models. If the runs were terminated at 72h, the FMA-versus-SMA comparison is a comparison of stopping times rather than architectures, since the single-band and most SMA models completed within the limit; the abstract's claim that joint pretraining improves over independent encoders would then be unsupported. Please state explicitly, for every result in Tables 6-11, which checkpoints were used and whether any runs were resumed, and if truncated, provide budget-matched comparisons.","section":"Sec. 5.1, Table 5"},{"comment":"The advertised 'approximately 10% improvement in F1-score' over single-band models is directly evidenced only in the Alcock synchronous table (Table 9), which contains single-band baseline rows. Table 10 (Alcock pseudo-asynchronous F1) and Table 11 (ATLAS asynchronous F1) report only SMA and FMA columns, with no single-band counterparts. Since the abstract makes a cross-dataset claim, the ATLAS column in particular needs either single-band baseline rows or a qualification that the gain is demonstrated on Alcock only.","section":"Tables 10 and 11"},{"comment":"The claim that jointly pretrained FMA encoders 'further improve performance' over independently pretrained SMA encoders is not consistently supported by the reported means. In Table 9 under 'All', SMA LSTM reaches 0.74 versus FMA LSTM 0.72, and SMA Transformer 0.73 versus FMA Transformer 0.71; similar reversals appear in Tables 10-11 at several sample sizes. The tables report only averages over five runs, without standard deviations or significance tests, so the small differences on which the FMA-over-SMA claim rests may reflect run-to-run noise. Please either add uncertainty estimates and significance testing, or soften the abstract and conclusions to describe the FMA-SMA difference as favorable only for specific fusion strategies or sample sizes.","section":"Sec. 5.3, Tables 9-11"}],"minor_comments":[{"comment":"The sentence 'the where embeddings, where embeddings z_i^{(k)} from N bands are aggregated' contains a duplicated phrase; it should read 'where the embeddings z_i^{(k)} from N bands are aggregated'.","section":"Sec. 3.2.1"},{"comment":"The mixing strategy is typeset as 'A VG' in the text but as 'AVG' in the tables; please unify the notation.","section":"Throughout"},{"comment":"The term 'fine-tuning' is used for a second stage of masked reconstruction in which labels are not used, and classification heads are trained afterwards in Sec. 5.3; this two-stage procedure should be clarified at the start of Sec. 5.2 to avoid confusing the reader.","section":"Sec. 5.2"},{"comment":"The inclusion of the UNK class in the Alcock catalog differs from the original Astromer setup; the paper should state how this affects comparability with published Astromer classification results.","section":"Sec. 4.2.1"},{"comment":"The figure caption says whiskers extend to points within 1.5 times the IQR, and the text invokes 'variance across runs'; reporting the numerical standard deviations in the caption or in Table 5 would support the stability claims made in Sec. 5.1.","section":"Fig. 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a natural extension of the authors' own Astromer model; two of the three authors are also authors of Donoso-Oliva et al. (2023), and the manuscript cites that work prominently. That lineage is not a problem, but it makes the documentation of the baseline and training-budget controls especially important. The main technical risk is the undocumented status of the FMA runs that produced the best results: if those numbers come from truncated 72-hour checkpoints, the central comparative claim is invalid. I would ask the authors to clarify this before any further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you care about transformer foundations for light curves. The core result is legitimate and mostly supported: fusing per-band Astromer embeddings and jointly pretraining the stack improves variable-star classification over single-band encoders. The Alcock tables directly show single-band F1 of 0.62–0.64 versus multiband 0.71–0.74, which is the roughly 10% claim. The comparison of six mixing strategies and of SMA versus FMA is a genuine, useful extension, and so is the synchronous-versus-asynchronous sampling test, which suggests strict temporal alignment is not essential. The authors are also honest about pretraining cost.\n\nSoft spots are real but fixable. The FMA variants that work best (TF, CXA, LSTM) mostly exceeded the cluster's 72-hour limit in Table 5, and the paper never says whether the reported numbers come from a 72-hour checkpoint, a resumed run, or a fully converged model. If those are truncated checkpoints, the FMA-versus-SMA comparison is partly a comparison of training effort. That is the main flaw. Second, the ATLAS classification tables have no single-band baseline, so the headline 10% improvement is only directly supported on Alcock. Third, there is no code, no data, and no per-run error bars, despite the text saying metrics are averaged over five runs. The absence of code is manageable, but the 72-hour ambiguity needs a sentence or a rerun. I would also like the fine-tuning setup stated more prominently: the authors fine-tune via unsupervised masked reconstruction on labeled catalogs, using labels only for stratified sampling. That is unusual and easy to miss.\n\nThe citation pattern is fine. The paper builds on Astromer and cites the relevant transformer and light-curve literature without overselling; it frames itself as an architectural and empirical contribution rather than a new physical result. The central argument holds up well enough to warrant referee time, but the checkpointing issue is load-bearing for the FMA-versus-SMA claim and should be addressed before publication. I would send it to peer review, ask for clarification on the 72-hour runs and ATLAS baselines, and encourage code release. This is a useful paper for anyone building LSST-era multiband classifiers, and a serious referee can make it stronger.","headline":"Solid empirical extension of Astromer to multiband light curves, with a credible 10% F1 gain on Alcock; but the best FMA runs may be truncated 72-hour checkpoints, and the ATLAS claim lacks a single-band baseline.","tokens_in":19899,"tokens_out":1872,"would_cite":true,"duration_ms":19085,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fusing per-band transformer embeddings into a single sequence improves variable-star classification by about ten F1 points over single-band models.","keywords":["Representation Learning","Multivariate Light Curves","Foundational Models","Astromer","multiband fusion","variable star classification","asynchronous time series","masked reconstruction"],"falsifier":"Extend the pre-training of the top FMA models past the 72-hour cutoff with a longer cluster budget and compare their F1-scores; if the roughly 10% advantage over single-band models shrinks or vanishes once training actually converges, the central claim would be refuted. A cheaper check is to plot validation RMSE versus wall-clock for the '>72' entries in Table 5; if the loss is still falling steeply at the cutoff, the reported numbers are stopping-time artifacts.","tokens_in":18866,"feed_emoji":"🔭","tokens_out":6977,"duration_ms":63174,"temperature":0.7,"pith_summary":"Astronomical surveys now record the same star in several photometric bands, but existing transformer-based light-curve models treat each band as an independent single-band sequence and ignore cross-band information. This paper tries to establish that a fusion layer placed between per-band encoders and the decoder, mapping the set of single-band embeddings into one unified sequence, recovers that information and improves downstream classification. Using the single-band Astromer encoder, a transformer pre-trained to reconstruct masked magnitudes, as the building block, the authors compare a simple multiband architecture that late-fuses independently pre-trained encoders with a full multiband architecture whose encoders are jointly pre-trained through the fusion layer. They report that both multiband designs outperform single-band models by roughly 10% in F1-score, that the jointly pre-trained version improves further, and that strict alignment of observation times across bands is not required to obtain the gain. The payoff would be a practical recipe for turning pre-trained single-band embeddings into multiband representations for surveys like ATLAS and the upcoming LSST.","feed_headline":"Fusing light-curve bands lifts classification F1 by 10%","feed_subtitle":"Multiband transformer embeddings beat single-band encoders even when each filter observes at different times.","key_machinery":"The load-bearing object is the mixing function $f$ that combines per-band encoder outputs $z_i=\\Phi_i(Z_i)\\in\\mathbb{R}^{L\\times d}$ into one sequence-level embedding before the decoder reconstructs masked observations, with the reconstruction loss averaged across bands and backpropagated through $f$ into every encoder. The paper's LFEM (late fusion embedding mixing) design pattern is what distinguishes multiband models: SMA applies $f$ only after independently pre-trained encoders are frozen, while FMA trains encoders and $f$ jointly so the embedding spaces become coordinated. The same $f$ is reused during fine-tuning, where the masked-reconstruction decoder is replaced by a classification head.","core_discovery":"The paper's central claim is that Multiband Astromer, a stack of band-specific Astromer encoders feeding a mixing function $f$ that produces a unified embedding $y=f(z_0,\\dots,z_{N-1})$, uses cross-band information to classify variable stars better than any single-band Astromer baseline. On the Alcock and ATLAS catalogs, both the Simple Multiband Architecture (late fusion of independently pre-trained encoders) and the Full Multiband Architecture (joint pre-training with the reconstruction loss averaged over bands) reach macro F1-scores about ten points higher than the single-band models. Among the six mixing strategies tried, element-wise averaging, learnable fusion, cross-attention, cross-attention with multi-head attention, cross-attention with LSTM, and an extra transformer block, the expressive ones, LSTM and transformer mixing, give the lowest reconstruction error and the most stable gains. The authors also claim that sampling each band's timestamps independently (asynchronous) instead of aligning all bands to the same timestamps changes performance only minimally, so synchronizing the data is not a prerequisite for multiband benefit.","pith_inferences":["Editorial inference: if the FMA advantage persists on a third survey with different filters, the same fusion recipe should transfer to LSST's six-band cadence without architectural changes, just by adding more per-band encoders.","Editorial inference: because the paper finds synchronous and asynchronous sampling nearly equivalent, a cheap test is to pre-train FMA on several random re-shufflings of the same synchronous MACHO data; if gains stay stable, temporal alignment itself is not the carrier of the cross-band signal.","Editorial inference: an ablation that freezes the FMA encoders and varies only the mixing layer would separate the contribution of joint pre-training from the contribution of the fusion architecture, which the current SMA versus FMA comparison does not fully isolate."],"forward_implications":["Multiband fusion can be dropped into existing single-band light-curve pipelines: pre-train or reuse one Astromer-style encoder per filter, add a mixing layer, and recover about ten F1 points on variable-star classification.","The jointly pre-trained FMA variant is the better choice when labeled data are scarce or noisy, since its advantages are largest at 20–100 samples per class; the modular SMA variant is the cheaper choice when only late fusion is affordable.","Surveys with different cadences per filter, such as ATLAS and LSST, do not need their multiband observations interpolated or synchronized to benefit from fusion; asynchronous inputs are nearly as good as synchronous ones.","Mixing strategies with explicit inter-band interaction, LSTM and transformer fusion, dominate simple averaging and multi-head variants, so the architecture of the fusion layer, not just the presence of multiple bands, drives the gain.","The roughly 10% F1 gain means cross-band information is genuinely complementary to temporal structure; single-band encoders leave that signal unused."],"supporting_citations":[{"why":"Defines the Astromer encoder architecture, masked-reconstruction pre-training objective, and fine-tuning protocol that this paper extends to multiple bands.","marker":"Donoso-Oliva, C. et al. 2023"},{"why":"Supplies the transformer self-attention mechanism and sinusoidal positional encoding used both inside Astromer and in the cross-attention and transformer mixing strategies.","marker":"Vaswani et al. 2017"},{"why":"Presents the MACHO survey whose two-band light curves are used to pre-train the multiband models.","marker":"Alcock et al. 1993"},{"why":"Provides the labeled Alcock variable-star catalog used for fine-tuning and classification evaluation.","marker":"Alcock et al. 2003"},{"why":"Provides the ATLAS variable-star catalog used to evaluate the models on a different survey with asynchronous cyan and orange band sampling.","marker":"Heinze et al. 2018"}],"fun_headline_variants":["Multiband fusion boosts variable-star F1 by 10%","Cross-band embeddings improve light-curve classification by 10%","Light-curve bands fused for 10% F1 gain over single-band","Asynchronous sampling no hurdle for multiband light-curve gains","Multiband transformer adds 10 points to F1 on variable stars"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the multiband models finished training; Table 5 in Section 5.1 records the FMA variants with cross-attention, LSTM, and transformer mixing as exceeding the 72-hour cluster limit on all five runs, and the paper does not state that those entries are completed checkpoints rather than truncated runs.","fun_headline_variants_meta":{"raw":{"variants":["Multiband fusion boosts variable-star F1 by 10%","Cross-band embeddings improve light-curve classification by 10%","Light-curve bands fused for 10% F1 gain over single-band","Asynchronous sampling no hurdle for multiband light-curve gains","Multiband transformer adds 10 points to F1 on variable stars"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000457,"raw_usage":{"total_tokens":2337,"prompt_tokens":1035,"completion_tokens":1302,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":1209}},"tokens_in":651,"tokens_out":1302,"duration_ms":9690,"temperature":1.0,"reasoning_tokens":1209,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:03:52.024852+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Extend the pre-training of the top FMA models past the 72-hour cutoff with a longer cluster budget and compare their F1-scores; if the roughly 10% advantage over single-band models shrinks or vanishes once training actually converges, the central claim would be refuted. A cheaper check is to plot validation RMSE versus wall-clock for the '>72' entries in Table 5; if the loss is still falling steeply at the cutoff, the reported numbers are stopping-time artifacts.","supporting_citations":[{"cited_title":"2017, in Advances in neural infor- mation processing systems","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer self-attention mechanism and sinusoidal positional encoding used both inside Astromer and in the cross-attention and transformer mixing strategies."},{"cited_title":"W., Allsman, R","cited_arxiv_id":null,"evidence_quote":"Presents the MACHO survey whose two-band light curves are used to pre-train the multiband models."},{"cited_title":"R., Becker, A., et al","cited_arxiv_id":null,"evidence_quote":"Provides the labeled Alcock variable-star catalog used for fine-tuning and classification evaluation."},{"cited_title":"N., Tonry, J","cited_arxiv_id":null,"evidence_quote":"Provides the ATLAS variable-star catalog used to evaluate the models on a different survey with asynchronous cyan and orange band sampling."}],"review_version":1}