Multivariate Time-series Transformer Embeddings for Light Curves

Gabriel Chiong; Ignacio Becker; Pavlos Protopapas

REVIEW 3 major objections 5 minor 42 references

Multivariate Time-series Transformer Embeddings for Light Curves

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fusing per-band transformer embeddings into a single sequence improves variable-star classification by about ten F1 points over single-band models.

desk verdict Solid empirical extension of Astromer to multiband light curves, with a credible 10% F1 gain on Alcock; but the best FMA runs may be truncated 72-hour checkpoints, and the ATLAS claim lacks a single-band baseline. read the letter →

arxiv 2506.11637 v1 pith:QE4FHWW7 submitted 2025-06-13 astro-ph.IM

Gabriel Chiong , Ignacio Becker , Pavlos Protopapas This is my paper · ORCID

classification astro-ph.IM

keywords RepresentationLearningMultivariateLightCurvesFoundationalModelsAstromermultibandfusionvariablestarclassificationasynchronoustimeseriesmaskedreconstruction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Astronomical surveys now record the same star in several photometric bands, but existing transformer-based light-curve models treat each band as an independent single-band sequence and ignore cross-band information. This paper tries to establish that a fusion layer placed between per-band encoders and the decoder, mapping the set of single-band embeddings into one unified sequence, recovers that information and improves downstream classification. Using the single-band Astromer encoder, a transformer pre-trained to reconstruct masked magnitudes, as the building block, the authors compare a simple multiband architecture that late-fuses independently pre-trained encoders with a full multiband architecture whose encoders are jointly pre-trained through the fusion layer. They report that both multiband designs outperform single-band models by roughly 10% in F1-score, that the jointly pre-trained version improves further, and that strict alignment of observation times across bands is not required to obtain the gain. The payoff would be a practical recipe for turning pre-trained single-band embeddings into multiband representations for surveys like ATLAS and the upcoming LSST.

What carries the argument

The load-bearing object is the mixing function $f$ that combines per-band encoder outputs $z_i=\Phi_i(Z_i)\in\mathbb{R}^{L\times d}$ into one sequence-level embedding before the decoder reconstructs masked observations, with the reconstruction loss averaged across bands and backpropagated through $f$ into every encoder. The paper's LFEM (late fusion embedding mixing) design pattern is what distinguishes multiband models: SMA applies $f$ only after independently pre-trained encoders are frozen, while FMA trains encoders and $f$ jointly so the embedding spaces become coordinated. The same $f$ is reused during fine-tuning, where the masked-reconstruction decoder is replaced by a classification head.

What would settle it

Extend the pre-training of the top FMA models past the 72-hour cutoff with a longer cluster budget and compare their F1-scores; if the roughly 10% advantage over single-band models shrinks or vanishes once training actually converges, the central claim would be refuted. A cheaper check is to plot validation RMSE versus wall-clock for the '>72' entries in Table 5; if the loss is still falling steeply at the cutoff, the reported numbers are stopping-time artifacts.

Watch

Extended reading notes

Core claim

The paper's central claim is that Multiband Astromer, a stack of band-specific Astromer encoders feeding a mixing function $f$ that produces a unified embedding $y=f(z_0,\dots,z_{N-1})$, uses cross-band information to classify variable stars better than any single-band Astromer baseline. On the Alcock and ATLAS catalogs, both the Simple Multiband Architecture (late fusion of independently pre-trained encoders) and the Full Multiband Architecture (joint pre-training with the reconstruction loss averaged over bands) reach macro F1-scores about ten points higher than the single-band models. Among the six mixing strategies tried, element-wise averaging, learnable fusion, cross-attention, cross-attention with multi-head attention, cross-attention with LSTM, and an extra transformer block, the expressive ones, LSTM and transformer mixing, give the lowest reconstruction error and the most stable gains. The authors also claim that sampling each band's timestamps independently (asynchronous) instead of aligning all bands to the same timestamps changes performance only minimally, so synchronizing the data is not a prerequisite for multiband benefit.

Load-bearing premise

The load-bearing premise is that the multiband models finished training; Table 5 in Section 5.1 records the FMA variants with cross-attention, LSTM, and transformer mixing as exceeding the 72-hour cluster limit on all five runs, and the paper does not state that those entries are completed checkpoints rather than truncated runs.

Editorial extensions

If this is right

Multiband fusion can be dropped into existing single-band light-curve pipelines: pre-train or reuse one Astromer-style encoder per filter, add a mixing layer, and recover about ten F1 points on variable-star classification.
The jointly pre-trained FMA variant is the better choice when labeled data are scarce or noisy, since its advantages are largest at 20–100 samples per class; the modular SMA variant is the cheaper choice when only late fusion is affordable.
Surveys with different cadences per filter, such as ATLAS and LSST, do not need their multiband observations interpolated or synchronized to benefit from fusion; asynchronous inputs are nearly as good as synchronous ones.
Mixing strategies with explicit inter-band interaction, LSTM and transformer fusion, dominate simple averaging and multi-head variants, so the architecture of the fusion layer, not just the presence of multiple bands, drives the gain.
The roughly 10% F1 gain means cross-band information is genuinely complementary to temporal structure; single-band encoders leave that signal unused.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the FMA advantage persists on a third survey with different filters, the same fusion recipe should transfer to LSST's six-band cadence without architectural changes, just by adding more per-band encoders.
Editorial inference: because the paper finds synchronous and asynchronous sampling nearly equivalent, a cheap test is to pre-train FMA on several random re-shufflings of the same synchronous MACHO data; if gains stay stable, temporal alignment itself is not the carrier of the cross-band signal.
Editorial inference: an ablation that freezes the FMA encoders and varies only the mixing layer would separate the contribution of joint pre-training from the contribution of the fusion architecture, which the current SMA versus FMA comparison does not fully isolate.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this one if you care about transformer foundations for light curves. The core result is legitimate and mostly supported: fusing per-band Astromer embeddings and jointly pretraining the stack improves variable-star classification over single-band encoders. The Alcock tables directly show single-band F1 of 0.62–0.64 versus multiband 0.71–0.74, which is the roughly 10% claim. The comparison of six mixing strategies and of SMA versus FMA is a genuine, useful extension, and so is the synchronous-versus-asynchronous sampling test, which suggests strict temporal alignment is not essential. The authors are also honest about pretraining cost.

Soft spots are real but fixable. The FMA variants that work best (TF, CXA, LSTM) mostly exceeded the cluster's 72-hour limit in Table 5, and the paper never says whether the reported numbers come from a 72-hour checkpoint, a resumed run, or a fully converged model. If those are truncated checkpoints, the FMA-versus-SMA comparison is partly a comparison of training effort. That is the main flaw. Second, the ATLAS classification tables have no single-band baseline, so the headline 10% improvement is only directly supported on Alcock. Third, there is no code, no data, and no per-run error bars, despite the text saying metrics are averaged over five runs. The absence of code is manageable, but the 72-hour ambiguity needs a sentence or a rerun. I would also like the fine-tuning setup stated more prominently: the authors fine-tune via unsupervised masked reconstruction on labeled catalogs, using labels only for stratified sampling. That is unusual and easy to miss.

The citation pattern is fine. The paper builds on Astromer and cites the relevant transformer and light-curve literature without overselling; it frames itself as an architectural and empirical contribution rather than a new physical result. The central argument holds up well enough to warrant referee time, but the checkpointing issue is load-bearing for the FMA-versus-SMA claim and should be addressed before publication. I would send it to peer review, ask for clarification on the 72-hour runs and ATLAS baselines, and encourage code release. This is a useful paper for anyone building LSST-era multiband classifiers, and a serious referee can make it stronger.

Referee Report

3 major / 5 minor

Summary. The paper extends the single-band transformer-based foundational model Astromer to multiband light curves by adding fusion mechanisms that combine per-band encoder embeddings. Two architectural paradigms are compared: a Simple Multiband Architecture (SMA), in which independently pretrained single-band encoders are fused only after pretraining, and a Full Multiband Architecture (FMA), in which encoders and fusion are jointly pretrained. Six mixing strategies (AVG, LF, CXA, MHA, LSTM, TF) are evaluated. The models are pretrained on 600k MACHO light curves, fine-tuned on the Alcock and ATLAS catalogs, and evaluated with masked-reconstruction RMSE and downstream classification F1-score. The central claims are that multiband models outperform single-band models by roughly 10% in F1-score, that jointly pretrained FMA models further improve over SMA models, and that synchronous versus asynchronous band sampling matters little for multiband performance.

Significance. If the empirical claims hold, the paper delivers a useful, reasonably efficient recipe for extending a single-band transformer foundation model to multiband light curves, with a systematic ablation of six fusion strategies across two pretraining paradigms and several data budget sizes. The comparison of mixing-layer parameter counts, wall-clock pretraining times, and classification performance at 20-500 labeled samples per class is a practical contribution for the transient/variable-star community. The paper also documents the compute trade-off between joint and independent pretraining, which is relevant for deployment. The main potential value is as a design template for future multiband foundation models. However, the headline results rest on two pieces of evidence that are not fully in place: the FMA runs that produced the best results are reported only through entries marked '>72' hours in the pretraining table, and the ATLAS classification tables lack single-band baselines.

major comments (3)

[Sec. 5.1, Table 5] The paper reports FMA pretraining times of '>72' hours for all five runs of the TF and CXA variants and for four of five LSTM runs, yet Tables 6-11 report final test results for those exact models. The text in Sec. 5.1 says these models 'often exceeded the cluster's 72-hour limit' but never states whether the reported numbers come from the 72-hour checkpoint, from resumed runs, or from fully converged models. If the runs were terminated at 72h, the FMA-versus-SMA comparison is a comparison of stopping times rather than architectures, since the single-band and most SMA models completed within the limit; the abstract's claim that joint pretraining improves over independent encoders would then be unsupported. Please state explicitly, for every result in Tables 6-11, which checkpoints were used and whether any runs were resumed, and if truncated, provide budget-matched comparisons.
[Tables 10 and 11] The advertised 'approximately 10% improvement in F1-score' over single-band models is directly evidenced only in the Alcock synchronous table (Table 9), which contains single-band baseline rows. Table 10 (Alcock pseudo-asynchronous F1) and Table 11 (ATLAS asynchronous F1) report only SMA and FMA columns, with no single-band counterparts. Since the abstract makes a cross-dataset claim, the ATLAS column in particular needs either single-band baseline rows or a qualification that the gain is demonstrated on Alcock only.
[Sec. 5.3, Tables 9-11] The claim that jointly pretrained FMA encoders 'further improve performance' over independently pretrained SMA encoders is not consistently supported by the reported means. In Table 9 under 'All', SMA LSTM reaches 0.74 versus FMA LSTM 0.72, and SMA Transformer 0.73 versus FMA Transformer 0.71; similar reversals appear in Tables 10-11 at several sample sizes. The tables report only averages over five runs, without standard deviations or significance tests, so the small differences on which the FMA-over-SMA claim rests may reflect run-to-run noise. Please either add uncertainty estimates and significance testing, or soften the abstract and conclusions to describe the FMA-SMA difference as favorable only for specific fusion strategies or sample sizes.

minor comments (5)

[Sec. 3.2.1] The sentence 'the where embeddings, where embeddings z_i^{(k)} from N bands are aggregated' contains a duplicated phrase; it should read 'where the embeddings z_i^{(k)} from N bands are aggregated'.
[Throughout] The mixing strategy is typeset as 'A VG' in the text but as 'AVG' in the tables; please unify the notation.
[Sec. 5.2] The term 'fine-tuning' is used for a second stage of masked reconstruction in which labels are not used, and classification heads are trained afterwards in Sec. 5.3; this two-stage procedure should be clarified at the start of Sec. 5.2 to avoid confusing the reader.
[Sec. 4.2.1] The inclusion of the UNK class in the Alcock catalog differs from the original Astromer setup; the paper should state how this affects comparability with published Astromer classification results.
[Fig. 6] The figure caption says whiskers extend to points within 1.5 times the IQR, and the text invokes 'variance across runs'; reporting the numerical standard deviations in the caption or in Table 5 would support the stability claims made in Sec. 5.1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the multiband improvement claim is an empirical architecture comparison, not a derivation that reduces to its inputs.

full rationale

The central claim—that multiband fusion improves F1-score by about 10% over single-band models and that jointly pre-trained FMA variants improve over independently pre-trained SMA variants—is an empirical result obtained by training and evaluating models on held-out Alcock and ATLAS data. The paper reports F1 and RMSE averaged over five runs and compares in-paper single-band baselines with multiband architectures under the same preprocessing and evaluation protocol. No fitted parameter is renamed as a prediction, and no equation defines the reported improvement in terms of the inputs used to produce it. The self-citations to Astromer are used for architectural provenance, hyperparameter selection, masking strategy, and dataset choice, but the multiband advantage is not justified by citing Astromer; it is evidenced by the paper's own Tables 6-11. There is no uniqueness theorem imported from the authors' prior work, no ansatz smuggled in via citation that carries the conclusion, and no renaming of a known empirical pattern as a new result. The disclosed fact that some FMA runs exceeded the cluster's 72-hour limit (Sec. 5.1, Table 5) and the absence of single-band rows in some ATLAS tables are experimental-validity caveats, not circular reductions. On the evidence quoted, no circular step is established.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on a stack of design choices inherited from Astromer and on transfer assumptions between surveys. None are fitted constants in a physical derivation. The main fragility is operational: reported FMA results may include runs that did not finish pre-training.

free parameters (5)

Sequence length L = 200
Fixed-length window used for all inputs; chosen to match Astromer and cap model size (Section 4.3.1).
Embedding dimension d = 256
Encoder output and attention dimension inherited from Astromer (Section 2.1).
Attention heads and encoder layers = h=4, M=2
Architecture hyperparameters inherited from Astromer; they set the capacity of each per-band encoder.
Masking schedule = 50% masked, 20% original, 10% random
Pre-training objective hyperparameters inherited from Astromer (Section 2.2); not derived from data.
SNR subset size = 600,000
Only the highest signal-to-noise MACHO light curves were used to shorten pre-training time (Section 4.1.2).

assumptions (4)

domain assumption Masked magnitude reconstruction produces embeddings that transfer to downstream classification tasks.
The whole method assumes that denoising randomly masked magnitudes yields representations useful for class labels; used throughout Sections 2.2 and 3.
domain assumption Sinusoidal positional encoding applied directly to irregularly sampled MJD values is a valid temporal representation.
Equation (3) and (4) are applied to the raw observation times without learned or relative encodings; this is an inherited Astromer design choice.
domain assumption ATLAS cyan and orange bands can stand in for MACHO blue and red bands for fine-tuning.
Table 4 justifies the band matching by approximate wavelength overlap, while Figure 5 shows a large photometric zero-point offset between ATLAS and MACHO that is not corrected.
domain assumption Zero-padding sequences shorter than L=200 is harmless.
Sequences with fewer than 200 observations are zero-padded (Section 4.3.1), with no weighting or masking of padding positions described.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multivariate Time-series Transformer Embeddings for Light Curves." pith.science (2026). https://pith.science/paper/QE4FHWW7

@misc{pith2026250611637,
  author       = {Pith},
  title        = {Pith review of: Multivariate Time-series Transformer Embeddings for Light Curves},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QE4FHWW7}},
  note         = {Machine review of arXiv:2506.11637}
}

read the original abstract

Astronomical surveys produce time-series data by observing stellar objects across multiple wavelength bands. Foundational transformer-based models, such as Astromer, encode each time-series as a sequence of embeddings of uniform dimensions. However, such models operate independently on each band at a single time and do not natively leverage information across telescope filters. We extend this framework by introducing a fusion mechanism that maps the collection of single-band embeddings to a unified sequence representation, enabling multiband modeling for downstream tasks. The challenge lies in devising a mechanism within the encoder to coordinate between data from different wavelengths, which are often recorded at asynchronous times. We pre-train multiband models on a subset of 600000 high signal-to-noise light curves from the MACHO survey and fine-tune them using the Alcock and ATLAS survey datasets. Experimental results show that both our proposed multiband architectures outperform the single-band models by approximately 10% in F1-score, with jointly pre-trained multiband encoders further improving performance over a collection of independently pre-trained single-band encoders. Furthermore, our experiments show that there are minimal differences in multiband performance when sampling individual band data asynchronously versus sampling all individual bands on the same set of time-steps. However, jointly pre-trained models can take more than twice the time to pre-train. These results demonstrate the trade-offs of the multiband approach where multivariate data are available.

Figures

Figures reproduced from arXiv: 2506.11637 by the authors.

**Figure 1.** The Astromer architecture for the masked reconstruction task with matrix dimensions shown. The default configuration uses h = 4 attention heads and M = 2 encoder layers. The encoder’s output embedding is derived from the final attention block. The decoder can be modified to suit the desired downstream task. contiguous fixed-length sequences of L = 200 observations. Sequences with L < 200 are zero-padded1 . 2.1. … view at source ↗

**Figure 2.** The Multiband Astromer encoder architecture with matrix dimensions shown. The encoder ingests N input bands, with each band processed by a distinct Astromer encoder. Following the LFEM design pattern, the fusion stage can be instantiated using various mixing strategies (e.g., AVG, LF, CXA, etc.), as described in Sect. 3.2. pre-trained independently. In contrast, FMA uses the fusion output during pre-training to re… view at source ↗

**Figure 3.** An example for comparing between the different sampling modes of synchronous, pseudo-asynchronous, and asynchronous. In this example, N = 3 and L = 2, so each of the three bands sample two observations from time t0, . . . , ti . The top table depicts the synchronous case, where each observation time sampled contains magnitude observations for all bands. The middle table depicts the pseudo-asynchronous case, where… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Example light curve showing observations in MJD and [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Magnitude distributions of the MACHO, Alcock, and ATLAS datasets across both bands used in our experiments. While MACHO and Alcock exhibit similar distributions due to their shared origin, the ATLAS distribution is offset due to differences in photometric zero points …

**Figure 6.** Figure 6: RMSE median over five runs for each model pre [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: Confusion matrix for FMA (TF mixing) on the Alcock dataset, pseudo-asynchronous mode. LPV is most accurately classified; RRc and EC show lower accuracy and greater variability. Mira CB DB Pulse Predicted Class Mira CB DB Pulse True Class 84.9 3.1 12.0 3.1 2.2 0.4 0.8 …

**Figure 8.** Figure 8: Confusion matrix for FMA (TF mixing) on the ATLAS dataset, asynchronous mode. Pulse class is most accurately predicted. CB and DB classes are frequently confused with each other and with Mira. multilayer perceptron, and more expressive mechanisms such as cross-attenti…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 33 canonical work pages

[1]

W., Allsman, R

Alcock, C., Akerlof, C. W., Allsman, R. A., et al. 1993, Nature, 365, 621

work page 1993
[2]

R., Becker, A., et al

Alcock, C., Alves, D. R., Becker, A., et al. 2003, The Astrophysical Journal, 598, 597–609

work page 2003
[3]

L., Kiros, J

Ba, J. L., Kiros, J. R., & Hinton, G. E. 2016, Layer Normalization

work page 2016
[4]

2020, Monthly Notices of the Royal Astronomical Society, 493, 2981

Becker, I., Pichara, K., Catelan, M., et al. 2020, Monthly Notices of the Royal Astronomical Society, 493, 2981

work page 2020
[5]

2025, A&A, 694, A183

Becker, I., Protopapas, P., Catelan, M., & Pichara, K. 2025, A&A, 694, A183

work page 2025
[6]

C., Kulkarni, S

Bellm, E. C., Kulkarni, S. R., Graham, M. J., et al. 2019, PASP, 131, 018002

2019
[7]

B., Ivezi ´c, Z., Jones, R

Bianco, F. B., Ivezi ´c, Z., Jones, R. L., et al. 2021, The Astrophysical Journal Supplement Series, 258, 1

work page 2021
[8]

A., Adeli, E., et al

Bommasani, R., Hudson, D. A., Adeli, E., et al. 2022, On the Opportunities and Risks of Foundation Models

work page 2022

Show all 42 references

[9]

2019, The Astronomical Journal, 158, 257

Boone, K. 2019, The Astronomical Journal, 158, 257

2019
[10]

2001, Machine learning, 45, 5

Breiman, L. 2001, Machine learning, 45, 5

2001
[11]

2024, A&A, 689, A289

Cabrera-Vives, G., Moreno-Cartagena, D., Astorga, N., et al. 2024, A&A, 689, A289

2024
[12]

Carroll, B. W. & Ostlie, D. A. 2017, An Introduction to Modern Astrophysics, 2nd edn. (Cambridge University Press)

2017
[13]

& Smith, H

Catelan, M. & Smith, H. A. 2015, Pulsating Stars (Wiley-VCH)

2015
[14]

C., Magnier, E

Chambers, K. C., Magnier, E. A., Metcalfe, N., et al. 2019, The Pan-STARRS1 Surveys

2019
[15]

& Moss, A

Charnock, T. & Moss, A. 2017, The Astrophysical Journal Letters, 837, L28

2017
[16]

2014, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), ed

Cho, K., van Merriënboer, B., Gulcehre, C., et al. 2014, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), ed. A. Moschitti, B. Pang, & W. Daelemans (Doha, Qatar: As- sociation for Computational Linguistics), 1724–1734 Cádiz-Ley...

2014
[17]

2024, Transformer-Based Astronomical Time Series Model with Uncertainty Estimation for Detecting Misclassified Instances

Donoso-Oliva, C. 2024, Transformer-Based Astronomical Time Series Model with Uncertainty Estimation for Detecting Misclassified Instances

2024
[18]

2019, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. 2019, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

2019
[19]

Donoso-Oliva, C., Cabrera-Vives, G., Protopapas, P., Carrasco-Davis, R., & Es- tevez, P. A. 2021, Monthly Notices of the Royal Astronomical Society, 505, 6069

2021
[20]

2023, A&A, 670, A54

Donoso-Oliva, C., Becker, I., Protopapas, P., et al. 2023, A&A, 670, A54

2023
[21]

2021, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Förster, F., Cabrera-Vives, G., Castillo-Navarrete, E., et al

Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. 2021, An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Förster, F., Cabrera-Vives, G., Castillo-Navarrete, E., et al. 2021, AJ, 161, 242 Gaia Collaboration, Prusti, T., de Bruijne, J. H. J., et al. 20...

2021
[22]

J., Djorgovski, S

Graham, M. J., Djorgovski, S. G., Drake, A. J., et al. 2014, Monthly Notices of the Royal Astronomical Society, 439, 703–718

2014
[23]

N., Tonry, J

Heinze, A. N., Tonry, J. L., Denneau, L., et al. 2018, AJ, 156, 241

2018
[24]

& Schmidhuber, J

Hochreiter, S. & Schmidhuber, J. 1997, Neural Computation, 9, 1735 Ivezi´c, Ž., Kahn, S. M., Tyson, J. A., et al. 2019, ApJ, 873, 111

1997
[25]

& Bloom, J

Jamal, S. & Bloom, J. S. 2020, The Astrophysical Journal Supplement Series, 250, 30

2020
[26]

L., et al

Kim, Dae-Won, Protopapas, Pavlos, Bailer-Jones, Coryn A. L., et al. 2014, A&A, 566, A43

2014
[27]

Kingma, D. P. & Ba, J. 2017, Adam: A Method for Stochastic Optimization

2017
[28]

2021, CAT: Cross Attention in Vision Trans- former

Lin, H., Cheng, X., Wu, X., et al. 2021, CAT: Cross Attention in Vision Trans- former

2021
[29]

2023, arXiv preprint arXiv:2308.06404 Article number, page 13 A&A proofs:manuscript no

Moreno-Cartagena, D., Cabrera-Vives, G., & Eyheramendy, S. 2023, arXiv preprint arXiv:2308.06404 Article number, page 13 A&A proofs:manuscript no. aa_manuscript

2023 arXiv
[30]

2025, Lever- aging Pre-Trained Visual Transformers for Multi-Band Photometric Light Curve Classification

Moreno-Cartagena, D., Protopapas, P., Cabrera-Vives, G., et al. 2025, Lever- aging Pre-Trained Visual Transformers for Multi-Band Photometric Light Curve Classification

2025
[31]

H., & Waldmann, I

Morvan, M., Nikolaou, N., Yip, K. H., & Waldmann, I. 2022, Don’t Pay Atten- tion to the Noise: Learning Self-supervised Representations of Light Curves with a Denoising Time Series Transformer

2022
[32]

S., Biswas, R., & Hložek, R

Muthukrishna, D., Narayan, G., Mandel, K. S., Biswas, R., & Hložek, R. 2019, PASP, 131, 118002 Möller, A. & de Boissière, T. 2019, Monthly Notices of the Royal Astronomical Society, 491, 4277

2019
[33]

2016, in Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16 (Red Hook, NY , USA: Curran Associates Inc.), 3889–3897

Neil, D., Pfeiffer, M., & Liu, S.-C. 2016, in Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS’16 (Red Hook, NY , USA: Curran Associates Inc.), 3889–3897

2016
[34]

2017, FATS: Feature Analysis for Time

Nun, I., Protopapas, P., Sim, B., et al. 2017, FATS: Feature Analysis for Time

2017
[35]

2012, arXiv e-prints, arXiv:1211.5063

Pascanu, R., Mikolov, T., & Bengio, Y . 2012, arXiv e-prints, arXiv:1211.5063

2012 arXiv
[36]

A., & Förster, F

Pimentel, O., Estévez, P. A., & Förster, F. 2022, The Astronomical Journal, 165, 18

2022
[37]

2018, Improving lan- guage understanding by generative pre-training

Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. 2018, Improving lan- guage understanding by generative pre-training

2018
[38]

W., Starr, D

Richards, J. W., Starr, D. L., Butler, N. R., et al. 2011, The Astrophysical Journal, 733, 10

2011
[39]

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. 2014, J. Mach. Learn. Res., 15, 1929–1958 Sánchez-Sáez, P., Lira, P., Cartier, R., et al. 2019, The Astrophysical Journal Supplement Series, 242, 10 Sánchez-Sáez, P., Reyes, I., Valenzuela, C., et a...

2014
[40]

L., Denneau, L., Heinze, A

Tonry, J. L., Denneau, L., Heinze, A. N., et al. 2018, PASP, 130, 064505

2018
[41]

2017, in Advances in neural infor- mation processing systems

Vaswani, A., Shazeer, N., Parmar, N., et al. 2017, in Advances in neural infor- mation processing systems

2017
[42]

A., Hosseinzadeh, G., Berger, E., et al

Villar, V . A., Hosseinzadeh, G., Berger, E., et al. 2020, The Astrophysical Jour- nal, 905, 94 Article number, page 14

2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

W., Allsman, R

Alcock, C., Akerlof, C. W., Allsman, R. A., et al. 1993, Nature, 365, 621

work page 1993

[2] [2]

R., Becker, A., et al

Alcock, C., Alves, D. R., Becker, A., et al. 2003, The Astrophysical Journal, 598, 597–609

work page 2003

[3] [3]

L., Kiros, J

Ba, J. L., Kiros, J. R., & Hinton, G. E. 2016, Layer Normalization

work page 2016

[4] [4]

2020, Monthly Notices of the Royal Astronomical Society, 493, 2981

Becker, I., Pichara, K., Catelan, M., et al. 2020, Monthly Notices of the Royal Astronomical Society, 493, 2981

work page 2020

[5] [5]

2025, A&A, 694, A183

Becker, I., Protopapas, P., Catelan, M., & Pichara, K. 2025, A&A, 694, A183

work page 2025

[6] [6]

C., Kulkarni, S

Bellm, E. C., Kulkarni, S. R., Graham, M. J., et al. 2019, PASP, 131, 018002

2019

[7] [7]

B., Ivezi ´c, Z., Jones, R

Bianco, F. B., Ivezi ´c, Z., Jones, R. L., et al. 2021, The Astrophysical Journal Supplement Series, 258, 1

work page 2021

[8] [8]

A., Adeli, E., et al

Bommasani, R., Hudson, D. A., Adeli, E., et al. 2022, On the Opportunities and Risks of Foundation Models

work page 2022