REVIEW 3 cited by
Positional Encodings for Light Curve Transformers: Playing with Positions and Attention
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We conducted empirical experiments to assess the transferability of a light curve transformer to datasets with different cadences and magnitude distributions using various positional encodings (PEs). We proposed a new approach to incorporate the temporal information directly to the output of the last attention layer. Our results indicated that using trainable PEs lead to significant improvements in the transformer performances and training times. Our proposed PE on attention can be trained faster than the traditional non-trainable PE transformer while achieving competitive results when transfered to other datasets.
Forward citations
Cited by 3 Pith papers
-
How Low Can We Go? Minimum Spectroscopic Requirements For Supernova Subtype Classification
ABC-SN classifies ten supernova subtypes with no performance loss down to R_λ=50 and SNR=5, and only minimal loss at R_λ=25.
-
ABC-SN: Attention Based Classifier for Supernova Spectra
ABC-SN, a transformer-based classifier, reaches 82.5% macro F1 on ten supernova subtypes, outperforming a retrained DASH at 58.9% on the same test set.
-
Multivariate Time-series Transformer Embeddings for Light Curves
Fusing embedding representations from multiple photometric bands improves transformer-based variable star classification by roughly 10% in F1, with only minor sensitivity to how asynchronously the bands are sampled.
Discussion (0). Continue with ORCID to comment.