Transformer models for astrophysical time series and the GRB prompt-afterglow relation

Transformer models have recently become very successful in the natural language domain. Their value as sequence-to-sequence translators there, also makes them a highly interesting technique for learning relationships between astrophysical time series. Our aim is investigate how well such a transformer neural network can establish causal temporal relations between different channels of a single-source signal. We thus apply a transformer model to the two phases of Gamma-Ray Bursts (GRBs), reconstructing one phase from the other. GRBs are unique instances where a single process and event produces two distinct time variable phenomena: the prompt emission and the afterglow. We here investigate if a transformer model can predict the afterglow flux from the prompt emission. If successful, such a predictive scheme might then be distilled to the most important underlying physics drivers in the future. We combine the transformer model with a novel dense neural network setup to directly estimate the starting value of the prediction. We find that the transformer model can, in some instances, successfully predict different phases of canonical afterglows, including the plateau phase. Hence it is a useful and promising new astrophysical analysis technique. For the GRB test case, the method marginally exceeds the baseline model overall, but still achieves accurate recovery of the prompt-afterglow fluence-fluence correlation in reconstructed light curves. Despite this progress, we conclude that consistent improvement over the baseline model is not yet achieved for the GRB case. We discuss the future improvements in data and modeling that are required to identify new physical-relation parameters or new insights into the single process driving both GRB phases.