Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

Bumblebee: Foundation Model for Particle Physics Discovery

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read Bumblebee, a transformer without positional encodings pre-trained on masked particle 4-vectors, improves top-pair mass reconstruction by 10-20% and fine-tunes to toponium (AUROC 0.877) and initial-state classification (AUROC 0.625).

desk verdict Bumblebee is a plausible permutation-invariant pretraining recipe, but the headline 10-20% gain is not yet attributable to pretraining because the baseline is unspecified and the from-scratch control is missing. read the letter →

arxiv 2412.07867 v1 pith:5MLG4HMJ submitted 2024-12-10 hep-ex cs.LGhep-ph

classification hep-excs.LGhep-ph
keywords foundationmodeltransformermaskedpre-trainingtopquarkreconstructionpermutationinvariancetoponiumLHCmissingtransverseenergy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a transformer borrowed from language modeling, stripped of positional encodings and fed particle 4-vectors as tokens, can act as a reusable foundation model for particle physics. On dileptonic top-pair events, pre-training with a masked-vector objective is claimed to improve the invariant-mass resolution of the reconstructed $t\bar{t}$ system by 10-20% relative to a supervised transformer. The same pre-trained weights then fine-tune to two downstream tasks: distinguishing toponium from ordinary top-pair production (AUROC 0.877) and classifying the initial state that produced the pair (AUROC 0.625). The motivation is that collision events have no natural ordering, so a permutation-invariant model that learns the mapping from reconstructed detector data to generator-level kinematics should transfer to new-particle searches. If true, this points toward reusable pre-trained models for LHC analyses rather than task-specific networks trained from scratch.

What carries the argument

The load-bearing mechanism is masked 4-vector pre-training on permutation-invariant set inputs. Bumblebee is an 8-layer, 57M-parameter bidirectional transformer encoder without positional encodings; each token is the unweighted sum of a linear embedding of $(p_T, \eta, \phi, m, \text{b-tag})$, a modified PDG-ID embedding, a level-type embedding, and a mask-status embedding. The Cloze-style objective forces the model to learn the event's kinematic topology, including the relation between invisible neutrinos, which appear only as missing transverse energy, and visible leptons and jets, and to learn the detector-level transformation from reconstruction to generator quantities. This same objective is then repurposed as the reconstruction head, and fine-tuning replaces it with classification tokens to perform new-particle discrimination.

What would settle it

Train the same 8-layer, 57M-parameter transformer on the reconstruction task from random initialisation, with the same dataset, optimizer, and number of update steps as pre-training, and measure $m(t\bar{t})$ resolution; if the randomly initialised model matches or beats the reported 10-20% gain, the improvement is not attributable to masked pre-training.

Watch

Extended reading notes

Core claim

Bumblebee's central claim is that masked pre-training on particle kinematics is a sufficient self-supervised objective for learning transferable LHC physics. The model embeds each particle's five-dimensional vector $(p_T, \eta, \phi, m, \text{b-tag})$ at both generator and reconstruction level, adds learned embeddings for particle identity, generator-versus-reconstruction level, and mask status, and omits positional encodings so the input is order-invariant. Pre-training randomly masks particle 4-vectors, for half the training one particle per event at probability $1/n_{\text{particles}}$ and for the other half all 4-vectors at one level, minimizing mean squared error on the missing vectors; validation and testing focus on predicting generator-level 4-vectors from reconstruction-level information, framed as dileptonic top reconstruction. The paper reports a 10-20% resolution gain over a supervised transformer, fine-tuned AUROC 0.877 for toponium-versus-$t\bar{t}$ discrimination, and AUROC 0.625 for gluon-gluon versus quark-antiquark initial-state classification, the positive class being gluon-gluon.

Load-bearing premise

The load-bearing assumption is that the 'supervised transformer' used for comparison in Fig. 2C is an equally capable model trained with the same data and effort; the paper never specifies that baseline's architecture, size, or training, so a weaker baseline could inflate the apparent pre-training gain.

Editorial extensions

If this is right

  • The masked pre-training objective itself doubles as the reconstruction task: summing the predicted generator-level daughter 4-vectors gives a 10-20% improvement in $m(t\bar{t})$ resolution over a supervised transformer, which is directly relevant for resonant $t\bar{t}$ searches.
  • A single set of pre-trained weights transfers to two classification tasks, toponium-versus-$t\bar{t}$ discrimination and initial-state classification, without changing the input representation.
  • Because the input representation treats each particle as an unordered token with no positional encoding, the model avoids imposing an arbitrary ordering convention on collision events.
  • The embedding procedure does not exclude photons, so the same architecture can be applied to final states with photons even though the demonstration uses dileptonic $t\bar{t}$ events.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A clean attribution test would be to train the same 8-layer, 57M-parameter architecture from random initialisation on the reconstruction task with identical data and compute; the current comparison does not specify the supervised transformer's size or training, so part of the reported gain could in principle come from capacity rather than pre-training.
  • If the representation is as general as claimed, the same pre-training recipe should transfer to other multi-neutrino topologies such as $t\bar{t}H$ and four-top production, where missing momenta dominate the reconstruction challenge; the authors name these as future work.
  • The 0.625 initial-state AUROC suggests the model captures a weak but real kinematic handle on the production mechanism; combining its representation with spin-correlation observables might push this toward practical use.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces Bumblebee, a BERT-inspired bidirectional transformer that omits positional encodings and embeds particle 4-vectors (pT, eta, phi, m) together with b-tag scores, level-type, modified PDG ID, and mask-status embeddings. The model is pre-trained on a masked-vector prediction task using dileptonic ttbar Monte Carlo events, where both generator- and reconstruction-level objects are included. The authors claim that pre-training improves the resolution of the reconstructed ttbar invariant mass by 10-20% relative to a supervised transformer (Fig. 2C), that fine-tuning gives toponium discrimination AUROC of 0.877 (Fig. 2A) and initial-state classification AUROC of 0.625 (Fig. 2B), and that an ablation study shows the PDG ID embedding is the most important input component (Fig. 3).

Significance. If the central claims hold, the paper offers a conceptually interesting step toward a permutation-invariant, physics-aware foundation model for LHC analyses: the embedding design is simple, the pre-training task is well matched to a real reconstruction problem (dileptonic ttbar with two neutrinos), and the reported improvement in m(ttbar) resolution at high invariant mass is potentially valuable for BSM resonance searches. The paper also honestly states its limitations (focus on one process, no photons yet) and provides an ablation of input embeddings. However, the significance is currently limited by the lack of a fully specified baseline and the absence of statistical uncertainties, which prevents the reader from verifying that the 10-20% improvement is real and attributable to pre-training rather than to model capacity or training choices.

major comments (3)
  1. [Section 5, Fig. 2C] The central claim of a 10-20% improvement in m(ttbar) resolution is not testable as presented because the 'supervised Transformer' baseline is not specified: its architecture, parameter count, input representation, loss function, training data size, and hyperparameters are all absent. Bumblebee is (L=8, dmodel=768, A=16, 57M parameters), but the reader cannot verify whether the baseline has comparable capacity. More importantly, there is no same-architecture control trained from random initialization on the reconstruction task; without such a control, the improvement could be due to model capacity, optimization, or regularization rather than the masked pre-training objective. Please add a complete description of the baseline and, crucially, a from-scratch Bumblebee trained directly on the reconstruction loss.
  2. [Abstract; Section 5, Fig. 2A] The abstract's statement that Bumblebee 'excels in downstream tasks, including toponium discrimination (AUROC 0.877)' is contradicted by the data in Fig. 2A, where the supervised Transformer achieves AUROC 0.934 and the DNN achieves 0.831. The paper should either temper the claim of excelling for toponium or provide an explanation (e.g., statistical uncertainty, different operating points, or a discussion of why the supervised Transformer is a stronger baseline for this task). As written, the downstream claim is inconsistent with the presented ROC curves.
  3. [Section 5, Fig. 2C] No statistical uncertainties are given for the AUC values or for the P84-P16 resolution curves. With a 7M-event sample and a 15% test split, the resolution differences shown in Fig. 2C may be statistically meaningful, but the reader cannot assess whether the 10-20% improvement is significant, particularly in the high-mass bins where the curve appears to cross. The initial-state classification AUC of 0.625 versus 0.601 for the supervised Transformer is a small difference that could easily be within uncertainty; please add error bars or confidence intervals.
minor comments (5)
  1. [Section 3.3] The masking probability is described as '(1/nparticles)%' which is dimensionally confusing; presumably each particle is masked with probability 1/n_particles, but this should be stated unambiguously as a fraction or percentage.
  2. [Section 3.2, Fig. 1] The text states that b-tag scores indicate a likelihood, but the figure shows only binary values 0 and 1; clarify whether b-tag scores are continuous and, if so, how they are embedded. The figure caption also mixes PDG ID values and level-type values in a way that is hard to parse.
  3. [Section 5] In the toponium discrimination description, the paper says a masked vector (1,0,0,0,0) is added for signal and (0,0,0,0,0) for background, and that the model is trained with weighted binary cross-entropy; it is unclear how a 5-vector target is used in a scalar classification loss. Please specify the loss computation on this vector.
  4. [Section 4] The toponium sample is generated with a toy model and the ttbar sample with POWHEG, but the description does not state whether the two samples are matched in kinematic acceptance or whether the same object/event selection is applied to both; this matters for interpreting the discrimination AUC. Please clarify.
  5. [References] Reference [37] (Schick and Schütze) is cited for the 'Cloze task', but the original BERT paper (Devlin et al., 2018) introduced the masked-language-model objective; consider citing the original source or explaining why the Cloze-task reference is the relevant one.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild circularity: the headline reconstruction result evaluates the same masked 4-vector prediction objective used in pre-training, so it is an in-distribution check rather than an independent transfer test; the downstream fine-tuning tasks provide separate evidence.

  1. other [Section 3.3 (Pre-training) and Section 5 (Dileptonic top quark reconstruction)]
    "Section 3.3: 'The model minimizes the batch-average mean squared error (MSE) on the predicted masked 4-vectors. Validation and testing of Bumblebee focus on predicting generator-level 4-vectors from reconstruction-level information, framed as dileptonic top quark reconstruction.' Section 5: 'Our pre-training task doubles as a dileptonic top quark reconstruction challenge. The reconstructed top quark is the sum of predicted generator-level daughter four-vectors.'"

    The generator-level 4-vector predictions used to compute the m(ttbar) resolution are exactly the targets of the pre-training MSE described in Section 3.3. Therefore the reported 10-20% improvement is a held-out evaluation of the pre-training objective itself, not an independent downstream task. The circularity is mild: the result is not algebraically forced, since the model could have failed on the withheld test set, but the 'prediction' is the same quantity the model was optimized to produce. The comparison is also weakened by the unspecified supervised Transformer baseline, which is a correctness risk rather than a circular step.

full rationale

The paper's core reconstruction claim is computed from the same generator-level 4-vector predictions that constitute the pre-training target, making the reconstruction evaluation in-distribution with respect to the masked-prediction objective. The text is transparent about this, explicitly stating that the pre-training task doubles as the reconstruction challenge. No load-bearing self-citations, imported uniqueness theorems, or renamed known results were found. The toponium discrimination and initial-state classification results are separate fine-tuning tasks with independent labels, which limit the overall circularity. The unspecified supervised Transformer baseline is a reproducibility/validity concern, not a demonstration of circular reasoning. Overall, the central claim has some independent content, so the circularity score is low.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims depend on domain choices in the input representation and simulation chain rather than on derivational equations. Model hyperparameters and hand-set encodings are listed as free parameters. No new physical entities are introduced. The most important audit point is that the pre-training target equals the reconstruction evaluation metric, so the resolution gain is an in-distribution check of the pre-training objective rather than independent evidence of generalization.

free parameters (4)
  • Mask probability = 1/n_particles percent per particle for half of training; full-level masking for other half
    Chosen by hand; controls the difficulty of the pre-training objective (Section 3.3).
  • Modified PDG ID mapping = b jet=5, non-b jet=41, MET=40, plus offset of +50
    Hand-assigned input labels; the ablation shows the PDG ID embedding is the most important single input, so the central result depends on this mapping (Section 3.2, Fig. 3).
  • Toponium class imbalance weight = 10:1
    Used in weighted binary cross-entropy for fine-tuning on toponium vs ttbar (Section 5).
  • Model hyperparameters = L=8, d_model=768, A=16, lr ~1e-4, batch size 16, 10 epochs
    Architecture and training choices; no pretrained weights are released, so replication must use these exact values (Sections 3.1, 3.3).
assumptions (4)
  • domain assumption Particle order in a collision event is physically meaningless, so permutation invariance is the correct inductive bias.
    Motivates removing positional encodings (Section 3.1); if order effects carry information in real detector data, the architecture discards it.
  • domain assumption Generator-level 4-vectors from Monte Carlo are an appropriate pre-training target and remain available after detector simulation.
    Pre-training uses truth-level quantities (Section 3.3); simulations are approximations of nature.
  • domain assumption DELPHES fast simulation with the default CMS card is a sufficient proxy for the real CMS detector.
    All results are on fast-simulated events (Section 4); real detector effects may differ.
  • domain assumption The masked 4-vector prediction objective transfers to downstream classification tasks.
    Central transfer claim; the ablation only checks embedding importance on validation loss, not downstream task success (Section 7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bumblebee: Foundation Model for Particle Physics Discovery." pith.science (2026). https://pith.science/paper/5MLG4HMJ

@misc{pith2026241207867,
  author       = {Pith},
  title        = {Pith review of: Bumblebee: Foundation Model for Particle Physics Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5MLG4HMJ}},
  note         = {Machine review of arXiv:2412.07867}
}
read the original abstract

Bumblebee is a foundation model for particle physics discovery, inspired by BERT. By removing positional encodings and embedding particle 4-vectors, Bumblebee captures both generator- and reconstruction-level information while ensuring sequence-order invariance. Pre-trained on a masked task, it improves dileptonic top quark reconstruction resolution by 10-20% and excels in downstream tasks, including toponium discrimination (AUROC 0.877) and initial state classification (AUROC 0.625). The flexibility of Bumblebee makes it suitable for a wide range of particle physics applications, especially the discovery of new particles.

Figures

Figures reproduced from arXiv: 2412.07867 by the authors.

Figure 1
Figure 1. The embedding procedure for Bumblebee for a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. A) The receiver operating characteristic (ROC) curve for Bumblebee fine-tuned on [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The comparison of the best validation loss obtained on the pre-training objective when [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A universal vision transformer for fast calorimeter simulations

    hep-ph 2026-01 conditional novelty 6.0 of 10

    A vision-transformer flow-matching model generates calorimeter showers across regular and irregular detector geometries at millisecond speeds, and pretraining plus fine-tuning cuts training cost by about half.

  2. Enhancing next token prediction based pre-training for jet foundation models

    hep-ph 2025-12 conditional novelty 6.0 of 10

    Using continuous particle features as input and combining next-token with masked-token pre-training markedly improves classification accuracy of the OmniJet jet foundation model without visibly hurting its generative quality.

  3. Measuring Quantum Discord at the LHC

    hep-ph 2024-12 conditional novelty 6.0 of 10

    The authors derive closed-form formulas for quantum discord in the ttbar system and predict it can be measured at the LHC with sub-percent precision using the kinematic method.

  4. Pretrained Event Classification Model for High Energy Physics Analysis

    hep-ph 2024-12 unverdicted novelty 6.0 of 10

    A pretrained GNN on 120M simulated LHC events improves downstream event-classification accuracy when training data are scarce, with benefits shrinking as data grow.

Reference graph

Works this paper leans on

42 extracted references · 19 canonical work pages · cited by 4 Pith papers

  1. [1]

    Afik and J

    Y . Afik and J. R. M. de Nova. Entanglement and quantum tomography with top quarks at the lhc. The European Physical Journal Plus , 136(9), 2021. doi: 10.1140/epjp/s13360-021-01902-1

  2. [2]

    J. A. Aguilar-Saavedra. Overview of models for the t¯t asymmetry. Il Nuovo Cimento C , 35(3): 167, 2012. doi: 10.1393/ncc/i2012-11240-7

  3. [3]

    J. A. Aguilar-Saavedra. Toponium hunter’s guide, 2024

  4. [4]

    Albertsson, P

    K. Albertsson, P. Altoe, D. Anderson, J. Anderson, M. Andrews, J. P. A. Espinosa, A. Aurisano, L. Basara, A. Bevan, W. Bhimji, D. Bonacorsi, B. Burkle, P. Calafiura, M. Campanelli, L. Capps, F. Carminati, S. Carrazza, Y . fan Chen, T. Childers, Y . Coadou, E. Coniavitis, K. Cranmer, C. David, D. Davis, A. D. Simone, J. Duarte, M. Erdmann, J. Eschle, A. Fa...

  5. [5]

    Alioli, P

    S. Alioli, P. Nason, C. Oleari, and E. Re. A general framework for implementing NLO calculations in shower Monte Carlo programs: the POWHEG BOX . JHEP, 06:043, 2010. doi: 10.1007/JHEP06(2010)043

  6. [6]

    Alwall, M

    J. Alwall, M. Herquet, F. Maltoni, O. Mattelaer, and T. Stelzer. Madgraph 5: going beyond. Journal of High Energy Physics , 2011(6), 2011. doi: 10.1007/jhep06(2011)128

  7. [7]

    Observation of higgs boson production in association with a top quark pair at the lhc with the atlas detector

    ATLAS Collaboration. Observation of higgs boson production in association with a top quark pair at the lhc with the atlas detector. Physics Letters B, 784:173, 2018. doi: 10.1016/j.physletb. 2018.07.035

  8. [9]

    J. Birk, A. Hallin, and G. Kasieczka. Omnijet- α: The first cross-task foundation model for particle physics, 2024

Show all 42 references
  1. [10]

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B....

  2. [11]

    Butter, N

    A. Butter, N. Huetsch, S. P. Schweitzer, T. Plehn, P. Sorrenson, and J. Spinner. Jet diffusion versus jetgpt – modern networks for the lhc, 2023

  3. [12]

    Observation of t¯tH production

    CMS Collaboration. Observation of t¯tH production. Physical Review Letters, 120(23), 2018. doi: 10.1103/physrevlett.120.231801

  4. [13]

    Measurement of the top quark polarization and t¯t spin correlations using dilepton final states in proton-proton collisions at √s = 13 tev

    CMS Collaboration. Measurement of the top quark polarization and t¯t spin correlations using dilepton final states in proton-proton collisions at √s = 13 tev. Physical Review D, 100(7), 2019. doi: 10.1103/physrevd.100.072002. 6

  5. [14]

    Search for resonant t¯t production in proton-proton collisions at √s = 13 tev

    CMS Collaboration. Search for resonant t¯t production in proton-proton collisions at √s = 13 tev. Journal of High Energy Physics , 2019(4):31, 2019. doi: 10.1007/JHEP04(2019)031

  6. [15]

    Measurement of the t¯t charge asymmetry in events with highly lorentz- boosted top quarks in pp collisions at √s = 13 tev

    CMS Collaboration. Measurement of the t¯t charge asymmetry in events with highly lorentz- boosted top quarks in pp collisions at √s = 13 tev. Physics Letters B , 846, 2023. doi: 10.1016/j.physletb.2023.137703

  7. [16]

    Observation of four top quark production in proton-proton collisions at√s 13 tev

    CMS Collaboration. Observation of four top quark production in proton-proton collisions at√s 13 tev. Physics Letters B, 847, 2023. doi: 10.1016/j.physletb.2023.138290

  8. [17]

    Observation of quantum entanglement in top quark pair production in proton-proton collisions at √s = 13 tev, 2024

    CMS Collaboration. Observation of quantum entanglement in top quark pair production in proton-proton collisions at √s = 13 tev, 2024

  9. [18]

    de Favereau, C

    J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V . Lemaître, A. Mertens, and M. Selvaggi. Delphes 3: a modular framework for fast simulation of a generic collider experiment. Journal of High Energy Physics, 2014(2), 2014. doi: 10.1007/jhep02(2014)057

  10. [19]

    Devlin, M

    J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, 2018

  11. [20]

    Fadin and V

    V . Fadin and V . Khoze. On the threshold behaviour of heavy top production.Zeitschrift für Physik C Particles and Fields, 48(4), 1990. doi: 10.1007/BF01614696

  12. [21]

    Finke, M

    T. Finke, M. Krämer, A. Mück, and J. Tönshoff. Learning the language of qcd jets with transformers. Journal of High Energy Physics , 2023(6), 2023. doi: 10.1007/jhep06(2023)184

  13. [22]

    Frixione, P

    S. Frixione, P. Nason, and C. Oleari. Matching NLO QCD computations with parton shower simulations: the POWHEG method. JHEP, 11:070, 2007. doi: 10.1088/1126-6708/2007/11/070

  14. [23]

    Frixione, G

    S. Frixione, G. Ridolfi, and P. Nason. A positive-weight next-to-leading-order Monte Carlo for heavy flavour hadroproduction. JHEP, 09:126, 2007. doi: 10.1088/1126-6708/2007/09/126

  15. [24]

    B. Fuks, K. Hagiwara, K. Ma, and Y .-J. Zheng. Signatures of toponium formation in lhc run 2 data. Physical Review D, 104(3), 2021. doi: 10.1103/physrevd.104.034023

  16. [25]

    Golling, L

    T. Golling, L. Heinrich, M. Kagan, S. Klein, M. Leigh, M. Osadchy, and J. A. Raine. Masked particle modeling on sets: Towards self-supervised high energy physics foundation models, 2024

  17. [26]

    K. He, H. Fan, Y . Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning, 2020

  18. [27]

    Huang, Y

    A. Huang, Y . Melkani, P. Calafiura, A. Lazar, D. T. Murnane, M.-T. Pham, and X. Ju. A language model for particle tracking, 2024

  19. [28]

    W.-L. Ju, G. Wang, X. Wang, X. Xu, Y . Xu, and L. L. Yang. Top quark pair production near threshold: single/double distributions and mass determination. Journal of High Energy Physics , 2020(6), 2020. doi: 10.1007/jhep06(2020)158

  20. [29]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization, 2017

  21. [30]

    Y . Kiyo, J. H. Kühn, S. Moch, M. Steinhauser, and P. Uwer. Top-quark pair production near threshold at lhc. The European Physical Journal C , 60(3):375, 2009. doi: 10.1140/epjc/ s10052-009-0892-7

  22. [31]

    A. J. Larkoski. Qcd masterclass lectures on jet physics and machine learning, 2024

  23. [32]

    Z. Liu, A. Tieu, N. Patel, A. Zhou, G. Soultanidis, Z. A. Fayad, T. Deyer, and X. Mei. Vision- mae: A foundation model for medical image segmentation and classification, 2024

  24. [33]

    Y . Luo, Z. Yang, F. Meng, Y . Li, J. Zhou, and Y . Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning, 2024

  25. [34]

    P. Nason. A new method for combining NLO QCD with shower Monte Carlo algorithms.JHEP, 11:040, 2004. doi: 10.1088/1126-6708/2004/11/040. 7

  26. [35]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. ...

  27. [36]

    Radford, J

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision, 2022

  28. [37]

    Schick and H

    T. Schick and H. Schütze. Exploiting cloze questions for few-shot text classification and natural language inference. CoRR, 2020

  29. [38]

    Schneider, A

    S. Schneider, A. Baevski, R. Collobert, and M. Auli. wav2vec: Unsupervised pre-training for speech recognition, 2019

  30. [39]

    Shmakov, M

    A. Shmakov, M. J. Fenton, T.-W. Ho, S.-C. Hsu, D. Whiteson, and P. Baldi. Spanet: Generalized permutationless set assignment for particle physics using symmetry preserving attention.SciPost Physics, 12(5), 2022. doi: 10.21468/scipostphys.12.5.178

  31. [40]

    Sjöstrand, S

    T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Rasmussen, and P. Z. Skands. An introduction to pythia 8.2.Computer Physics Communications, 191:159, 2015. doi: 10.1016/j.cpc.2015.01.024

  32. [41]

    Sumino and H

    Y . Sumino and H. Yokoya. Bound-state effects on kinematical distributions of top quarks at hadron colliders. Journal of High Energy Physics , 2010(9), 2010. doi: 10.1007/jhep09(2010) 034

  33. [42]

    Touvron, T

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. Llama: Open and efficient foundation language models, 2023

  34. [43]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. CoRR, 2017. 8

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.