Pith. sign in

REVIEW 5 cited by

Transformer models: an introduction and catalog

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2302.07730 v4 pith:OZVV4CLS submitted 2023-02-12 cs.CL

Xavier Amatriain , Ananth Sankar , Jie Bing , Praveen Kumar Bodigutla , Timothy J. Hazen , Michaeel Kazi This is my paper · ORCID
classification cs.CL
keywords modelstransformercatalogintroductiontrainedappearanceaspectsbert
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

In the past few years we have seen the meteoric appearance of dozens of foundation models of the Transformer family, all of which have memorable and sometimes funny, but not self-explanatory, names. The goal of this paper is to offer a somewhat comprehensive but simple catalog and classification of the most popular Transformer models. The paper also includes an introduction to the most important aspects and innovations in Transformer models. Our catalog will include models that are trained using self-supervised learning (e.g., BERT or GPT3) as well as those that are further trained using a human-in-the-loop (e.g. the InstructGPT model used by ChatGPT).

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Machine Learning for the Cluster Reconstruction in the CALIFA Calorimeter at R3B

    physics.ins-det 2025-06 conditional novelty 6.0 of 10

    An edge-detection neural network combined with agglomerative pre-clustering improves cluster reconstruction accuracy in simulated CALIFA calorimeter events by about 34% relative to the standard R3B algorithm.

  2. Bridging the Evaluation Gap: Leveraging Large Language Models for Topic Model Evaluation

    cs.CL 2025-02 conditional novelty 6.0 of 10

    LLM-based metrics for coherence, repetitiveness, diversity, and topic-document alignment rate topic models, but scores shift substantially depending on which LLM does the judging.

  3. IG-GAN: A Generative Adversarial Network for Aerodynamic Data Generation Based on Intrinsic Geometry

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A Bézier-coefficient generator plus RBF discriminator yields 97% and 83% lower test MSE than SSL-Transformer on Burgers’ velocity and ONERA M6 aerodynamic coefficients.

  4. A Hybrid Framework for Subject Analysis: Integrating Embedding-Based Regression Models with Large Language Models

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Using an ML-predicted label count to constrain LLM generation and post-editing outputs to the LCSH vocabulary lifts subject-heading prediction F1 from 0.135 to 0.300 on a 2,100-book test set.

  5. AttentionSmithy: A Modular Framework for Rapid Transformer Development and Customization

    cs.LG 2025-02 conditional novelty 5.0 of 10

    A modular transformer framework is introduced and validated by replicating the original transformer, running neural architecture search on WMT14 translation, and fine-tuning a BERT-style model for cell type classification.

Pith tools