Pith. sign in

REVIEW 4 major objections 3 minor 28 references

SOI Matters: Analyzing Multi-Setting Training Dynamics in Pretrained Language Models via Subsets of Interest

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multi-source training lifts out-of-distribution accuracy by up to 7 percent.

desk verdict Useful SOI taxonomy and new multi-setting comparisons, but the headline OOD gains are confounded with training-set size and single-run noise. read the letter →

arxiv 2507.15236 v1 pith:SNDHU6XK submitted 2025-07-21 cs.CL cs.LG

Shayan Vassef , Amirhossein Dabiriaghdam , Mohammadreza Bakhtiari , Yadollah Yaghoobzadeh This is my paper · ORCID
classification cs.CLcs.LG
keywords SubsetsofInteresttrainingdynamicsforgettableexamplesmulti-tasklearningmulti-sourcemulti-lingualout-of-distributiongeneralizationtwo-stagefine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that fine-grained, per-example learning trajectories can explain when combining datasets helps or hurts. It introduces Subsets of Interest (SOI), six categories of training examples built from per-epoch correctness: unlearned, always correct, forgettable once, forgettable at least twice, early-learned, and late-learned. Using SOI transition heatmaps, the paper compares single-task, single-source, and single-language fine-tuning against paired multi-setting fine-tuning on pretrained language models. It reports that multi-source training consistently improves out-of-distribution accuracy by up to 7%, while multi-task gains appear mostly for similar task pairs and multilingual transfers are asymmetric. It then shows that a second fine-tuning stage on SOI-selected subsets yields additional out-of-distribution gains, most clearly in the multi-task setting.

What carries the argument

The central object is the Subsets of Interest (SOI) taxonomy, which labels every training example by its per-epoch correctness sequence over ten fine-tuning epochs: Unlearned, Always Correct, one-time forgettable, at-least-twice forgettable, Early-Learned (first correct at or before epoch 5), and Late-Learned. The paper pairs this taxonomy with SOI transition heatmaps, which count how many examples move between categories when a single-setting model is replaced by a multi-setting model, and with dataset cartography, which plots confidence against variability. This machinery turns the coarse question of whether joint training helps into a sample-level account of which examples become easier, harder, or unstable, and the second-stage fine-tuning uses those transitions to pick a training subset.

What would settle it

Re-run the multi-source sentiment experiments (IMDB, Yelp, Sentiment140) with several random seeds per configuration and rebuild the SOI categories and transition heatmaps each time. If category assignments, heatmap transitions, or the reported 3 to 7 percent out-of-distribution gains change materially across seeds, or disappear when the early- versus late-learned boundary is moved from epoch 5, then the claimed characterization of training dynamics and the derived second-stage selection strategy would not hold.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the way individual training examples migrate between six learning-behavior categories under a configuration change is diagnostic of out-of-distribution generalization. Multi-source training, where the same sentiment task is trained on three different data distributions, shifts many examples into more stable, learned categories and consistently improves out-of-distribution accuracy by up to seven points on Sentiment140. Multi-task training only helps when the paired tasks are similar, such as paraphrase and entailment, and harms dissimilar pairs; multi-lingual benefits are not reciprocal, so one language can improve another language's out-of-distribution performance without the reverse holding. A two-stage fine-tuning scheme, where the second stage trains on a subset chosen from the SOI transition heatmap, adds further out-of-distribution gains, especially for multi-task models.

Load-bearing premise

The framework's validity rests on the assumption that one fine-tuning run per configuration gives stable enough per-epoch correctness trajectories to define the SOI categories and transition heatmaps, even though the early- versus late-learned boundary is an arbitrary midpoint of the ten-epoch schedule and no seed variance is reported.

Editorial extensions

If this is right

  • Multi-source fine-tuning on same-task data is a dependable way to improve out-of-distribution accuracy for encoder-based classifiers, with reported gains of 3 to 7 percent over single-source baselines.
  • Similar task pairs such as paraphrase and entailment can produce mutual out-of-distribution gains of 1.8 and 6.9 percent, while dissimilar pairs such as sentiment and paraphrase can degrade both tasks.
  • Cross-lingual benefits are asymmetric: Persian improves French out-of-distribution performance, but French does not reciprocate, so gains cannot be assumed to transfer both ways.
  • Fine-tuning a second stage on SOI-selected subsets, specifically the transition-diagonal strategy, yields further out-of-distribution gains in multi-task settings and can partially recover first-stage losses.
  • SOI categories provide a diagnostic for data quality, because examples that stay unlearned or repeatedly forgettable mark persistent hard regions of the training set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The early-learned versus late-learned boundary at epoch 5 is tied to the 10-epoch schedule; the same taxonomy run under longer or shorter schedules would shift category memberships, so the quantitative claims should be read relative to that schedule.
  • Because all result tables report one run per configuration, seed sensitivity is untested; a multi-seed replication that checks whether SOI heatmaps and out-of-distribution gains are stable would directly test the framework's reliability.
  • The second-stage selector was chosen on multi-task data and applied to the other settings; an extension would be to search per-setting selectors and compare with first-stage-only multi-source training to see whether remaining gains come from selection or simply from extra training epochs.
  • SOI-style trajectory categories could be applied to decoder-style language models and to curriculum learning, where training order is staged by category, both of which the paper lists as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper introduces Subsets of Interest (SOI), a six-category taxonomy of training examples derived from per-epoch correctness trajectories during fine-tuning, and uses it to study multi-task, multi-source, and multi-lingual learning with BERT and XLM-R. The authors present SOI transition heatmaps and dataset cartography visualizations, compare single-setting versus multi-setting fine-tuning on ID and OOD benchmarks, and propose a second-stage fine-tuning strategy that selects examples based on SOI transitions. The main claims are that multi-source learning consistently improves OOD performance by up to 7% and that SOI-guided second-stage fine-tuning yields further gains.

Significance. If the central claims held, the SOI taxonomy would offer a useful diagnostic tool for understanding example-level learning dynamics, and the multi-source result would be practically relevant for improving OOD robustness in classification fine-tuning. The paper also ships code and builds on established ideas such as forgetting events and dataset cartography, which is a reasonable foundation. However, the current evidence is not sufficient to support the headline claims: the multi-source comparison is confounded with training-set size, all results come from a single run per condition, and the second-stage selection rule was chosen on the same OOD benchmarks used for evaluation. The diagnostic value of the SOI framework itself depends on the stability of the categories across seeds and on the arbitrary ELE/LLE epoch threshold, neither of which is tested.

major comments (4)
  1. [Section 3.2.2 / Table 2] The multi-source versus single-source comparison is confounded with training-set size and optimizer steps. Each single-source model is trained on 40,000 examples (Section 3.2.2), while each multi-source pair (IY, SY, IS) is trained on 80,000 examples under the same 10-epoch schedule. The reported OOD gains, e.g., Sentiment140 76.0 to 83.0 in Table 2, are therefore equally consistent with a simple 'more data' effect. The paper does not include any control in which the single-source model sees the same total number of examples or the same number of optimizer steps. Because the abstract and conclusion attribute the gains specifically to multi-source learning, this confound is load-bearing and needs to be addressed experimentally.
  2. [Section 5.2.1 / Tables 1-3] The second-stage fine-tuning set was selected by maximizing average OOD performance on the same multi-task OOD combinations that are later reported as improvements, and the same selected strategy was then applied to the multi-source and multi-lingual setups whose OOD results are also reported in Tables 1 and 2. This selection-on-the-evaluation-set procedure biases the second-stage results in the paper's favor. The authors should use a held-out validation split or a pre-registered rule for choosing method III, and should report the OOD performance of all six candidate strategies on each combination rather than only the selected one.
  3. [Tables 1, 2, and 3] All tables report a single run per configuration with no variance, confidence intervals, or significance tests. Many of the headline differences are small; for example, French-Persian OOD is 52.2 versus 52.2, English-French changes from 51.9 to 51.8, and several second-stage changes are around 0.1-0.5 points. Without multiple seeds, the claims of 'consistent' improvement and of additional second-stage gains are not statistically established. This also affects the SOI categories themselves, since the per-epoch correctness trajectories from a single run are used to define UNE, ACE, ELE, LLE, and the forgettable subsets.
  4. [Section 4.1] The ELE/LLE boundary is defined as 'first correct prediction on or before epoch 5' solely because all training runs use 10 epochs. This threshold is acknowledged to be influenced by the schedule, but no sensitivity analysis is provided. Since ELE/LLE is one of the six SOI categories and is used in the heatmap-based selection strategies, the framework should demonstrate that the categories and the resulting conclusions are robust to reasonable variations of this threshold.
minor comments (3)
  1. [Throughout] There are several formatting and consistency issues: '1t-F RGE' and '≥ 2t-F RGE' appear with an inconsistent space before 'RGE', and the text sometimes writes 'FRGE' without the space; the paper also alternates between 'Persian' and 'Farsi' (e.g., Table 1 caption and Figure 2 caption).
  2. [Appendix B and C] The appendix cross-references are stale: Appendix B says the cartography visualizations complement 'Section 5', but the relevant section is Section 4.2, and Appendix C says Section 6 presents the heatmap analysis, but the heatmap method is introduced in Section 4.3.
  3. [Section 3.2.2] The dataset citations for IMDB, Yelp, and Sentiment140 are non-canonical. The paper should cite the original data sources (e.g., Maas et al. 2011 for IMDB; Zhang et al. 2015 or the Yelp dataset; Go et al. 2009 for Sentiment140) rather than secondary papers that only describe analyses of these datasets.

Circularity Check

1 steps flagged · score 4.0 of 10

Second-stage SOI gains are selected on the target OOD metric; core SOI taxonomy and first-stage comparisons are not circular.

  1. fitted input called prediction [Section 5.2.1 (Heatmap-Based Fine-Tuning Set Selection), evaluated in Section 5.2.2 and Table 3]
    "we experimented with multiple subsampled sets, each defined by a specific heatmap-based criterion, and selected the one that achieved the best average OOD performance across the three multi-task combinations. The selected strategy was then applied to the multi-source and multi-lingual setups as well (see below)."

    The second-stage result is advertised as showing that SOI-guided subset selection achieves 'additional performance improvements.' But the method is chosen by optimizing the exact evaluation metric on the exact evaluation sets later reported in Table 3: after enumerating six strategies, the paper selects the one with the best average OOD on the three multi-task combinations, and then reports those same multi-task OOD numbers as the outcome. The reported gains are therefore not an independent test of SOI; they are the maximum of a small search over the reported target values. A strategy selected on the target cannot serve as evidence that the strategy produces the target outcome.

full rationale

The SOI taxonomy is defined directly from per-epoch correctness trajectories in Section 4.1, and the transition heatmaps merely tabulate movements between those trajectory-defined categories; no result in the taxonomy is derived from the outcome it is later used to explain. The first-stage single-setting versus multi-setting comparisons are external OOD evaluations that do not use SOI categories as inputs, so the multi-source 'up to 7%' result is not circular, although it is confounded by doubled training data in the multi-setting condition, which is a validity threat rather than a circularity under this rubric. The self-citation to Yaghoobzadeh et al. (2021) is motivational and definitional, not load-bearing. The one genuine circular step is localized to the second-stage fine-tuning claim: Section 5.2.1 selects the subset-selection strategy by maximizing average OOD on the exact multi-task combinations whose second-stage improvements are reported in Table 3, making those gains partly selected-on-target rather than independent evidence. The multi-source and multi-lingual second-stage results were not used for selection and show no clear gains, so the issue is contained. Overall score 4 reflects this partial circularity while the central SOI derivation and first-stage comparisons remain self-contained.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on the stability and representativeness of per-epoch correctness trajectories, on the chosen ELE/LLE epoch threshold, and on the validity of the OOD benchmarks (train-set subsets and NLLB translations). No new physical entities are introduced; SOI is a definitional taxonomy.

free parameters (3)
  • ELE/LLE epoch threshold = 5 (of 10 epochs)
    Defined in Section 4.1: examples first correct at or before epoch 5 are Early-Learned, after epoch 5 are Late-Learned; this arbitrary midpoint directly changes the composition of ELE and LLE.
  • Second-stage selection rule (method III) = Diagonal transition entries excluding ACE->ACE
    Selected in Section 5.2.1 as the rule with best average OOD on the multi-task combinations; this is a choice fitted to the evaluation metric, not a pre-registered rule.
  • Fine-tuning schedule = 10 epochs (stage 1), 4 epochs (stage 2)
    Chosen by hand without sensitivity analysis; SOI category definitions and ELE/LLE split depend on the 10-epoch stage-1 schedule.
assumptions (3)
  • domain assumption Per-epoch prediction correctness trajectories are a stable and meaningful measure of example-level learning behavior.
    The entire SOI taxonomy (Section 4.1) is built on this assumption, inherited from cartography and forgetting-event literature.
  • domain assumption Translated Burmese versions of intent datasets are valid OOD proxies for cross-lingual generalization.
    Section 3.2.3 uses NLLB-200 translations of English, French, and Persian intent data as OOD; translation artifacts could dominate the measured transfer.
  • domain assumption Using parts of existing training sets (RTE train, QQP train, SST-2 subsets) as OOD test sets gives representative out-of-distribution measurements.
    Section 3.2.1 defines OOD evaluation this way; these are train-set subsets from different benchmarks, not held-out test sets from the same distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SOI Matters: Analyzing Multi-Setting Training Dynamics in Pretrained Language Models via Subsets of Interest." pith.science (2026). https://pith.science/paper/SNDHU6XK

@misc{pith2026250715236,
  author       = {Pith},
  title        = {Pith review of: SOI Matters: Analyzing Multi-Setting Training Dynamics in Pretrained Language Models via Subsets of Interest},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SNDHU6XK}},
  note         = {Machine review of arXiv:2507.15236}
}
read the original abstract

This work investigates the impact of multi-task, multi-lingual, and multi-source learning approaches on the robustness and performance of pretrained language models. To enhance this analysis, we introduce Subsets of Interest (SOI), a novel categorization framework that identifies six distinct learning behavior patterns during training, including forgettable examples, unlearned examples, and always correct examples. Through SOI transition heatmaps and dataset cartography visualization, we analyze how examples shift between these categories when transitioning from single-setting to multi-setting configurations. We perform comprehensive experiments across three parallel comparisons: multi-task vs. single-task learning using English tasks (entailment, paraphrase, sentiment), multi-source vs. single-source learning using sentiment analysis datasets, and multi-lingual vs. single-lingual learning using intent classification in French, English, and Persian. Our results demonstrate that multi-source learning consistently improves out-of-distribution performance by up to 7%, while multi-task learning shows mixed results with notable gains in similar task combinations. We further introduce a two-stage fine-tuning approach where the second stage leverages SOI-based subset selection to achieve additional performance improvements. These findings provide new insights into training dynamics and offer practical approaches for optimizing multi-setting language model performance.

Figures

Figures reproduced from arXiv: 2507.15236 by the authors.

Figure 1
Figure 1. Unified Architecture for Our Multi-Setting [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Dataset cartography Map for Single-Lingual [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Single-Task Learning (ST) cartography showing the distribution of examples for Entailment, Sentiment, [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Single-Source Learning (SS) cartography showing the distribution of examples for IMDB, Sentiment140, [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Single-Lingual Learning (SL) cartography showing the distribution of examples for English, French, and [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Multi-Task Learning (MT) transition heatmaps showing SOI transitions for all task combinations. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Multi-Source Learning (MS) transition heatmaps showing SOI transitions for all source combinations. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Multi-Lingual Learning (ML) transition heatmaps showing SOI transitions for all language combinations. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 9 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ronan Collobert and Jason Weston. 2008. https://doi.org/10.1145/1390156.1390177 A unified architecture for natural language processing: Deep neural networks with multitask learning . In Proceedings of the 25th International Conference on Machine Learning, ICML '08, page 160–167, New York, NY, USA. Association for Computing Machinery

    arXiv 2008

  4. [4]

    Pierre Colombo, Emile Chapuis, Matthieu Labeau, and Chlo \'e Clavel. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.656 Code-switched inspired losses for spoken dialog representations . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8320--8337, Online and Punta Cana, Dominican Republic. Association for ...

    work page doi:10.18653/v1/2021.emnlp-main.656 2021

  5. [5]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning at scale . In Proceedings of the 58th Annual Meeting of the Association for Comp...

    doi:10.18653/v1/2020.acl-main.747 2020

  6. [6]

    Toutanova

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina N. Toutanova. 2018. https://arxiv.org/abs/1810.04805 Bert: Pre-training of deep bidirectional transformers for language understanding

    arXiv 2018

  7. [7]

    Dolan and Chris Brockett

    William B. Dolan and Chris Brockett. 2005. https://aclanthology.org/I05-5002 Automatically constructing a corpus of sentential paraphrases . In Proceedings of the Third International Workshop on Paraphrasing ( IWP 2005)

    work page 2005

  8. [8]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

Show all 28 references
  1. [9]

    Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, and Prem Natarajan. 2023. https://doi.org/10.18653/v1/2...

    2023 doi

  2. [10]

    Mohammadreza Ghaffarzadeh-Esfahani, Mahdi Ghaffarzadeh-Esfahani, Arian Salahi-Niri, Hossein Toreyhi, Zahra Atf, Amirali Mohsenzadeh-Kermani, Mahshad Sarikhani, Zohreh Tajabadi, Fatemeh Shojaeian, Mohammad Hassan Bagheri, et al. 2024. Large language models versus classical mach...

    2024 arXiv

  3. [11]

    Mohammad W Habib and Zainab N Sultani. 2021. Twitter sentiment analysis using different machine learning and feature extraction techniques. Al-Nahrain Journal of Science, 24(3):50--54

    2021

  4. [12]

    S Hemalatha and Ramathmika Ramathmika. 2019. Sentiment analysis of yelp reviews by machine learning. In 2019 International Conference on Intelligent Computing and Control Systems (ICCS), pages 700--704. IEEE

    2019

  5. [13]

    Jeremy Howard and Sebastian Ruder. 2018. https://doi.org/10.18653/v1/P18-1031 Universal language model fine-tuning for text classification . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328--339, Melb...

    2018 doi

  6. [14]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  7. [15]

    Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. https://doi.org/10.1609/aaai.v32i1.12022 Scitail: A textual entailment dataset from science question answering . Proceedings of the AAAI Conference on Artificial Intelligence, 32(1)

    2018 doi

  8. [16]

    Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K

    Stefan Larson, Anish Mahendran, Joseph J. Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K. Kummerfeld, Kevin Leach, Michael A. Laurenzano, Lingjia Tang, and Jason Mars. 2019. https://www.aclweb.org/anthology/D19-1131 An evaluation dataset for intent classificati...

    2019

  9. [17]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. http://arxiv.org/abs/1907.11692 Roberta: A robustly optimized bert pretraining approach

    2019 arXiv

  10. [18]

    NLLB Team , Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia-Gonzalez, Pran...

    2022

  11. [19]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  12. [20]

    Ankita Rane and Anand Kumar. 2018. https://doi.org/10.1109/COMPSAC.2018.00114 Sentiment classification system of twitter data for us airline service analysis . In 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC), volume 01, pages 769--773

    2018

  13. [21]

    Rizwanul Islam Rudra and Anilkumar Kothalil Gopalakrishnan. 2023. Sentiment analysis of consumer reviews using machine learning approach. In 2023 20th International Joint Conference on Computer Science and Software Engineering (JCSSE), pages 49--54. IEEE

    2023

  14. [22]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. https://www.aclweb.org/anthology/D13-1170 Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 Conferenc...

    2013

  15. [23]

    Smith, and Yejin Choi

    Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.746 Dataset cartography: Mapping and diagnosing datasets with training dynamics . In Proceedings of the 2020 C...

    2020 doi

  16. [24]

    Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. 2019. https://openreview.net/forum?id=BJlxm30cKm An empirical study of example forgetting during deep neural network learning . In International Conference on Lear...

    2019

  17. [25]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  18. [26]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461

    2018 arXiv

  19. [27]

    Yadollah Yaghoobzadeh, Soroush Mehri, Remi Tachet des Combes, T. J. Hazen, and Alessandro Sordoni. 2021. https://doi.org/10.18653/v1/2021.eacl-main.291 Increasing robustness to spurious correlations using forgettable examples . In Proceedings of the 16th Conference of the Euro...

    2021 doi

  20. [28]

    Hao Yu, Zachary Yang, Kellin Pelrine, Jean Francois Godbout, and Reihaneh Rabbany. 2023. Open, closed, or small language models for text classification? arXiv preprint arXiv:2308.10092

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.