Pith. sign in

REVIEW 2 cited by

Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2110.01518 v1 pith:HISKM452 submitted 2021-10-04 cs.CL

Prajjwal Bhargava , Aleksandr Drozd , Anna Rogers This is my paper · ORCID
classification cs.CL
keywords generalizationheuristicsmodelsadaptersadversariallyarchitecturesbert-basedbeyond
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cross-Lingual Pitfalls: Automatic Probing Cross-Lingual Weakness of Multilingual Large Language Models

    cs.CL 2025-05 conditional novelty 6.0 of 10

    An adversarial beam-search method generates over 6,000 bilingual question pairs that reliably make multilingual LLMs perform far worse in non-English languages than in English.

  2. Team DACTYL at PAN 2026: Bayesian Data Mixing and Empirical X-risk Minimization for AI-text Detection

    cs.CL 2026-07 conditional novelty 5.0 of 10

    A Bayesian-uncertainty text filter, partial-AUROC training, and MCGrad calibration produce the second-ranked AI-text detector (0.974 mean score) on the PAN 2026 leaderboard.

Pith tools