Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

Towards Debiasing Fact Verification Models

T0 review · 5 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FEVER claims leak their labels, and a symmetric test set proves it.

desk verdict FEVER bias finding is real and worth citing, but the symmetric test set needs a claim-only baseline before its measured drop can be read as proof of unbiasedness. read the letter →

arxiv 1908.05267 v2 pith:4NUPXHJR submitted 2019-08-14 cs.CL

Tal Schuster , Darsh J Shah , Yun Jie Serene Yeo , Daniel Filizzola , Enrico Santus , Regina Barzilay This is my paper · ORCID
classification cs.CL
keywords factverificationdatasetbiasannotationartifactsFEVERsymmetrictestsetreweightingregularizationclaim-onlyclassificationnaturallanguageinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the FEVER fact-verification benchmark is easier than it looks: crowdworker editing habits left giveaway phrasing in claims, so a classifier that reads only the claim reaches 61.7% accuracy against a 33.3% majority baseline. The giveaway is concentrated in negation bigrams like “did not,” “failed to,” and “unable to,” which are strongly associated with the REFUTES label. To force models to use the evidence, the authors build a symmetric test set in which every n-gram is equally likely under either label; on it, evidence-aware FEVER-trained models fall from roughly 81–86% to 56–59%. They then introduce a model-agnostic reweighting scheme that flattens n-gram–label correlations and recovers 3.3–3.4 absolute points on the symmetric set. If this is right, published FEVER scores overstate how much reasoning the models actually do.

What carries the argument

Two mechanisms carry the argument. The first is the symmetric test set: for each original claim-evidence pair, the authors manually generate a counterpart with the same SUPPORTS or REFUTES label but a contrary fact, and then create two cross pairs by swapping claims with evidence from the other pair. The resulting four combinations make $p(\mathrm{label}\mid w)=1/2$ for every n-gram in either input, so claim-only guessing is a coin flip. The second is a reweighting regularization in which Local Mutual Information $\operatorname{LMI}(w,l)=p(w,l)\log(p(l\mid w)/p(l))$ identifies giveaway n-grams, and weights $\alpha$ are optimized to minimize total n-gram-label bias across the training claims; training loss for instance i is then scaled by $1+\alpha^{(i)}$, which is architecture-independent.

What would settle it

Re-annotate all 956 symmetric pairs with fresh annotators: if agreement falls well below 94%, or if crossing original and generated sentences fails to invert the label in many cases, the symmetry property is broken. A complementary check is to train a claim-only classifier on reweighted data and see whether it still beats 50% on the symmetric set; if it does, unmeasured cues remain.

Watch

Extended reading notes

Core claim

The paper’s central claim is that FEVER’s labels are partially predictable from the claim alone because of annotation artifacts, not because of contextual reasoning or even world knowledge. The evidence is threefold: a claim-only BERT beats the majority baseline by a wide margin; high-LMI bigrams in claims, especially negations, predict REFUTES with probabilities around 0.8–0.9 in both training and development; and the same models drop dramatically on a symmetric test set where $p(\mathrm{label}\mid w)=1/2$ by construction. The paper further claims that its reweighting regularization—scaling each training example’s loss by a weight that makes giveaway n-grams label-uniform—improves symmetric-set accuracy of ESIM and BERT by 3.4 and 3.3 absolute points, with a small cost on the original biased dev set.

Load-bearing premise

The symmetric test set is a valid unbiased instrument, which requires that every generated and crossed claim-evidence pair is fluent, factual, correctly labeled, and genuinely inverse when crossed; the authors checked only 30% of the 956 pairs (94% agreement, 2% grammar issues), and the source pairs were chosen only from cases NSMN already predicted correctly.

Editorial extensions

If this is right

  • FEVER leaderboard scores on the original benchmark overstate evidence-based reasoning; the symmetric set offers a cleaner lower-bound test and should be added alongside the retrieval-based evaluation.
  • The reweighting method is independent of model architecture, so it can be dropped into any classification objective where training data carry annotation artifacts.
  • Debiased training trades a small amount of accuracy on the biased dev set for robustness on the unbiased set, so gains should be reported on both.
  • A model that performs well on the symmetric set is using the evidence rather than the surface form of the claim, which is what fact verification is supposed to measure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Likely consequence: other fact-verification and NLI benchmarks built by similar crowdworker editing may carry the same giveaway-phrase bias, and the paper’s symmetric pairing is a cheap diagnostic for detecting it before expensive training.
  • The reweighting flattens only n-gram-level cues; semantic or world-knowledge shortcuts that do not reduce to surface bigrams would survive it, so combining reweighting with counterfactual data augmentation or adversarial filtering is a natural follow-up.
  • Because only 30% of the symmetric pairs were independently validated, the headline drop estimates carry annotation noise; full re-annotation or a larger symmetric set would tighten the numbers.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper studies annotation bias in the FEVER fact-verification dataset. The authors train a claim-only BERT classifier and obtain 61.7% accuracy versus a 33.3% majority baseline, and use LMI-based n-gram analysis to show that REFUTES-associated bigrams such as 'did not' and 'refused to' are stable across train and dev. They construct a symmetric test set by manually generating counterfactual claim-evidence pairs for 239 original pairs, and report that evidence-aware models drop from roughly 81-86% on FEVER dev to 56-59% on the new set. They then propose an instance-reweighting regularizer (Eq. 3) that flattens n-gram-label correlations, and report absolute gains of 3.4 and 3.3 points for ESIM and BERT on the symmetric set. The paper concludes that FEVER models exploit claim-side cues and that the proposed reweighting improves evidence-based verification.

Significance. The bias diagnosis is a solid and valuable contribution: the claim-only BERT baseline, the LMI analysis, and the consistent p(l|w) patterns in train and dev are reproducible evidence of annotation artifacts. The proposed symmetric test set and the model-agnostic reweighting objective are useful ideas and could become community resources. The paper is also clearly written and ships code and data via a public repository. However, the experimental support for the central claims is incomplete: the symmetric test set's validity is asserted rather than verified, the source pairs are selected using the very model whose bias is being studied, and the reweighting gains are measured on a set constructed from the same n-gram marginal criterion that the objective minimizes. These issues are fixable, but they are load-bearing for the paper's evaluation claims.

major comments (5)
  1. [Section 3, Table 3] The central claim that the symmetric test set makes claim-only prediction equivalent to random guessing is asserted but never tested. The construction guarantees only marginal independence: for any n-gram appearing in the data, p(label|n-gram)=0.5 by construction. A trained neural classifier can exploit longer-range lexical patterns, syntax, sentence length, punctuation, and stylistic regularities introduced by the manual generation process, none of which are controlled by the construction. The paper reports no claim-only BERT or InferSent accuracy on the symmetric set, so the drop from 81.8% to 58.7% for NSMN in Table 3 is not shown to be caused by bias removal rather than distribution shift. Please add this control: a claim-only model should be at or near chance on the symmetric set; if it is not, the performance drop cannot be attributed to elimination of claim-side cues.
  2. [Section 5, footnote 7, Table 3] The source pairs for the symmetric set were 'randomly picked from the cases which NSMN correctly predicts.' This introduces selection bias: the symmetric set is generated from an easier, model-agreeing subset of FEVER, while the FEVER DEV column in Table 3 reports accuracy on the full dev set. The 81.8% versus 58.7% comparison therefore conflates distribution shift with removal of claim-side bias. Please report accuracy on the matched 239 original source pairs, and either draw source pairs independently of model correctness or demonstrate that the selected subset is representative of the dev distribution.
  3. [Section 4, Eq. (3), Table 4] The reweighting objective minimizes exactly the per-n-gram label bias that the symmetric test set is designed to neutralize, and Table 4 confirms that the objective is satisfied (reweighted p(l|w) around 0.33-0.35 for the top bigrams). The 3.3- and 3.4-point gains in Table 3 therefore partly measure how well the method satisfies its own objective rather than a general improvement in evidence-based verification. To support the external claim, please validate on an unbiased set constructed independently of the n-gram marginal criterion, and compare with simpler debiasing baselines such as removing or downweighting high-LMI n-grams, or training on n-gram-balanced subsets.
  4. [Section 5, Table 3] All reported accuracies are single runs on 956 pairs with no confidence intervals or significance tests. The absolute gains of 3.3 and 3.4 points correspond to roughly 32 examples in a binary split, which could be within run-to-run noise for neural models. Please report means and standard deviations over multiple random seeds, and use a paired test such as McNemar's for the BASE versus R.W comparisons.
  5. [Section 5, Symmetric Test Set] Only 285 of 956 pairs (30%) were annotated, with 94% agreement and 2% grammatical errors. This checks label agreement, but not the stronger property that every generated claim is factually coherent and that every cross pair carries the inverse label. Since the entire argument depends on the correctness of the unannotated pairs, please either release full annotations, enlarge the annotated sample, or at minimum report annotation results separately for the generated and cross pairs, and examine whether annotation errors correlate with model errors.
minor comments (5)
  1. [Section 3] The sentence 'This new test set completely eliminates the ability of models to rely on cues from claims' overstates what is shown; the construction eliminates the specific n-gram-level label correlations identified in Section 2, not all possible claim-only cues. Please soften the wording to match what is proven.
  2. [Table 3] The column heading 'GENERATED' is ambiguous: the text sometimes refers to the full SYMMETRIC TEST SET (956 pairs) and sometimes to the generated pairs only. Please define exactly which subset is scored in each column and give the number of examples.
  3. [Section 4, Eq. (3)] The optimization procedure for Eq. (3) is not described; please state how the objective is solved, including the optimizer, the range of the regularization coefficient lambda, and any convergence criteria, so that the preprocessing step is reproducible.
  4. [Section 4, Eq. (2)] The indicator I[w_j^(i)] is used for n-gram presence, but if a claim contains w_j multiple times the counting convention is ambiguous; please clarify whether this is a binary presence indicator or an occurrence count.
  5. [Section 1] The description of NSMN as 'the leading evidence-aware system in the FEVER Shared Task' should be qualified with the year and, if possible, a citation to the shared task report.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the bias analysis, symmetric test construction, and reweighting evaluation are empirically separate steps.

full rationale

The paper's central findings are not circular. The claim-only BERT result (61.7%) and the n-gram LMI analysis in Section 2 are direct empirical observations on FEVER, independent of the paper's later constructions. The symmetric test set is constructed to enforce p(l|w) = 0.5 for individual n-grams, and the reported drops (e.g., NSMN 81.8% to 58.7%) are measured accuracies on that new set, not consequences of the construction by definition. The reweighting method in Eq. 3 is fit on the FEVER training set only, and the reported gains on the symmetric test set (ESIM +3.4, BERT +3.3) are held-out evaluations against a separate instrument; the objective is not the evaluation metric, so improvement is not forced by construction. The paper does cite prior work, but no load-bearing argument reduces to a self-citation chain or to an imported uniqueness theorem. The assertion that claim-only classification in the symmetric set is 'equivalent to a random guess' is a validity concern, because marginal n-gram balance does not guarantee that a trained neural classifier cannot exploit longer-range or stylistic cues; however, this is an unsupported or incomplete inference, not a circular reduction. Similarly, the selection of source pairs from NSMN-correct cases and the partial 285-pair annotation bear on how representative the symmetric set is, not on whether the paper's derivation assumes its own conclusion. Overall, the empirical claims are self-contained and the evaluation is not definitionally tied to the training objective in a way that would make the results circular.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

No invented entities are introduced. The load-bearing assumptions are the validity of the symmetric test set and the preservation of evidence signal under reweighting; both are stated but not fully verified. The optimization in Eq. 3 also depends on unreported choices for lambda and the n-gram vocabulary.

free parameters (3)
  • lambda regularization coefficient = Not reported.
    Eq. 3 balances the max over classes of per-n-gram bias against the L2 norm of instance weights; the paper never states its value, and the reported gains may depend on it.
  • n-gram vocabulary V and n-gram range = Not reported.
    The reweighting objective sums over all n-grams in claims, but the paper does not specify unigrams, bigrams, or higher orders, nor how V is constructed. This choice controls which biases are flattened.
  • Instance weights alpha(i) = Optimized via Eq. 3 on FEVER training set; values not reported.
    Eq. 4 uses (1+alpha(i)) to reweight each loss. The fitted weights are the method's output, but their values and sensitivity are not reported, so the stability of the 3.3-3.4 point gains cannot be assessed.
assumptions (2)
  • domain assumption Manually generated symmetric pairs and their cross pairs are fluent, factual, and correctly labeled.
    Section 3 and Figure 1 describe the construction; only 285 of 956 pairs were annotated (94% agreement, 2% grammatical errors), so the rest are assumed valid.
  • domain assumption Reweighting by claim n-gram content preserves the evidence-label signal needed for fact verification.
    Eq. 4 reweights the loss using claim content only, and Table 3 shows a small FEVER-dev accuracy loss; the paper does not prove that evidence reasoning is retained.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Debiasing Fact Verification Models." pith.science (2026). https://pith.science/paper/4NUPXHJR

@misc{pith2026190805267,
  author       = {Pith},
  title        = {Pith review of: Towards Debiasing Fact Verification Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4NUPXHJR}},
  note         = {Machine review of arXiv:1908.05267}
}
read the original abstract

Fact verification requires validating a claim in the context of evidence. We show, however, that in the popular FEVER dataset this might not necessarily be the case. Claim-only classifiers perform competitively with top evidence-aware models. In this paper, we investigate the cause of this phenomenon, identifying strong cues for predicting labels solely based on the claim, without considering any evidence. We create an evaluation set that avoids those idiosyncrasies. The performance of FEVER-trained models significantly drops when evaluated on this test set. Therefore, we introduce a regularization method which alleviates the effect of bias in the training data, obtaining improvements on the newly created test set. This work is a step towards a more sound evaluation of reasoning capabilities in fact verification models.

Figures

Figures reproduced from arXiv: 1908.05267 by the authors.

Figure 1
Figure 1. An illustration of a REFUTES claim-evidence pair from the FEVER dataset (A) that is used to gener￾ate a new pair (B). From the combination of the ORIGI￾NAL and manually GENERATED pairs, we obtain a total of four pairs creating symmetry. through common sense knowledge), contextual verification is key for a fact-checking task (Al￾hindi et al., 2018). Datasets should ideally eval￾uate this ability. To assess whether th… view at source ↗
Figure 2
Figure 2. Pearson r scores of p(l|w) for the top LMI￾ranked bigrams in the train and development sets. We calculated the Pearson correlation score be￾tween p(l|w) for both train and development set [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Percentage of claims containing at least one [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Riemannian Geometry for Pre-trained Language Model Embeddings

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Aggregating per-token pullback metrics via the Fréchet mean on the SPD manifold outperforms Euclidean mean pooling for sentence classification, with most of the gain attributable to geometric aggregation rather than l...

  2. HF-RAG: Hierarchical Fusion-based RAG with Multiple Sources and Rankers

    cs.IR 2025-09 conditional novelty 4.0 of 10

    By first fusing multiple retrievers within labeled and unlabeled sources with RRF, then merging z-score normalized lists, HF-RAG improves fact-verification F1 in-domain and out-of-domain.

Reference graph

Works this paper leans on

25 extracted references · 11 canonical work pages · cited by 2 Pith papers

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Tariq Alhindi, Savvas Petridis, and Smaranda Muresan. 2018. https://doi.org/10.18653/v1/W18-5513 Where is your evidence: Improving fact-checking by justification modeling . In Proceedings of the First Workshop on Fact Extraction and VER ification ( FEVER ) , pages 85--90, Brussels, Belgium. Association for Computational Linguistics

    doi:10.18653/v1/w18-5513 2018

  4. [4]

    Bowman, Gabor Angeli, Christopher Potts, and Christopher D

    Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. https://doi.org/10.18653/v1/D15-1075 A large annotated corpus for learning natural language inference . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632--642. Association for Computational Linguistics

    doi:10.18653/v1/d15-1075 2015

  5. [5]

    Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. http://www.aclweb.org/anthology/P17-1152 Enhanced lstm for natural language inference . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1657--1668

    work page 2017

  6. [6]

    Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37--46

    1960

  7. [7]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

    doi:10.18653/v1/n19-1423 2019

  8. [8]

    Stefan Evert. 2005. The statistics of word cooccurrences: word pairs and collocations

    work page 2005

Show all 25 references
  1. [9]

    Liu, Matthew Peters, Michael Schmitz, and Luke S

    Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017. http://arxiv.org/abs/arXiv:1803.07640 Allennlp: A deep semantic natural language processing platform

    2017 arXiv

  2. [10]

    Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. https://doi.org/10.18653/v1/N18-2017 Annotation artifacts in natural language inference data . In Proceedings of the 2018 Conference of the North American Chapter of the Ass...

    2018 doi

  3. [11]

    Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2016. https://arxiv.org/abs/1511.02301 The goldilocks principle: Reading children's books with explicit memory representations . In International Conference on Learning Representations 2016

    2016 arXiv

  4. [12]

    Robin Jia and Percy Liang. 2017. https://doi.org/10.18653/v1/D17-1215 Adversarial examples for evaluating reading comprehension systems . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2021--2031. Association for Computational ...

    2017 doi

  5. [13]

    Heinrich Jiang and Ofir Nachum. 2019. https://arxiv.org/abs/1901.04966 Identifying and correcting label bias in machine learning . arXiv preprint arXiv:1901.04966

    2019 arXiv

  6. [14]

    Divyansh Kaushik and Zachary C. Lipton. 2018. http://aclweb.org/anthology/D18-1546 How much reading does reading comprehension require? a critical investigation of popular benchmarks . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, p...

    2018

  7. [15]

    Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. https://www.aclweb.org/anthology/P19-1334 Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pa...

    2019

  8. [16]

    Yixin Nie, Haonan Chen, and Mohit Bansal. 2019. https://arxiv.org/abs/1811.07039 Combining fact extraction and verification with neural semantic matching networks . In Association for the Advancement of Artificial Intelligence

    2019 arXiv

  9. [17]

    Timothy Niven and Hung-Yu Kao. 2019. https://www.aclweb.org/anthology/P19-1459 Probing neural network comprehension of natural language arguments . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4658--4664, Florence, Italy. As...

    2019

  10. [18]

    McAllester

    Takeshi Onishi, Hai Wang, Mohit Bansal, Kevin Gimpel, and David A. McAllester. 2016. https://aclweb.org/anthology/D16-1241 Who did what: A large-scale person-centered cloze dataset . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pa...

    2016

  11. [19]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. http://www.aclweb.org/anthology/D14-1162 Glove: Global vectors for word representation . In Proceedings of the 2014 conference on empirical methods in natural language processing, pages 1532--1543

    2014

  12. [20]

    Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. http://www.aclweb.org/anthology/N18-1202 Deep contextualized word representations . In Proceedings of the 2018 Conference of the North American Chapter of the As...

    2018

  13. [21]

    Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. https://doi.org/10.18653/v1/S18-2023 Hypothesis only baselines in natural language inference . In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics,...

    2018 doi

  14. [22]

    Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. http://aclweb.org/anthology/P18-2124 Know what you don't know: Unanswerable questions for squad . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784-...

    2018

  15. [23]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. https://doi.org/10.18653/v1/N19-1421 C ommonsense QA : A question answering challenge targeting commonsense knowledge . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associa...

    2019 doi

  16. [24]

    James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. http://www.aclweb.org/anthology/N18-1074 Fever: a large-scale dataset for fact extraction and verification . In Proceedings of the 2018 Conference of the North American Chapter of the Associati...

    2018

  17. [25]

    Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merri \"e nboer, Armand Joulin, and Tomas Mikolov. 2016. https://arxiv.org/abs/1502.05698 Towards ai-complete question answering: A set of prerequisite toy tasks . In International Conference on Learning Re...

    2016 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.