Pith. sign in

REVIEW 4 major objections 5 minor 163 references

Decomposed Entailment for Factuality Checking and Hallucination Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read HallDetect shows that claim-level decomposition with contrastive multi-scale NLI detects hallucinations better than quantized generative judges on three of four benchmarks.

desk verdict An honest, well-scoped frugal hallucination detector with a real design contribution; the single-run evaluation and missing MiniCheck comparison are the main gaps, but the core claim holds within its stated scope. read the letter →

arxiv 2608.05823 v1 pith:7DREMN56 submitted 2026-08-06 cs.CL

Achir Oukelmoun , Nasredine Semmar , Gaël De Chalendar This is my paper
classification cs.CL
keywords hallucinationdetectiondecomposition-basedfactualitynaturallanguageinferenceclaimverificationmulti-scalechunkingquantizationfaithfulnessevaluationaudittrail
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents HallDetect, a hallucination-detection framework that checks generated text claim by claim rather than as a single block. It tries to show that on consumer-grade hardware, with all language-model components quantized to 4-bit precision, this claim-level route with a compact entailment verifier beats generic generative judges and embedding similarity baselines on three of four benchmarks, and that it stays stable across three backbone families. The authors also argue the method is audit-friendly: each atomic claim is paired with the source span that best supports or best contradicts it. The central case is that localized contradictions get diluted by whole-document checks, and that decomposing the response and searching source chunks at multiple scales recovers them.

What carries the argument

The load-bearing object is the contrastive per-claim NLI score $$ \operatorname{Score}(c_i)=\max_{k\in\mathcal K}P_{\mathrm{NLI}}(E\mid k,c_i)-\max_{k\in\mathcal K}P_{\mathrm{NLI}}(C\mid k,c_i), $$ where $\mathcal K$ is the union of chunks from five dyadic source partitions ($m\in\{1,2,4,8,16\}$ sentence groups, at most 31 chunks), and the best entailment and best contradiction chunks are chosen independently. The final $F_{ED}$ score aggregates the per-claim scores with a geometric mean, with the edge case that any $\operatorname{Score}(c_i)\le0$ sets the mean to 0 and flags the response. This enforces an asymmetric, auditing-oriented rule: one confidently contradicted claim is enough to mark the whole response as hallucinated, while the geometric form keeps a graded signal over the non-contradicted regime.

What would settle it

A probe set of long, anaphora-heavy news summaries whose pronoun antecedents lie in different sentence groups, scored with HallDetect and compared against human labels, would settle the coreference risk: if recall on those documents falls materially below matched documents with explicit noun repetition, the chunking premise breaks. Separately, a rerun under the same 4-bit protocol that fails to reproduce HallDetect's recall advantage over SelfCheckGPT on QAGS-CNN/DM would refute the headline comparison.

Watch

Extended reading notes

Core claim

HallDetect's central discovery is that a fixed, non-autoregressive NLI model (DeBERTa-v3-Large, 435M parameters) can serve as a stable verifier for hallucination detection when paired with generative atomic-claim extraction, provided the source is represented as a multi-scale library of chunks. For each claim, the verifier independently selects the chunk that maximizes entailment probability and the chunk that maximizes contradiction probability, then aggregates the per-claim scores so that one confidently contradicted claim flags the entire response. The claim is not that the method beats full-precision state-of-the-art detectors; it is that under a matched 4-bit budget the decomposition-plus-multiscale combination recovers recall that holistic NLI loses, and that replacing generative reasoning with a discriminative encoder removes the volatility that quantization induces in chain-of-thought judges.

Load-bearing premise

The multi-scale chunking of the source into $m\in\{1,2,4,8,16\}$ groups must preserve the pronoun references and cross-sentence links that a claim needs to be judged; the paper itself notes that pronouns severed from distant antecedents by chunking propagate as spurious contradictions.

Editorial extensions

If this is right

  • Under the paper's protocol, a single consumer GPU can run a full hallucination audit because the source document never enters a generative context; only the short response is decoded once.
  • The recall-oriented aggregation means that one confidently contradicted claim is sufficient to flag a response, matching an auditing workflow where a missed hallucination costs more than a false alarm.
  • The claim-to-span audit trail lets a human reviewer inspect exactly which sentence in the source supports or contradicts each generated claim, instead of re-reading the whole document.
  • Stability across Mistral, Gemma, and Llama under 4-bit quantization suggests that the discriminative verifier shields judgments from the artifacts that make chain-of-thought judges volatile.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the extraction prompt is copy-faithful, extracted claims can carry pronouns whose antecedents live outside every chunk; if the NLI model cannot resolve them, false contradictions would accumulate exactly in anaphora-heavy genres such as dialogue, and a targeted probe comparing pronoun-rich and pronoun-free extracts would test this.
  • The dyadic schedule caps the chunk library at 31 chunks regardless of document length, so verification cost scales with the number of claims rather than document length, which suggests the approach could extend to very long sources without re-engineering the chunk library.
  • The paper stops short of comparing against MiniCheck, a similarly compact discriminative fact-checker; a head-to-head under the same 4-bit protocol would show whether the advantage comes from the multi-scale contrastive scoring itself or from the specific verifier choice.
  • Because all reported F1 scores use a fixed 0.5 threshold, a practical deployment would likely need per-domain threshold calibration to balance the deliberately recall-heavy operating point.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents HallDetect, a decomposition-based hallucination detection framework. A 4-bit quantized LLM extracts atomic claims from a generated response; each claim is verified against a multi-scale chunk library of the source document using a single DeBERTa-v3 NLI model; per-claim contrastive scores are aggregated via an asymmetric geometric mean that flags the response if any claim is confidently contradicted. The framework is evaluated on four benchmarks (QAGS-CNN/DM, TofuEval, FalseSum, PubMedQA) under a matched 4-bit consumer-hardware protocol, with precision, recall, and F1 reported at a fixed 0.5 threshold. The paper claims that HallDetect outperforms comparably resourced generative and embedding-based baselines on three of four benchmarks, remains stable across three backbone families (Mistral, Llama, Gemma), and provides a claim-to-span audit trail.

Significance. If the empirical claims hold, HallDetect is a useful, resource-efficient, and auditable alternative to LLM-as-a-judge for intrinsic faithfulness checking. The design is transparent: no parameter is fitted to evaluation labels, the threshold is fixed a priori, and the verification stage uses a compact encoder that runs on consumer hardware. The paper also ships an anonymous code repository, which aids reproducibility. However, the significance is tempered by the lack of statistical grounding in the main results, the absence of comparisons to specialized discriminative fact-checkers such as MiniCheck, and unresolved questions about whether the recall gains reflect genuine contradiction detection or artifacts of multi-scale chunking.

major comments (4)
  1. [Section 6, Tables 1-4; Section 9] All reported results are single deterministic runs with no error bars, confidence intervals, or significance tests. Several decisive comparisons are close (e.g., PubMedQA Gemma: HallDetect F1 0.632 vs SelfCheck 0.668; QAGS-CNN/DM Mistral: 0.762 vs 0.731). Without variance estimates or paired tests, the central claim of outperforming baselines on three of four benchmarks is not established. Please report multiple runs (e.g., varying claim-extraction temperature or random seed) with bootstrap confidence intervals, or a paired test across instances, and state how many runs were averaged.
  2. [Section 4.3-4.4, Eq. (1); Section 9] The per-claim score selects the best contradiction chunk independently across up to 31 overlapping chunks, including single-sentence chunks at m=8 and m=16. As the Limitations concede, pronouns severed from distant antecedents by chunking propagate as spurious contradictions. The near-saturated recall (0.94-1.0) and very low precision on TofuEval (0.224) are consistent with this mechanism. To support the claim that the recall gain over holistic NLI reflects genuine detection rather than a chunking artifact, please add a quantitative analysis of flagged claims (e.g., human annotation of a sample, or an ablation with claim decontextualization) and report the rate of contradictions attributable to coreference breaks.
  3. [Section 6.1.1] The stability claim is confounded because verification is always done by the same fixed DeBERTa model; only claim extraction varies across backbones. The lower variance of HallDetect relative to LLM CoT is therefore expected by design, and does not by itself demonstrate that the framework's decomposition or aggregation improves stability. To attribute stability to the framework, please decompose the variance (e.g., fix the extractor and vary the verifier, or use the backbone as an NLI judge over the same extracted claims) and show that the variance reduction is not solely due to the fixed discriminative verifier.
  4. [Section 2.2 and Section 9] The paper does not compare against MiniCheck, a compact DeBERTa-based fact-checker that is the closest related system. The absence of this comparison leaves open whether the multi-scale contrastive formulation adds value over an existing trained detector. Since the paper identifies MiniCheck as the single most important next experiment, adding that comparison (or justifying why the scope restriction to generative and embedding baselines is sufficient for the paper's claims) would substantially strengthen the contribution.
minor comments (5)
  1. [Section 4.5, Eq. (2)] The notation 'F EDscore' uses a space between 'F' and 'ED'; please use a single token such as FEDscore consistently throughout.
  2. [Appendix A.1, Table 5] In the row 'Claim cap min(10,4·n sent)', the subscript in 'n sent' is missing; it should read min(10, 4·n_sent) with a definition of n_sent.
  3. [Appendix A.4] The description of granularity m=16 for a document with n_sent=8 is ambiguous: since chunk_size = ceil(n_sent/m) = 1, m=16 and m=8 produce the same single-sentence chunks, yielding 23 distinct windows rather than 31. Please clarify this edge case.
  4. [Section 4.4] The string 'W ANLI' should be 'WANLI'.
  5. [Section 9] In the 'Single runs, backbones, and scope' paragraph, the coreference limitation is stated but not quantified; please also note whether the spurious-contradiction rate is expected to vary across backbones and datasets, given that extraction is the only backbone-dependent stage.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identified: HallDetect's score and aggregation are fixed design choices applied to an external NLI model, with no fitted parameters, no self-citation chain, and independent benchmark evaluation.

full rationale

No circular derivation found. HallDetect's per-claim score (Eq. 1) is a fixed function of probabilities from the external DeBERTa-v3 NLI model over a fixed multi-scale chunk library, and the final F_ED score (Eq. 2) is an a priori asymmetric geometric mean with a fixed threshold of 0.5. No parameter is fitted to the benchmark labels, no per-dataset tuning is performed, and no load-bearing step invokes a self-citation: the reference list contains no prior work by the present authors, and all cited systems (FActScore, SummaC, MiniCheck, etc.) are external. The paper's own Limitations section concedes confounded ablations, the missing MiniCheck comparison, and coreference-induced spurious contradictions; these are correctness and scope risks, not circularities, because the evaluation compares a fixed pipeline against fixed external baselines on external gold labels. The near-saturated recall is explicitly presented as an intended consequence of the asymmetric aggregation, not as a predicted quantity that reduces to the definition of the score. The central claim is therefore self-contained against external benchmarks and receives a circularity score of 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method relies on existing models and a priori design choices. No new entities are postulated. The main assumptions are about the reliability of the NLI model, the faithfulness of claim extraction, and the appropriateness of the conjunction-style aggregation for auditing.

free parameters (5)
  • Chunking granularities m = {1, 2, 4, 8, 16}
    Dyadic schedule chosen a priori to span context lengths; no empirical selection reported (Section 4.3).
  • Claim cap = min(10, 4 * n_sent)
    Cap on extracted claims per response; chosen by the authors, not tuned to benchmarks (Appendix A.1).
  • LLM sampling temperature = 0.2
    Used for claim extraction; fixed across experiments (Appendix A.1).
  • Decision threshold = 0.5
    Fixed threshold for the hallucinated class; the paper reports no per-dataset tuning (Section 3.1).
  • Max new tokens = 512
    Limit on generated extraction length; specified in Appendix A.1.
assumptions (5)
  • domain assumption DeBERTa-v3-large NLI model provides reliable entailment/contradiction probabilities for atomic claims against source chunks
    The entire verification stage rests on the zero-shot reliability of this off-the-shelf model (Section 4.4).
  • domain assumption Atomic claims extracted by a 4-bit quantized LLM are faithful, verbatim representations of the response's propositions
    The prompt instructs copy-faithful extraction, but quantization and prompt adherence are not validated (Section 4.2, Appendix A.3).
  • domain assumption A response is hallucinated if any single claim is contradicted; geometric mean with hard cutoff encodes a conjunction over claims
    This is a design choice for auditing, not an empirically established decision rule (Section 4.5).
  • domain assumption The source document alone is sufficient evidence for intrinsic faithfulness; extrinsic knowledge is out of scope
    The method deliberately targets intrinsic hallucinations only (Section 2, Limitations).
  • standard math Standard probability and geometric-mean mathematics are valid
    No nonstandard mathematical axioms are used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decomposed Entailment for Factuality Checking and Hallucination Detection." pith.science (2026). https://pith.science/paper/7DREMN56

@misc{pith2026260805823,
  author       = {Pith},
  title        = {Pith review of: Decomposed Entailment for Factuality Checking and Hallucination Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7DREMN56}},
  note         = {Machine review of arXiv:2608.05823}
}
read the original abstract

The reliability of Large Language Models (LLMs) is often compromised by factual inconsistencies, including hallucinations---cases where generated content is not supported by the underlying source. We present HallDetect, a lightweight, reference-free, and black-box framework for hallucination detection that we evaluate not only on summarization but across a broader range of source-grounded generation settings. HallDetect builds on decomposition-based factuality evaluation: generated content is decomposed into atomic claims, each verified by a compact encoder-based entailment model through a contrastive formulation over a multi-scale library of source chunks, and aggregated with an asymmetric score in which a single confidently contradicted claim flags the response. Under a controlled protocol in which all methods share the same 4-bit quantized backbones and consumer-grade hardware budget, HallDetect outperforms comparably resourced generative and embedding-based baselines on three of four benchmarks while remaining stable across backbone families, and yields a claim-to-span audit trail that localizes each error.

Figures

Figures reproduced from arXiv: 2608.05823 by the authors.

Figure 1
Figure 1. The HallDetect architecture. The pipeline bifurcates the input: the response [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

163 extracted references · 30 canonical work pages

  1. [1]

    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    Falsesum: Generating document-level NLI examples for recognizing factual inconsistency in summarization , author=. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    2022

  2. [2]

    Artificial intelligence , volume=

    Explanation in artificial intelligence: Insights from the social sciences , author=. Artificial intelligence , volume=. 2019 , publisher=

    2019

  3. [3]

    arXiv preprint arXiv:2505.11574 , year=

    QUANTIZATION MEETS REASONING: EXPLORING AND MITIGATING DEGRADATION OF LOW-BIT LLMS IN MATHEMATICAL REASONING , author=. arXiv preprint arXiv:2505.11574 , year=

    arXiv

  4. [4]

    Harvard Data Science Review , volume=

    How is ChatGPT’s behavior changing over time? , author=. Harvard Data Science Review , volume=. 2024 , publisher=

    2024

  5. [5]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  6. [6]

    Advances in neural information processing systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume=

  7. [7]

    Proceedings of the 2022 conference of the north american chapter of the association for computational linguistics: Human language technologies , pages=

    QAFactEval: Improved QA-based factual consistency evaluation for summarization , author=. Proceedings of the 2022 conference of the north american chapter of the association for computational linguistics: Human language technologies , pages=

    2022

  8. [8]

    arXiv preprint arXiv:2410.21819 , year=

    Self-preference bias in llm-as-a-judge , author=. arXiv preprint arXiv:2410.21819 , year=

    arXiv

Show all 163 references
  1. [9]

    Intelligent Systems Conference , pages=

    Llms will always hallucinate, and we need to live with this , author=. Intelligent Systems Conference , pages=. 2025 , organization=

    2025

  2. [10]

    Pubmedqa: A dataset for biomedical research question answering , author=. Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages=

    2019

  3. [11]

    Proceedings of the 30th International Conference on Intelligent User Interfaces , pages=

    Limitations of the llm-as-a-judge approach for evaluating llm outputs in expert knowledge tasks , author=. Proceedings of the 30th International Conference on Intelligent User Interfaces , pages=

  4. [12]

    arXiv preprint arXiv:2410.13341 , year=

    Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data , author=. arXiv preprint arXiv:2410.13341 , year=

  5. [13]

    Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , pages=

    Evaluating the factual consistency of abstractive text summarization , author=. Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP) , pages=

    2020

  6. [14]

    arXiv preprint arXiv:2305.11747 , year=

    Halueval: A large-scale hallucination evaluation benchmark for large language models , author=. arXiv preprint arXiv:2305.11747 , year=

    arXiv

  7. [15]

    Transactions of the Association for Computational Linguistics , volume=

    Lost in the middle: How language models use long contexts , author=. Transactions of the Association for Computational Linguistics , volume=

  8. [16]

    arXiv preprint arXiv:1803.05355 , year=

    FEVER: a large-scale dataset for fact extraction and VERification , author=. arXiv preprint arXiv:1803.05355 , year=

    arXiv

  9. [17]

    arXiv preprint arXiv:2111.09543 , year=

    Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing , author=. arXiv preprint arXiv:2111.09543 , year=

    arXiv

  10. [18]

    Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages=

    On the dangers of stochastic parrots: Can language models be too big?�� , author=. Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages=

    2021

  11. [19]

    Text summarization branches out , pages=

    Rouge: A package for automatic evaluation of summaries , author=. Text summarization branches out , pages=

  12. [20]

    Computational Linguistics , volume=

    Bias and fairness in large language models: A survey , author=. Computational Linguistics , volume=. 2024 , publisher=

    2024

  13. [21]

    ACM Journal of Data and Information Quality , volume=

    Biases in large language models: origins, inventory, and discussion , author=. ACM Journal of Data and Information Quality , volume=. 2023 , publisher=

    2023

  14. [22]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    arXiv

  15. [23]

    arXiv preprint arXiv:2005.00661 , year=

    On faithfulness and factuality in abstractive summarization , author=. arXiv preprint arXiv:2005.00661 , year=

    2005 arXiv

  16. [24]

    ACM Transactions on Information Systems , volume=

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions , author=. ACM Transactions on Information Systems , volume=. 2025 , publisher=

    2025

  17. [25]

    An Example of Using Google Scholar, ResearchGate and Academia

    The Role of Social Networking Services as Tools for Evaluating Scientific Achievements. An Example of Using Google Scholar, ResearchGate and Academia. edu by Selected Polish Representatives of Social Communication and Media Sciences , author=

  18. [26]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Combining fact extraction and verification with neural semantic matching networks , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  19. [27]

    Evaluating Programming Proficiency of Large Language Models: Assessing Large Language Models’ Effectiveness in Function and Class Generation, Code Commenting, Robustness, and Security , author=

  20. [28]

    Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

    Ranking generated summaries by correctness: An interesting but challenging application for natural language inference , author=. Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

  21. [29]

    arXiv preprint arXiv:2406.11267 , year=

    Mitigating large language model hallucination with faithful finetuning , author=. arXiv preprint arXiv:2406.11267 , year=

    arXiv

  22. [30]

    arXiv preprint arXiv:2506.05243 , year=

    CLATTER: Comprehensive Entailment Reasoning for Hallucination Detection , author=. arXiv preprint arXiv:2506.05243 , year=

    arXiv

  23. [31]

    Proceedings of the 28th International Conference on Computational Linguistics , pages=

    CharBERT: Character-aware Pre-trained Language Model , author=. Proceedings of the 28th International Conference on Computational Linguistics , pages=

  24. [32]

    IJCAI: proceedings of the conference/sponsored by the International Joint Conferences on Artificial Intelligence , year=

    A Survey on Word Meta-Embedding Learning , author=. IJCAI: proceedings of the conference/sponsored by the International Joint Conferences on Artificial Intelligence , year=

  25. [33]

    Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Learning word meta-embeddings , author=. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  26. [34]

    NL4IA Workshop Proceedings , year=

    Easy-to-use combination of POS and BERT model for domain-specific and misspelled terms , author=. NL4IA Workshop Proceedings , year=

  27. [35]

    context-predicting semantic vectors , author=

    Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors , author=. Proceedings of ACL (Volume 1: Long Papers) , pages=

  28. [36]

    2022 IEEE International Symposium on Circuits and Systems (ISCAS) , pages=

    Efficient Fine-Tuning of BERT Models on the Edge , author=. 2022 IEEE International Symposium on Circuits and Systems (ISCAS) , pages=. 2022 , organization=

    2022

  29. [37]

    2014 , booktitle=

    Glove: Global vectors for word representation , author=. 2014 , booktitle=

    2014

  30. [38]

    Transactions of the association for computational linguistics , volume=

    Enriching word vectors with subword information , author=. Transactions of the association for computational linguistics , volume=. 2017 , publisher=

    2017

  31. [39]

    JMIR medical informatics , volume=

    Word embedding for the French natural language in health care: comparative study , author=. JMIR medical informatics , volume=. 2019 , publisher=

    2019

  32. [40]

    APSIPA transactions on signal and information processing , volume=

    Evaluating word embedding models: Methods and experimental results , author=. APSIPA transactions on signal and information processing , volume=. 2019 , publisher=

    2019

  33. [41]

    Facebook AI Research and Inria and ALMAnaCH , title =

  34. [42]

    arXiv preprint arXiv:2302.10204 , year=

    A Benchmark of Nested Named Entity Recognition Approaches in Historical Structured Documents , author=. arXiv preprint arXiv:2302.10204 , year=

    arXiv

  35. [43]

    ACL 2020-58th Annual Meeting of the Association for Computational Linguistics , year=

    CamemBERT: a Tasty French Language Model , author=. ACL 2020-58th Annual Meeting of the Association for Computational Linguistics , year=

    2020

  36. [44]

    Expert Systems with Applications , volume=

    An experimental comparison of classification algorithms for imbalanced credit scoring data sets , author=. Expert Systems with Applications , volume=. 2012 , publisher=

    2012

  37. [45]

    Gradient Boosting Trees , author =

  38. [46]

    AMIA Annual Symposium Proceedings , volume=

    Launching into clinical space with medspaCy: a new clinical text processing toolkit in Python , author=. AMIA Annual Symposium Proceedings , volume=. 2021 , organization=

    2021

  39. [47]

    arXiv preprint arXiv:2110.12609 , year=

    No News is Good News: A Critique of the One Billion Word Benchmark , author=. arXiv preprint arXiv:2110.12609 , year=

    arXiv

  40. [48]

    Proceedings of ACL (Volume 1: Long Papers) , pages=

    Language Model Evaluation Beyond Perplexity , author=. Proceedings of ACL (Volume 1: Long Papers) , pages=

  41. [49]

    Proceedings of The Third Workshop on Representation Learning for NLP , pages=

    Evaluating Word Embeddings in Multi-label Classification Using Fine-Grained Name Typing , author=. Proceedings of The Third Workshop on Representation Learning for NLP , pages=

  42. [50]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  43. [51]

    2019 , school=

    Ensemble Learning for Extremely Imbalanced Data Flows , author=. 2019 , school=

    2019

  44. [52]

    Neural Computation , volume=

    The effect of class imbalance on Precision-Recall Curves , author=. Neural Computation , volume=. 2021 , publisher=

    2021

  45. [53]

    ACM computing surveys (CSUR) , volume=

    A survey of predictive modeling on imbalanced domains , author=. ACM computing surveys (CSUR) , volume=. 2016 , publisher=

    2016

  46. [54]

    Embedding

    El Boukkouri, Hicham and Ferret, Olivier and Lavergne, Thomas and Zweigenbaum, Pierre , year =. Embedding. Proceedings of the 57th

  47. [55]

    2023 20th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA) , pages=

    A study on the relevance of generic word embeddings for sentence classification in hepatic surgery , author=. 2023 20th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA) , pages=. 2023 , organization=

    2023

  48. [56]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , year =

    The Internal State of an LLM Knows When It`s Lying , author =. Findings of the Association for Computational Linguistics: EMNLP 2023 , year =

    2023

  49. [57]

    European journal of nuclear medicine and molecular imaging , volume=

    Large language models (LLM) and ChatGPT: a medical student perspective , author=. European journal of nuclear medicine and molecular imaging , volume=. 2023 , publisher=

    2023

  50. [58]

    ITM web of conferences , volume=

    Kernel density estimation and its application , author=. ITM web of conferences , volume=. 2018 , organization=

    2018

  51. [59]

    Diagnostic pathology , volume=

    Challenges and barriers of using large language models (LLM) such as ChatGPT for diagnostic medicine with a focus on digital pathology--a recent scoping review , author=. Diagnostic pathology , volume=. 2024 , publisher=

    2024

  52. [60]

    Transactions of the Association for Computational Linguistics , volume=

    Summeval: Re-evaluating summarization evaluation , author=. Transactions of the Association for Computational Linguistics , volume=. 2021 , publisher=

    2021

  53. [61]

    arXiv preprint arXiv:1903.00161 , year=

    DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs , author=. arXiv preprint arXiv:1903.00161 , year=

    1903 arXiv

  54. [62]

    arXiv preprint arXiv:1603.07771 , year=

    Neural text generation from structured data with application to the biography domain , author=. arXiv preprint arXiv:1603.07771 , year=

    arXiv

  55. [63]

    Asking and Answering Questions to Evaluate the Factual Consistency of Summaries , url=

    Wang, Alex and Cho, Kyunghyun and Lewis, Mike , year=. Asking and Answering Questions to Evaluate the Factual Consistency of Summaries , url=. doi:10.18653/v1/2020.acl-main.450 , journal=

    2020 doi

  56. [64]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  57. [65]

    arXiv preprint arXiv:2310.18344 , year=

    Chainpoll: A high efficacy method for llm hallucination detection , author=. arXiv preprint arXiv:2310.18344 , year=

    arXiv

  58. [66]

    Selected Works of Murray Rosenblatt , pages=

    Remarks on some nonparametric estimates of a density function , author=. Selected Works of Murray Rosenblatt , pages=. 2011 , publisher=

    2011

  59. [67]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

    CamemBERT: a Tasty French Language Model , author=. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

  60. [68]

    Procedia CIRP , volume=

    Extended kernel density estimation for anomaly detection in streaming data , author=. Procedia CIRP , volume=. 2022 , publisher=

    2022

  61. [69]

    The annals of mathematical statistics , volume=

    On estimation of a probability density function and mode , author=. The annals of mathematical statistics , volume=. 1962 , publisher=

    1962

  62. [70]

    Theory of Probability & Its Applications , volume=

    Non-parametric estimation of a multivariate probability density , author=. Theory of Probability & Its Applications , volume=. 1969 , publisher=

    1969

  63. [71]

    An evaluation summary method based on combination of automatic and textual complexity metrics (

    Ellouze, Samira Walha and Jaoua, Maher and Belguith, Lamia Hadrich , editor =. An evaluation summary method based on combination of automatic and textual complexity metrics (. Proceedings of. 2013 , pages =

    2013

  64. [72]

    Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems , pages=

    Fill in the BLANC: Human-free quality estimation of document summaries , author=. Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems , pages=

  65. [73]

    Biostatistics & Epidemiology , volume=

    A tutorial on kernel density estimation and recent advances , author=. Biostatistics & Epidemiology , volume=. 2017 , publisher=

    2017

  66. [74]

    Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

    A Field Guide to Automatic Evaluation of LLM-Generated Summaries , author=. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages=

  67. [75]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    Tofueval: Evaluating hallucinations of llms on topic-focused dialogue summarization , author=. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2024

  68. [76]

    Factual consistency evaluation of summarization in the Era of large language models , volume =

    Luo, Zheheng and Xie, Qianqian and Ananiadou, Sophia , urldate =. Factual consistency evaluation of summarization in the Era of large language models , volume =. doi:10.1016/j.eswa.2024.124456 , abstract =

    2024

  69. [77]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

    SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization , author=. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages=

  70. [78]

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Ask, Assess, and Refine: Rectifying Factual Consistency and Hallucination in LLMs with Metric-Guided Feedback Learning , author=. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  71. [79]

    arXiv e-prints , pages=

    A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods , author=. arXiv e-prints , pages=

  72. [80]

    , title =

    Turing, Alan M. , title =. Mind , volume =

  73. [81]

    Nature , volume =

    Learning Representations by Back-Propagating Errors , author =. Nature , volume =

  74. [82]

    Proceedings of the 10th European Conference on Artificial Intelligence (ECAI) , pages =

    Planning as Satisfiability , author =. Proceedings of the 10th European Conference on Artificial Intelligence (ECAI) , pages =

  75. [83]

    Artificial Intelligence , volume =

    Collaborative Plans for Complex Group Action , author =. Artificial Intelligence , volume =

  76. [84]

    The Entropy Formula for the

    Grisha Perelman , howpublished =. The Entropy Formula for the

  77. [85]

    Causality , author =

  78. [86]

    In: Advances in Neural Information Processing Systems 30 (NIPS 2017) , author=

    Attention Is All You Need. In: Advances in Neural Information Processing Systems 30 (NIPS 2017) , author=

    2017

  79. [87]

    Assessing GPT-3.5 and GPT-4 in Generating International Classification of Diseases Billing Codes , author=

  80. [88]

    arXiv preprint arXiv:2401.01313 , year=

    A comprehensive survey of hallucination mitigation techniques in large language models , author=. arXiv preprint arXiv:2401.01313 , year=

    arXiv

  81. [89]

    Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval , pages=

    Summarizing text documents: Sentence selection and evaluation metrics , author=. Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval , pages=

  82. [90]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Infolm: A new metric to evaluate summarization & data2text generation , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  83. [91]

    Advances in Neural Information Processing Systems , volume=

    Retrieval-augmented generation for knowledge-intensive nlp tasks , author=. Advances in Neural Information Processing Systems , volume=

  84. [92]

    Artificial Intelligence in Medicine , volume=

    Knowledge graph assisted end-to-end medical dialog generation , author=. Artificial Intelligence in Medicine , volume=. 2023 , publisher=

    2023

  85. [93]

    arXiv preprint arXiv:2310.03951 , year=

    Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinations , author=. arXiv preprint arXiv:2310.03951 , year=

    arXiv

  86. [94]

    arXiv preprint arXiv:2303.04360 , year=

    Does synthetic data generation of llms help clinical text mining? , author=. arXiv preprint arXiv:2303.04360 , year=

    arXiv

  87. [95]

    arXiv preprint arXiv:2309.11495 , year=

    Chain-of-verification reduces hallucination in large language models , author=. arXiv preprint arXiv:2309.11495 , year=

    arXiv

  88. [96]

    Advances in neural information processing systems , volume=

    Distributed representations of words and phrases and their compositionality , author=. Advances in neural information processing systems , volume=

  89. [97]

    IEEE transactions on knowledge and data engineering , volume=

    Extended isolation forest , author=. IEEE transactions on knowledge and data engineering , volume=. 2019 , publisher=

    2019

  90. [98]

    Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

    Bleu: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th annual meeting of the Association for Computational Linguistics , pages=

  91. [99]

    Proceedings of the Sixth Conference on Machine Translation , pages=

    A fine-grained analysis of BERTScore , author=. Proceedings of the Sixth Conference on Machine Translation , pages=

  92. [100]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    arXiv

  93. [101]

    CNIL anonymization , title =

  94. [102]

    CNIL re-inditification , title =

  95. [103]

    Li, Hongzhao and Wang, Hongyu and Sun, Xia and He, Hua and Feng, Jun , year = 2024, month = jul, pages =. Prompt-. 2024. doi:10.1109/ICME57554.2024.10687707 , urldate =

    2024

  96. [104]

    European journal of nuclear medicine and molecular imaging , volume=

    Large language models (LLM) and ChatGPT: what will the impact on nuclear medicine be? , author=. European journal of nuclear medicine and molecular imaging , volume=. 2023 , publisher=

    2023

  97. [105]

    European Journal of Nuclear Medicine and Molecular Imaging , volume=

    Large language models (LLM) and ChatGPT: a medical student perspective , author=. European Journal of Nuclear Medicine and Molecular Imaging , volume=. 2023 , publisher=

    2023

  98. [106]

    Revista do Col

    Innovations in surgical training: exploring the role of artificial intelligence and large language models (LLM) , author=. Revista do Col. 2023 , publisher=

    2023

  99. [107]

    European Semantic Web Conference , pages=

    Knowledge injection to counter large language model (llm) hallucination , author=. European Semantic Web Conference , pages=. 2023 , organization=

    2023

  100. [108]

    Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI) , pages=

    BioSimCSE: BioMedical Sentence Embeddings using Contrastive learning , author=. Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI) , pages=

  101. [109]

    Radiology , volume=

    Natural language processing in radiology: a systematic review , author=. Radiology , volume=. 2016 , publisher=

    2016

  102. [110]

    Big data , volume=

    On the safety of machine learning: Cyber-physical systems, decision sciences, and data products , author=. Big data , volume=. 2017 , publisher=

    2017

  103. [111]

    Counterfactual explanations without opening the black box: Automated decisions and the GDPR , author=. Harv. JL & Tech. , volume=. 2017 , publisher=

    2017

  104. [112]

    arXiv preprint arXiv:2305.00050 , year=

    Causal reasoning and large language models: Opening a new frontier for causality , author=. arXiv preprint arXiv:2305.00050 , year=

    arXiv

  105. [113]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Unified vision-language pre-training for image captioning and vqa , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  106. [114]

    2023 20th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA) , pages=

    A Study on the Relevance of Generic Word Embeddings for Sentence Classification in Hepatic Surgery , author=. 2023 20th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA) , pages=. 2023 , organization=

    2023

  107. [115]

    2021 , school=

    Improving a Search Engine for Answering User Questions in Natural Language , author=. 2021 , school=

    2021

  108. [116]

    Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks , author=. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    2021

  109. [117]

    CamemBERT: a Tasty French Language Model , author=

  110. [118]

    arXiv preprint arXiv:2305.18410 , year=

    Understanding Breast Cancer Survival: Using Causality and Language Models on Multi-omics Data , author=. arXiv preprint arXiv:2305.18410 , year=

    arXiv

  111. [119]

    arXiv preprint arXiv:2308.13067 , year=

    Causal parrots: Large language models may talk causality but are not causal , author=. arXiv preprint arXiv:2308.13067 , year=

    arXiv

  112. [120]

    Advances in Neural Information Processing Systems , volume=

    CEBaB: Estimating the causal effects of real-world concepts on NLP model behavior , author=. Advances in Neural Information Processing Systems , volume=

  113. [121]

    2023 International Joint Conference on Neural Networks (IJCNN) , pages=

    A Novel Clinical Trial Prediction-Based Factual Inconsistency Detection Approach for Medical Text Summarization , author=. 2023 International Joint Conference on Neural Networks (IJCNN) , pages=. 2023 , organization=

    2023

  114. [122]

    npj Digital Medicine , volume=

    Evaluating large language models on medical evidence summarization , author=. npj Digital Medicine , volume=. 2023 , publisher=

    2023

  115. [123]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  116. [124]

    Publications Manual , year = "1983", publisher =

    1983

  117. [125]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    1981

  118. [126]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  119. [127]

    Dan Gusfield , title =. 1997

    1997

  120. [128]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  121. [129]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  122. [130]

    Wikipedia , month = jan, year =

    Sentence embedding , copyright =. Wikipedia , month = jan, year =

  123. [131]

    Wikipedia , month = dec, year =

    Word embedding , copyright =. Wikipedia , month = dec, year =

  124. [132]

    How the GDPR will change the world , author=. Eur. Data Prot. L. Rev. , volume=. 2016 , publisher=

    2016

  125. [133]

    Proceedings of the 2024

    Fu, Jinlan and Ng, See-Kiong and Jiang, Zhengbao and Liu, Pengfei , editor =. Proceedings of the 2024. 2024 , pages =. doi:10.18653/v1/2024.naacl-long.365 , abstract =

    2024 doi

  126. [134]

    Proceedings of the 2022

    Honovich, Or and Aharoni, Roee and Herzig, Jonathan and Taitelbaum, Hagai and Kukliansy, Doron and Cohen, Vered and Scialom, Thomas and Szpektor, Idan and Hassidim, Avinatan and Matias, Yossi , editor =. Proceedings of the 2022. 2022 , pages =. doi:10.18653/v1/2022.naacl-main....

    2022 doi

  127. [135]

    and Le, Quoc V

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Ichter, Brian and Xia, Fei and Chi, Ed H. and Le, Quoc V. and Zhou, Denny , month = nov, year =. Chain-of-thought prompting elicits reasoning in large language models , isbn =. Proceedings of the 36th

  128. [136]

    Jolanta Chmielik and Natalia Grabar , Date-Added =. D. TAL , Number =

  129. [137]

    Enriched sublexical representations to access morphological structures

    Chiara Celata and Basilio Calderone and Fabio Montermini , Date-Added =. Enriched sublexical representations to access morphological structures. A psycho-computational account , Volume =. TAL , Number =

  130. [138]

    G. Mod. TAL , Number =

  131. [139]

    Moranapho : un syst

    Jean-Fran. Moranapho : un syst. TAL , Number =

  132. [140]

    Turunen and Sebastian Spiegler and Oskar Kohonen and Mikko Kurimo , Date-Added =

    Sami Virpioja and Ville T. Turunen and Sebastian Spiegler and Oskar Kohonen and Mikko Kurimo , Date-Added =. Empirical Comparison of Evaluation Methods for Unsupervised Learning of Morphology , Volume =. TAL , Number =

  133. [141]

    Structure and Interpretation of Computer Programs

    Harold Abelson and Gerald Jay Sussman and Julie Sussman. Structure and Interpretation of Computer Programs. 1985

    1985

  134. [142]

    Visual Information Extraction with Lixto

    Robert Baumgartner and Georg Gottlob and Sergio Flesca. Visual Information Extraction with Lixto. Proceedings of the 27th International Conference on Very Large Databases. 2001

    2001

  135. [143]

    Brachman and James G

    Ronald J. Brachman and James G. Schmolze. An overview of the KL-ONE knowledge representation system. Cognitive Science. 1985

    1985

  136. [144]

    Complexity results for nonmonotonic logics

    Georg Gottlob. Complexity results for nonmonotonic logics. Journal of Logic and Computation. 1992

    1992

  137. [145]

    Hypertree Decompositions and Tractable Queries

    Georg Gottlob and Nicola Leone and Francesco Scarcello. Hypertree Decompositions and Tractable Queries. Journal of Computer and System Sciences. 2002

    2002

  138. [146]

    Levesque

    Hector J. Levesque. Foundations of a functional approach to knowledge representation. Artificial Intelligence. 1984

    1984

  139. [147]

    Levesque

    Hector J. Levesque. A logic of implicit and explicit belief. Proceedings of the Fourth National Conference on Artificial Intelligence. 1984

    1984

  140. [148]

    On the compilability and expressive power of propositional planning formalisms

    Bernhard Nebel. On the compilability and expressive power of propositional planning formalisms. Journal of Artificial Intelligence Research. 2000

    2000

  141. [149]

    Langley , title =

    P. Langley , title =. Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , address =. 2000 , pages =

    2000

  142. [150]

    T. M. Mitchell. The Need for Biases in Learning Generalizations. 1980

    1980

  143. [151]

    M. J. Kearns , title =

  144. [152]

    Machine Learning: An Artificial Intelligence Approach, Vol. I. 1983

    1983

  145. [153]

    R. O. Duda and P. E. Hart and D. G. Stork. Pattern Classification. 2000

    2000

  146. [154]

    Suppressed for Anonymity , author=

  147. [155]

    Newell and P

    A. Newell and P. S. Rosenbloom. Mechanisms of Skill Acquisition and the Law of Practice. Cognitive Skills and Their Acquisition. 1981

    1981

  148. [156]

    A. L. Samuel. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 1959

    1959

  149. [157]

    Min, Sewon and Krishna, Kalpesh and Lyu, Xinxi and Lewis, Mike and Yih, Wen-tau and Koh, Pang Wei and Iyyer, Mohit and Zettlemoyer, Luke and Hajishirzi, Hannaneh , booktitle =

  150. [158]

    Chen, Sihao and Buthpitiya, Senaka and Fabrikant, Alex and Roth, Dan and Schuster, Tal , booktitle =

  151. [159]

    Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024) , pages =

    A Closer Look at Claim Decomposition , author =. Proceedings of the 13th Joint Conference on Lexical and Computational Semantics (*SEM 2024) , pages =

    2024

  152. [160]

    Molecular Facts: Desiderata for Decontextualization in

    Gunjal, Anisha and Durrett, Greg , booktitle =. Molecular Facts: Desiderata for Decontextualization in

  153. [161]

    Wan, David and Sinha, Koustuv and Iyer, Srinivasan and Celikyilmaz, Asli and Bansal, Mohit and Pasunuru, Ramakanth , booktitle =

  154. [162]

    and Hearst, Marti A

    Laban, Philippe and Schnabel, Tobias and Bennett, Paul N. and Hearst, Marti A. , journal =

  155. [163]

    Tang, Liyan and Laban, Philippe and Durrett, Greg , booktitle =

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.