Pith. sign in

REVIEW 4 major objections 8 minor 1 cited by

HuDEx: Integrating Hallucination Detection and Explainability for Enhancing the Reliability of LLM responses

T0 review · 4 major / 8 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that HuDEx, an 8-billion-parameter LoRA fine-tune of Llama 3.1, detects LLM hallucinations more accurately than Llama3 70B and GPT-4o on four benchmarks, with explanations judged 97–99% as good as human-written ones.

desk verdict A practical 8B hallucination detector with explanations, but the Section 5.1 wording about test sets used during training must be clarified before the headline claim can be trusted. read the letter →

arxiv 2502.08109 v1 pith:WAYO72BI submitted 2025-02-12 cs.CL cs.AI

Sujeong Lee , Hayoung Lee , Seongsoo Heo , Wonik Choi This is my paper · ORCID
classification cs.CLcs.AI
keywords hallucinationdetectionexplainabilityLLMreliabilityLoRAfine-tuningHaluEvalFactCHDFaithDialLLM-as-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

HuDEx is built around a simple idea: a hallucination detector should say why, not just that. The authors fine-tune a compact Llama 3.1 8B model with LoRA on the HaluEval, FactCHD, and FaithDial datasets so that a single pass produces both a verdict — hallucinated or not — and a plain-language explanation of the reasoning. Because two of those datasets lack explanations, the authors had Llama3 70B generate them and kept only the cases where the generated label agreed with the gold label (93.7% of samples), then spot-checked the explanations by human review. The central claim is that this two-task model beats much larger frontier models — Llama3 70B and GPT-4o — in detection accuracy on all four test sets (for example, 89.6% versus 86.6% and 82.7% on HaluEval QA), and produces explanations scored by an LLM judge at 97–99% of the human-written FactCHD gold explanations. The stated motivation is that pairing detection with explanation improves reliability: both users and the generating model itself can see and act on the mistake.

What carries the argument

The load-bearing object is HuDEx itself: Llama 3.1 8B adapted with LoRA and trained on two tasks at once — hallucination detection and hallucination explanation — on the same examples. Three pieces carry the argument. The data pipeline: HaluEval and FaithDial explanations are machine-generated by Llama3 70B and filtered by requiring the generated label to match the gold label (93.7% agreement), with a statistically sampled human check at a 99% confidence level. The inference prompt: a fixed hallucination-expert persona plus adaptive task stages that branch on whether the prompt supplies background knowledge, forcing the model to reason from the source when it exists and from its own knowledge when it does not. The evaluation protocol: detection is scored as binary accuracy against four test sets, while explanation quality is graded by an LLM judge on two 3-point criteria, factuality and clarity.

What would settle it

Ask the authors for the exact training split and re-run the detection experiments with any test instances that also appear in fine-tuning removed; if accuracy drops substantially, the central claim is an artifact. Independently, re-score a random sample of HuDEx's explanations with human raters instead of an LLM judge: the reliable-explanations claim would be falsified if human factuality and clarity scores fall well below the reported 97–99% conversion ratios.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that interpreting a hallucination and detecting it are the same task, and that training them together makes a small model a stronger judge than a much larger general-purpose one. HuDEx takes a response and optional background knowledge, and with a hallucination-expert persona and stage-structured prompt it first classifies the response and then justifies the classification in natural language. The authors report that this 8B model outperforms Llama3 70B and GPT-4o in binary detection accuracy on the HaluEval dialogue (80.6%), HaluEval QA (89.6%), FactCHD (70.3%), and FaithDial (58.8%) test sets, and that in zero-shot settings it leads on HaluEval summarization (77.9%) while trailing GPT-4o on HaluEval general (72.6% versus 78.0%). For explanations, an LLM judge scores HuDEx's explanations within 2–3% of the original FactCHD human-written explanations and better than Llama3 70B on clarity in two of three datasets. The paper frames this as a move beyond evaluation-only benchmarks: a detector that explains itself can be used both by end users and as a tool to evaluate other LLMs.

Load-bearing premise

The whole detection scoreboard rests on the assumption that the test examples were not seen during fine-tuning; the paper's wording about using test sets 'which were also used during training' leaves this ambiguous, so the reported lead over GPT-4o depends on the training and test splits being cleanly separated.

Editorial extensions

If this is right

  • A specialized 8B detector can replace frontier models for hallucination screening in knowledge-grounded settings, cutting cost and latency while improving accuracy.
  • Detection and justification come from one forward pass, so an end user or an auditing pipeline gets the reason along with the verdict, not an unexplained flag.
  • The same model can serve as a hallucination-focused judge for evaluating other LLMs, giving a more granular assessment than generic correctness scoring.
  • Zero-shot results on HaluEval summarization indicate that the detection skills transfer to at least some unseen task types without retraining.
  • The 93.7% label-agreement filter offers a concrete recipe for building explanation-annotated hallucination training sets from unlabeled benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stress test the paper does not run: evaluate HuDEx against freshly annotated, never-published samples from the same task distributions; if its lead over GPT-4o evaporates outside the four fixed test sets, the generality claim would need revision.
  • Because the explanations were taught by Llama3 70B, the model's stated reasons will inherit that teacher's blind spots; training the same pipeline with a different explanation source (human annotations, or a smaller model) would isolate how much of the detection gain comes from the explanation data itself.
  • The conclusion sketches an automated feedback loop; a natural next experiment is feeding HuDEx's explanations back to the generating model and measuring whether rewriting its answer reduces hallucination rates on a second pass.
  • The zero-shot gap on HaluEval general suggests the detector transfers best to knowledge-grounded tasks; adding ungrounded user-query hallucination data to training would test whether the approach extends to the hardest, most open-ended cases.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes HuDEx, a Llama-3.1-8B model fine-tuned with LoRA on the HaluEval dialogue/QA, FactCHD, and FaithDial datasets to perform binary hallucination detection and generate natural-language explanations for its decisions. The central empirical claim is that HuDEx surpasses larger models such as Llama3 70B and GPT-4o in detection accuracy on four in-distribution test sets (Table 3) and produces explanations nearly as good as the human-authored FactCHD explanations (Tables 5 and 6). The paper also reports zero-shot detection results on HaluEval summarization and HaluEval general (Table 4). The authors frame the contribution as an integration of hallucination detection with explanation generation, aimed at practical reliability.

Significance. If the claims hold, the paper demonstrates a useful direction: a compact 8B fine-tuned model can serve as a specialized hallucination detector with explanations, potentially offering a more interpretable and cheaper alternative to frontier general-purpose LLMs for model evaluation. The integration of detection and explanation is a worthwhile goal, and the training/inference pipeline is described in enough detail to be replicable in principle. The paper also attempts a zero-shot evaluation on unseen HaluEval subsets, which is a positive step. However, the significance is currently tempered by the lack of statistical rigor (single runs, no confidence intervals), the ambiguity about whether the test sets were used in training, and the heavy reliance on LLM-generated explanations and LLM-as-judge evaluation without reported human validation. The explanation results are only competitive (97\u201399% conversion ratio on FactCHD) rather than demonstrating a clear advantage, so the main contribution rests on the detection accuracy numbers.

major comments (4)
  1. [\u00a75.1] The sentence 'we used the test sets from HaluEval dialogue, HaluEval QA, FaithDial and FactCHD, which were also used during training' is ambiguous and potentially self-incriminating. If the exact test instances appeared in the LoRA fine-tuning data, the Table 3 accuracies (80.6, 89.6, 70.3, 58.8) reflect memorization rather than generalization, and the headline claim of surpassing Llama3 70B and GPT-4o is invalid. This is load-bearing because the abstract's central claim depends entirely on these four numbers. The authors must state unambiguously that only the official training splits were used for fine-tuning and that the test splits were held out, and they should provide evidence (e.g., data split configuration, code, or a clarification in the text) to rule out contamination.
  2. [\u00a76.1.2] The abstract says HuDEx 'surpasses larger LLMs, such as Llama3 70B and GPT-4, in hallucination detection accuracy,' but the zero-shot results in Table 4 show GPT4o achieving 78.0 on HaluEval general versus HuDEx's 72.6. The claim is therefore too broad. The abstract and conclusion should be qualified to the in-distribution benchmarks or to the HaluEval summarization zero-shot case, and the paper should discuss why performance does not transfer to the general user-query setting.
  3. [\u00a76.1 and \u00a75.2.1] All detection results in Tables 3 and 4 are point estimates from a single evaluation run, with no error bars, confidence intervals, or significance tests. For instance, the HaluEval QA difference between HuDEx (89.6) and GPT4o (86.6) may be within noise, especially given that accuracy on these benchmarks typically varies with decoding settings and prompt details. The paper should report multiple runs (e.g., different seeds or repeated sampling) and provide variance estimates, or at least explicitly acknowledge the lack of statistical significance testing and temper the superiority claims accordingly.
  4. [\u00a73.2 and \u00a76.2] The explanation-quality evaluation is not sufficiently grounded. For HaluEval and FaithDial, the explanations were generated by Llama3 70B (the same model used as a detection baseline), and for all datasets the judge is GPT4o. The paper states that human evaluation was conducted on a statistically sampled subset, but no results of that human evaluation are reported, and the sampling parameters (99% confidence, p=0.5, 2% margin of error) do not specify the achieved sample size or the measured defect rate. Additionally, the training-data filtering step that retains only the 93.7% of examples where Llama3 70B's predicted labels matched the gold labels could bias the model toward easy or idiosyncratic cases. The reliability of the explanation claim needs either reported human-evaluation numbers or a critical discussion of these limitations.
minor comments (8)
  1. [\u00a71] The model name is spelled inconsistently: 'HuDEx' in the title and most of the paper, but 'HuDex' in Section 1. Please unify the spelling.
  2. [\u00a72.2] There is a typo in 'categorize hallucinations into two broad two types' - 'two' is repeated.
  3. [\u00a74.2 and Figure 3] The persona generation process ('we provided ChatGPT with task details') is under-specified. It is unclear which ChatGPT model was used and how the persona selection was performed; more detail would aid reproducibility.
  4. [\u00a75.2.2] The phrase 'we used GPT4o as the judge for experiment' is missing an article; also, the version of GPT4o and decoding parameters are not reported, which are relevant for a judge-based evaluation.
  5. [\u00a73.2] The terms 'anomalies' (4.2%) and 'responses that failed to understand the prompt' (0.5%) are not defined. What constitutes an anomaly, and how were these cases filtered out?
  6. [\u00a76.1.1] The sentence 'On the FactCHD dataset, HuDEx outperformed Llama3 70B by ~11%' is ambiguous: 70.3 vs 59.4 is a difference of 10.9 percentage points, which is about 18% relative improvement. Please specify which measure is being used.
  7. [References] Some reference entries are incomplete or inconsistently formatted (e.g., [25] lacks venue information, and [29] lists only page numbers without a publisher). Please verify the bibliography.
  8. [Tables 3 and 4] The tables would benefit from confidence intervals or at least the number of samples per benchmark. The test sizes are partially given in Table 1 (e.g., 1,000 for HaluEval) but not repeated in the results tables, making the practical significance of the differences hard to judge.

Circularity Check

1 steps flagged · score 6.0 of 10

Section 5.1's statement that test sets 'were also used during training' makes Table 3's detection gains a fit, not a prediction; the abstract's 'surpasses larger LLMs' claim rests on those in-distribution numbers.

  1. fitted input called prediction [Section 5.1 (Datasets) and Table 3]
    "For the detection and explanation generation experiments, we used the test sets from HaluEval dialogue, HaluEval QA, FaithDial and FactCHD, which were also used during training."

    The abstract claims HuDEx 'surpasses larger LLMs ... in hallucination detection accuracy' based on Table 3, which reports accuracy on these test sets. The quoted sentence says the test sets were also used during training. If literal, the LoRA-finetuned parameters were fit to the exact labels being scored, so the Table 3 accuracies (80.6, 89.6, 70.3, 58.8) are memorization, not prediction; comparing them to Llama3 70B/GPT4o is not a generalization test. The zero-shot results in Table 4 do not support the headline: HuDEx loses on HaluEval general (72.6 vs 78.0). No clarification that only training splits were used, and no code/data released to verify.

full rationale

The paper is an empirical fine-tuning study with no mathematical derivation and no self-citations. The central detection claim rests on Table 3, and Section 5.1 states that the test sets 'were also used during training.' Under the paper's own wording, the model was trained on the same instances it is evaluated on, so the reported accuracies are fitted values rather than predictions; this is load-bearing because the abstract's headline comparison to larger LLMs depends entirely on those numbers. The zero-shot results in Table 4 provide some independent evidence but are mixed: HuDEx wins on summarization (77.9 vs 69.55/61.9) but loses on HaluEval general (72.6 vs GPT4o 78.0), so they do not rescue the unqualified claim. Table 1's separate train/test counts suggest the intended reading may be standard splits, but the manuscript never explicitly states that only training splits were used, and no code or data are released to verify separation. The explanation evaluation uses GPT4o as an LLM judge and Llama3 70B-generated training explanations; this is a validity concern but not circular because the judge is external to the trained model and the explanations are not used as detection labels. Overall, the in-distribution detection 'prediction' reduces to a fit under the paper's literal wording, while the zero-shot results retain independent content, warranting a partial-circularity score of 6.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The paper relies on benchmark labels, LLM-generated explanation targets, and an LLM judge, but introduces no new physical or conceptual entities. HuDEx itself is a trained system built from existing components, not a postulated entity with independent evidence requirements.

free parameters (2)
  • LoRA hyperparameters
    Rank, alpha, dropout, learning rate, batch size, and epochs are not reported; these hand-chosen settings affect the detection accuracy that supports the headline claim.
  • Explanation training data retention threshold = 93.7% matched labels
    Section 3.2 discards 4.2% anomalies and 0.5% clarification requests and keeps only matched labels; this data filtering choice shapes the trained model and is not varied or analyzed.
assumptions (5)
  • domain assumption Benchmark labels in HaluEval, FactCHD, and FaithDial are correct ground truth for hallucination status.
    Detection accuracy is computed against these labels; if labels are noisy, reported accuracies inherit the noise.
  • domain assumption The test splits used for evaluation were not part of the training instances.
    Section 5.1 says test sets were also used during training in an ambiguous way; if test instances were seen during fine-tuning, the detection results are invalid.
  • domain assumption GPT4o produces valid factuality and clarity judgments for hallucination explanations.
    Section 6.2.2 uses GPT4o as the sole judge, with no human agreement statistics reported for test explanations.
  • domain assumption Llama3 70B-generated explanations are acceptable training targets for teaching explanation quality.
    Section 3.2 uses Llama3 70B to create explanation supervision; any systematic errors in those explanations become training signal.
  • domain assumption LoRA fine-tuning preserves the base model's language and reasoning abilities while adapting to the task.
    Section 4.1 assumes LoRA is sufficient to learn the detection and explanation tasks; no evidence is given about capacity or forgetting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HuDEx: Integrating Hallucination Detection and Explainability for Enhancing the Reliability of LLM responses." pith.science (2026). https://pith.science/paper/WAYO72BI

@misc{pith2026250208109,
  author       = {Pith},
  title        = {Pith review of: HuDEx: Integrating Hallucination Detection and Explainability for Enhancing the Reliability of LLM responses},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WAYO72BI}},
  note         = {Machine review of arXiv:2502.08109}
}
read the original abstract

Recent advances in large language models (LLMs) have shown promising improvements, often surpassing existing methods across a wide range of downstream tasks in natural language processing. However, these models still face challenges, which may hinder their practical applicability. For example, the phenomenon of hallucination is known to compromise the reliability of LLMs, especially in fields that demand high factual precision. Current benchmarks primarily focus on hallucination detection and factuality evaluation but do not extend beyond identification. This paper proposes an explanation enhanced hallucination-detection model, coined as HuDEx, aimed at enhancing the reliability of LLM-generated responses by both detecting hallucinations and providing detailed explanations. The proposed model provides a novel approach to integrate detection with explanations, and enable both users and the LLM itself to understand and reduce errors. Our measurement results demonstrate that the proposed model surpasses larger LLMs, such as Llama3 70B and GPT-4, in hallucination detection accuracy, while maintaining reliable explanations. Furthermore, the proposed model performs well in both zero-shot and other test environments, showcasing its adaptability across diverse benchmark datasets. The proposed approach further enhances the hallucination detection research by introducing a novel approach to integrating interpretability with hallucination detection, which further enhances the performance and reliability of evaluating hallucinations in language models.

Figures

Figures reproduced from arXiv: 2502.08109 by the authors.

Figure 1
Figure 1. Schematic Representation of Our Hallucination Model [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of HuDEx: Training and Inference [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of Persona and Steps Used in Inference Prompts [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety

    cs.SE 2025-06 accept novelty 5.0 of 10

    A new survey organizes LLM interpretation methods by workflow stage and connects them to safety enhancement strategies and tools, covering around 70 works.

Reference graph

Works this paper leans on

30 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [1]

    Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with gpt-4, 2023

    2023

  2. [2]

    Felm: Benchmarking factuality evaluation of large language models

    Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. Felm: Benchmarking factuality evaluation of large language models. InThirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023

    work page 2023

  3. [3]

    Truthfulqa: Measuring how models mimic human falsehoods

    Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors,Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252. Association for Computational Linguistics, 5 2022

    work page 2022

  4. [4]

    Qafacteval: Improved qa-based factual con- sistency evaluation for summarization

    Alexander Fabbri, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. Qafacteval: Improved qa-based factual con- sistency evaluation for summarization. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza 8 A PREPRINT - F EBRUARY 13, 2025 Ruiz, editors, Proceedings of the 2022 Conference of the North American Chapter of the Association for Co...

    work page 2025

  5. [5]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł{}ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6000–6010. Curran Associates Inc., 2017

    work page

  6. [6]

    A survey of large language models, 2023

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. A survey of large language models, 2023

    2023

  7. [7]

    Emergent abilities of large language models, 2022

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models, 2022

    work page 2022

  8. [8]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner-Fushman, Blaise Agüera y Arcas, Dale Webster, Greg S. Corrado,...

    work page 2023

Show all 30 references
  1. [9]

    Fingpt: Open-source financial large language models

    Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. Fingpt: Open-source financial large language models. FinLLM Symposium at IJCAI 2023, 2023

    2023

  2. [10]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Comput. Surv., 55, 3 2023

    2023

  3. [11]

    On faithfulness and factuality in abstractive summarization

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational ...

    1906

  4. [12]

    Neural path hunter: Reducing hallucination in dialogue systems via path grounding

    Nouha Dziri, Andrea Madotto, Osmar Zaiane, and Avishek Joey Bose. Neural path hunter: Reducing hallucination in dialogue systems via path grounding. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen tau Yih, editors, Proceedings of the 2021 Conference on Emp...

    2021

  5. [13]

    The factual inconsistency problem in abstractive text summarization: A survey, 2023

    Yichong Huang, Xiachong Feng, Xiaocheng Feng, and Bing Qin. The factual inconsistency problem in abstractive text summarization: A survey, 2023

    2023

  6. [14]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023

    2023

  7. [15]

    Realtime qa: what’s the answer right now? In Proceedings of the 37th International Conference on Neural Information Processing Systems

    Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Velocity Yu, Dragomir Radev, Noah A Smith, Yejin Choi, and Kentaro Inui. Realtime qa: what’s the answer right now? In Proceedings of the 37th International Conference on Neural Information Proc...

    2024

  8. [16]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In FAccT 2021 - Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610–623. Associa...

    2021

  9. [17]

    Overcoming a theoretical limitation of self-attention, 2022

    David Chiang and Peter Cholak. Overcoming a theoretical limitation of self-attention, 2022

    2022

  10. [18]

    Batgpt: A bidirectional autoregessive talker from generative pre-trained transformer, 2023

    Zuchao Li, Shitou Zhang, Hai Zhao, Yifei Yang, and Dongjie Yang. Batgpt: A bidirectional autoregessive talker from generative pre-trained transformer, 2023

    2023

  11. [19]

    A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity

    Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V Do, Yan Xu, and Pascale Fung. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. In...

    2023

  12. [20]

    Judging llm-as-a-judge with mt- bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt- bench and chatbot arena. In Proceedings of the 37th International C...

    2024

  13. [21]

    Benchmarking foundation models with language-model-as-an- examiner, 2023

    Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, Jiayin Zhang, Juanzi Li, and Lei Hou. Benchmarking foundation models with language-model-as-an- examiner, 2023

    2023

  14. [22]

    G-eval: Nlg evaluation using gpt-4 with better human alignment

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proc...

    2023

  15. [23]

    Prd: Peer rank and discussion improve large language model based evaluations, 2023

    Ruosen Li, Teerth Patel, and Xinya Du. Prd: Peer rank and discussion improve large language model based evaluations, 2023

    2023

  16. [24]

    Halueval: A large-scale hallucination evaluation benchmark for large language models

    Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. Halueval: A large-scale hallucination evaluation benchmark for large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Lan...

    2023

  17. [25]

    Factchd: Benchmarking fact-conflicting hallucination detection, 2024

    Xiang Chen, Duanzheng Song, Honghao Gui, Chenxi Wang, Ningyu Zhang, Yong Jiang, Fei Huang, Chengfei Lv, Dan Zhang, and Huajun Chen. Factchd: Benchmarking fact-conflicting hallucination detection, 2024

    2024

  18. [26]

    Faithdial: A faithful benchmark for information-seeking dialogue

    Nouha Dziri, Ehsan Kamalloo, Sivan Milton, Osmar Zaiane, Mo Yu, Edoardo M Ponti, and Siva Reddy. Faithdial: A faithful benchmark for information-seeking dialogue. Transactions of the Association for Computational Linguistics, 10:1473–1490, 2022

    2022

  19. [27]

    Evaluating attribution in dialogue systems: The begin benchmark

    Nouha Dziri, Hannah Rashkin, Tal Linzen, and David Reitter. Evaluating attribution in dialogue systems: The begin benchmark. Transactions of the Association for Computational Linguistics, 10:1066–1083, 2022

    2022

  20. [28]

    The llama 3 herd of models, 2024

    Llama Team and Ai @ Meta. The llama 3 herd of models, 2024

    2024

  21. [29]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representa- tions, 2022

    2022

  22. [30]

    Gpt-4 technical report, 2024

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2025

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.