Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

An Extensive Evaluation of Factual Consistency in Large Language Models for Data-to-Text Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that factual consistency in data-to-text generation generally improves with model size, that Llama 2 models are the most consistent overall, and that source-reference divergence reliably lowers consistency scores.

desk verdict Useful breadth, credible size/ranking findings, but the divergence claim outruns the evidence because the metrics lean on references. read the letter →

arxiv 2411.19203 v1 pith:MJWD2AGE submitted 2024-11-28 cs.CL

Joy Mahapatra , Utpal Garain This is my paper · ORCID
classification cs.CL
keywords data-to-textgenerationfactualconsistencylargelanguagemodelshallucinationsource-referencedivergenceaveragerateofchangeautomaticevaluationmetricshuman
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to establish three empirical regularities about how large language models generate factual text from structured data such as tables, graphs, and meaning representations. Across five datasets, twelve models from five families, four automatic metrics, and human annotations, it finds that Llama 2 models usually produce the most factually consistent text, that increasing model size within a family generally improves consistency as measured by the average rate of change (AROC), and that source-reference divergence—when the reference text carries extra or different information than the source—lowers consistency scores for every model. Smaller models such as T5-large and BART-large remain competitive on large, lexically repetitive datasets such as E2E. Because scoring is reference-based, "factual consistency" here means agreement with the reference text rather than with the raw source, which shapes the third finding.

What carries the argument

The load-bearing object is the evaluation setup rather than a new algorithm. Four reference-based metrics—SummaC-Conv (natural-language-inference scoring with document-level aggregation), NEOverlap (named-entity overlap), AlignScore (multitask information alignment), and QAFactEval (question generation and answering)—score each generated text against the reference. The paper's summary statistic is AROC, the log-log rate of change in factual consistency per model-size change, $AROC = \frac{\log f(M_2) - \log f(M_1)}{\log m_2 - \log m_1}$, where $f(M)$ is a metric score and $m$ is trainable parameter count. For the divergence analysis, source-reference divergence is computed as $div(s,r) = 1 - \frac{LCS(s,r)}{\max(|s|,|r|)}$, where $LCS$ is longest common subsequence length. These two quantities carry the argument: positive AROC supports the scaling claim, and the divergence bins support the claim that reference drift lowers consistency.

What would settle it

Take the high-divergence test groups from the paper's Section 6 and have human annotators score generated text against the source data instead of against the reference; if the drop in consistency with divergence weakens or reverses, the third finding is an artifact of reference-based scoring rather than a property of model behavior.

Watch

Extended reading notes

Core claim

The paper's central claim is that factual consistency in data-to-text generation is not a single model property but a joint effect of model size, model family, and dataset characteristics. Concretely, the Llama 2 family outperforms BART, T5, OPT, and BLOOM on almost every dataset under SummaC-Conv, NEOverlap, AlignScore, and QAFactEval, with human annotations agreeing on the ordering. At the same time, T5-large and BART-large beat larger OPT and BLOOM models on E2E and DART, which are large and lexically homogeneous, indicating that small models can win when datasets are big and repetitive. AROC values—the log-scaled change in consistency per log-scaled change in parameter count—are positive for most families and datasets, which the authors read as evidence that growing model size generally improves factual consistency. Finally, when the reference text contains information not in the source, consistency scores fall for every family, and larger models fall less, suggesting that they are more robust to source-reference divergence.

Load-bearing premise

The evaluation treats the reference text as the factual context for scoring, so a model that follows the source data when the reference has drifted away from it is scored as less factually consistent.

Editorial extensions

If this is right

  • Within a model family, increasing trainable parameters—especially in T5 and BLOOM—tends to raise factual consistency scores across the five datasets.
  • Models from the Llama 2 family are the safest default for reference-level factual consistency, with the largest margins on DART and WebNLG.
  • On large, lexically repetitive datasets like E2E, smaller encoder-decoder models can match or beat much larger decoder-only models.
  • When source and reference diverge, every family's consistency score drops, so dataset cleaning or divergence-aware evaluation is needed before trusting reported scores.
  • Human evaluation agrees with the automatic metrics on the ordering of model families, supporting the use of reference-based automatic metrics in similar settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One implication the paper leaves implicit is that its size trend is a statement about matching references, not about faithfulness to source data; ordering could shift under source-grounded evaluation.
  • Because AROC is log-scaled on both axes, the steep gains of T5 and BLOOM relative to Llama 2 may partly reflect the small size of those families rather than faster learning.
  • A testable extension would be to filter or down-weight high-divergence training instances, which the paper's divergence finding suggests should improve reference-based consistency.
  • In safety-critical deployments such as medical or financial report generation, the paper's results argue for preferring a larger model family even when computational budget favors a small one.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports an evaluation of factual consistency in data-to-text generation (DTG) across five datasets (E2E, ViGGo, WikiTableText, DART, WebNLG) and twelve models from five LLM families (BART, T5, OPT, BLOOM, Llama 2), using four automatic metrics (SummaC-Conv, NEOverlap, AlignScore, QAFactEval) plus a small human annotation study. Three findings are claimed: Llama 2 generally produces more factually consistent text; smaller T5/BART models can be competitive on large, lexically less diverse datasets; increasing model size generally improves factual consistency as measured by AROC; and source-reference divergence reduces factual consistency.

Significance. If the findings hold, the paper would provide a useful comparative benchmark and practical guidance for model selection in DTG. The study has genuine breadth: twelve models, five datasets, three DTG task types, four complementary automatic metrics, and a human evaluation. The explicit investigation of source-reference divergence is a valuable direction. However, the third headline finding is threatened by a reference-based metric artifact, and the statistical support for the other findings is incomplete. The first two findings are plausible but need stronger uncertainty quantification before they can be considered established.

major comments (3)
  1. [§6, Figs. 11–14; §5.2; §3.3] The third headline finding is not established by the reported experiment. Section 3.3 defines the context for factual-consistency evaluation as the reference text 'or sometimes source data,' and Section 5.2 explicitly states that NEOverlap treats the reference text as the context. Section 6 reuses NEOverlap, SummaC-Conv, AlignScore, and QAFactEval but does not state which context is used for SummaC-Conv and AlignScore in that section. When the reference is the context, the high-divergence group contains references with content absent from the source; a model that faithfully reproduces the source will necessarily score lower on any reference-context metric. The downward slopes in Figures 11–14 are therefore at least partly mechanical for three of the four metrics. QAFactEval uses the source text for answer checking, but it is a single metric, and no source-based recomputation or human evaluation binned by divergence is reported. To support the claim that source-reference divergence reduces factual consistency of LLMs, the authors should either recompute the analysis with the source as context for all metrics, or report human annotations split by divergence level, or explicitly narrow the claim to reference-based agreement.
  2. [§4.3, Tables 2–5] The statistical significance reporting is incomplete and cannot be checked from the manuscript. The text states only that Welch's t-test was used 'with a significance level of p < 0.05 and a sample size of 6'; it does not define what the six observations are, how many generated outputs underlie each reported score, or whether the t-test is applied to per-instance metric scores, bootstraps, or model runs. No variance, confidence interval, or effect size is reported in Tables 2–5, and some adjacent differences are extremely small (e.g., Table 2, OPT on E2E: 0.3637 vs. 0.3621). The blanket statement that 'all results statistically significant' is not verifiable. The authors should provide the test definition, standard deviations or confidence intervals, and either exact p-values or a correction for multiple comparisons.
  3. [§5.5, Figs. 7–10; Eqs. (3)–(5)] The AROC analysis uses only the smallest and largest model in each family, so it does not actually test monotonicity of the size effect. Equations (3)–(5) define AROC as the chord slope between f(M1) and f(M2), and Figures 7–10 plot only those endpoint slopes. This is particularly problematic for OPT and BLOOM, where three sizes were evaluated and the intermediate point is available; for example, Table 2 shows OPT on E2E does not improve monotonically (0.3637, 0.3597, 0.3621), yet the two-endpoint AROC compresses this into a single number. The claim that larger models 'generally enhance' factual consistency should be supported by a regression over all available sizes, pairwise comparisons, or at least a discussion of the intermediate points.
minor comments (5)
  1. [Figure 5 and §6 captions] The model name/size is inconsistent: Figure 5 and the Section 6 figure captions use 'Llama2-6B,' while Tables 2–5 use 'Llama2-7B'; these should be unified.
  2. [Table 6] The caption says 'A lower value indicates higher factual consistency,' but Section 7 describes yes/no annotations aggregated as percentages; clarify whether Table 6 reports inconsistency rates rather than consistency rates.
  3. [Eq. (1)] Equation (1) uses argmax over the loss L(Mθ(s), r); maximizing a loss is nonstandard and should be argmin, or L should be defined explicitly as a negative log-likelihood.
  4. [Table 1] The table caption says it shows 'average length (in tokens),' but the columns list TTR, unique tokens, and total tokens; update the caption or add the missing average-length column.
  5. [§4.3] The paper says source and generated text lengths are fixed at 256 tokens but does not describe the truncation policy for longer sources or references.

Circularity Check

1 steps flagged · score 6.0 of 10

Third headline finding is partly constructed from reference-based metrics; the rest of the evaluation is independent.

  1. self definitional [Section 5.2 and Section 6 (NEOverlap context; divergence formula and conclusion)]
    "In the DTG context, the reference text is treated as the context, and the generated text as the claim. ... div(s, r) = 1− 1/max(|s|, |r|) LCS(s, r) ... First, the factual consistency of all LLMs, regardless of model family or size, decreases with an increase in source-reference divergence."

    For NEOverlap (and, by the paper's own Section 3.3 convention, any metric using the reference as context), factual consistency is measured as agreement between generated text and reference. Source-reference divergence is defined as LCS distance between source and reference. When the reference diverges from the source, it contains content absent from the source, so a faithful source-following generation cannot match the reference; the metric therefore drops by construction as divergence increases. The Section 6 conclusion restates this metric property as a property of the LLMs.

full rationale

No parameters are fitted, no uniqueness theorem is imported, and no load-bearing self-citation appears; the citations are to external metric/model papers. The first two findings (Llama 2 superiority and the AROC model-size trend) are computed directly from external metrics and are not forced by any definition. The third finding, however, is partially circular: with NEOverlap and any metric taking the reference as context, high source-reference divergence mechanically lowers scores because the reference contains facts absent from the source, independent of model behavior. The paper does not supply a source-based or human-evaluation version of the divergence split for the other reference-context metrics, so part of that headline claim reduces to metric construction. Because the circularity affects one of three load-bearing findings and is not a self-citation chain, the score is 6 rather than higher.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

None of the central claims are derived from a fitted model or a closed-form theory; they are aggregate observations over experiments. The only free choices are experimental hyperparameters, which are reported but not swept, and the metric definitions, which are external. The assumptions listed above are the premises that make the measurements interpretable.

assumptions (4)
  • domain assumption The four automatic metrics (SummaC-Conv, NEOverlap, AlignScore, QA-FactEval) are valid operationalizations of factual consistency.
    Sections 5.1-5.4 describe each metric but do not independently verify that they measure factual consistency in DTG; the human evaluation is the only check, and it is small.
  • domain assumption The reference text is an appropriate context for evaluating factual consistency of generated text.
    Sections 3.3 and 5.2 set the reference as context; this choice shapes the divergence finding and may not reflect source-grounded truth.
  • domain assumption QLoRA fine-tuning with the stated hyperparameters is a representative way to adapt each LLM family to DTG.
    Section 4.3 fixes one configuration for all models; results may differ with full fine-tuning or other PEFT methods.
  • domain assumption LCS-based source-reference divergence captures semantic divergence.
    Section 6 defines divergence by unigram LCS; this ignores paraphrastic divergence and synonyms, yet the analysis depends on the grouping into low-to-high divergence buckets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Extensive Evaluation of Factual Consistency in Large Language Models for Data-to-Text Generation." pith.science (2026). https://pith.science/paper/MJWD2AGE

@misc{pith2026241119203,
  author       = {Pith},
  title        = {Pith review of: An Extensive Evaluation of Factual Consistency in Large Language Models for Data-to-Text Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MJWD2AGE}},
  note         = {Machine review of arXiv:2411.19203}
}
read the original abstract

Large Language Models (LLMs) have shown exceptional performance across various Data-to-Text Generation (DTG) tasks. However, generating factually consistent text in DTG remains challenging for LLMs. Despite this, in-depth evaluations of LLM factual consistency for DTG remain missing in the current literature. This paper addresses this gap by providing an extensive evaluation of factual consistency in LLMs for DTG. Our evaluation covers five widely used DTG datasets (E2E, ViGGo, WikiTableText, DART, and WebNLG) and five prominent LLM families (T5, BART, OPT, BLOOM, and Llama 2). To ensure a thorough evaluation of factual consistency, we use four state-of-the-art automatic metrics and include essential human assessments. Our extensive evaluations reveals three key findings regarding factual consistency in LLMs for DTG. First, Llama 2 often excels in generating factually consistent text, although smaller models like T5 and BART can achieve strong factual consistency on larger, lexically less-diverse datasets. Second, the average rate of change (AROC) indicates that increasing model size (number of model trainable parameters) generally enhances factual consistency of LLMs in DTG. Third, we observe that source-reference divergence (i.e., when the reference text diverges semantically from the source) typically reduces the factual consistency of LLMs in DTG.

Figures

Figures reproduced from arXiv: 2411.19203 by the authors.

Figure 1
Figure 1. The three primary types of Data-to-Text Generation (DTG): graph-to-text, meaning representation [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Two popular language model architectures: encoder-decoder (left) and decoder-only (right). The [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An example of lack of factual consistency from the DART (graph-to-text) dataset, where the claim [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: An illustration of source-reference divergence from E2E Dusek et al. [2020] dataset. The reference [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: All twelve LLMs across five widely used families (BART, T5, OPT, BLOOM, and Llama2) arranged [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: AROC of the function f(·) (representing factual consistency) for the model size interval [m1, m2] between two LLMs, M1 and M2. LLM families differ significantly in model size scales; for instance, T5 sizes are in millions, while Llama2 sizes are in billions, impacting …
Figure 7
Figure 7. Figure 7: AROC values for all LLM families across the five DTG datasets based on [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: AROC values for all LLM families across the five DTG datasets based on [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: AROC values for all LLM families across the five DTG datasets based on [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: AROC values for all LLM families across the five DTG datasets based on [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Impact of source-reference divergence on factual consistency (measured through [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]
Figure 12
Figure 12. Figure 12: Impact of source-reference divergence on factual consistency (measured through [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Impact of source-reference divergence on factual consistency (measured through [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]
Figure 14
Figure 14. Figure 14: Impact of source-reference divergence on factual consistency (measured through [PITH_FULL_IMAGE:figures/full_fig_p011_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AssertBench: A Benchmark for Evaluating Self-Assertion in Large Language Models

    cs.CL 2025-06 conditional novelty 4.0 of 10

    AssertBench measures how often LLMs keep the same true/false evaluation of a fact across contradictory user framings, and finds most tested models agree with the user's framing more when they do not know the fact.

Reference graph

Works this paper leans on

44 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceedings of the ICLR , 2015. URL http://arxiv.org/abs/1409.0473

    arXiv 2015

  2. [2]

    Table-to-text: Describing table region with natural language

    Junwei Bao, Duyu Tang, Nan Duan, Zhao Yan, Yuanhua Lv, Ming Zhou, and Tiejun Zhao. Table-to-text: Describing table region with natural language. In Proceedings of the AAAI , pages 5020--5027, 2018. URL https://doi.org/10.1609/aaai.v32i1.11944

    work page doi:10.1609/aaai.v32i1.11944 2018

  3. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    work page 2020

  4. [4]

    The price of debiasing automatic metrics in natural language evalaution

    Arun Tejasvi Chaganty, Stephen Mussmann, and Percy Liang. The price of debiasing automatic metrics in natural language evalaution. In Proceedings of the ACL , pages 643--653, 2018. URL https://aclanthology.org/P18-1060/

    work page 2018

  5. [5]

    Qlora: Efficient finetuning of quantized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. In Proceedings of the NeurIPS , 2023. URL http://papers.nips.cc/paper\_files/paper/2023/hash/1feb87871436031bdc0f2beaa62a049b-Abstract-Conference.html

    work page 2023

  6. [6]

    Parikh, Ming - Wei Chang, Dipanjan Das, and William W

    Bhuwan Dhingra, Manaal Faruqui, Ankur P. Parikh, Ming - Wei Chang, Dipanjan Das, and William W. Cohen. Handling divergent reference texts when evaluating table-to-text generation. In Proceedings of the ACL , pages 4884--4895, 2019. URL https://doi.org/10.18653/v1/p19-1483

    doi:10.18653/v1/p19-1483 2019

  7. [7]

    The hitchhiker's guide to testing statistical significance in natural language processing

    Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. The hitchhiker's guide to testing statistical significance in natural language processing. In Proceedings of the ACL , pages 1383--1392, 2018. URL https://aclanthology.org/P18-1128/

    work page 2018

  8. [8]

    Evaluating the state-of-the-art of end-to-end natural language generation: The E2E NLG challenge

    Ondrej Dusek, Jekaterina Novikova, and Verena Rieser. Evaluating the state-of-the-art of end-to-end natural language generation: The E2E NLG challenge. Computer Speech and Language, 59: 0 123--156, 2020. URL https://doi.org/10.1016/j.csl.2019.06.009

    doi:10.1016/j.csl.2019.06.009 2020

Show all 44 references
  1. [9]

    Fabbri, Chien - Sheng Wu, Wenhao Liu, and Caiming Xiong

    Alexander R. Fabbri, Chien - Sheng Wu, Wenhao Liu, and Caiming Xiong. Qafacteval: Improved qa-based factual consistency evaluation for summarization. In Proceedings of the NAACL-HLT , pages 2587--2601, 2022. URL https://doi.org/10.18653/v1/2022.naacl-main.187

    2022 doi

  2. [10]

    The webnlg challenge: Generating text from RDF data

    Claire Gardent, Anastasia Shimorina, Shashi Narayan, and Laura Perez - Beltrachini. The webnlg challenge: Generating text from RDF data. In Proceedings of the INLG , pages 124--133, 2017. URL https://doi.org/10.18653/v1/w17-3518

    2017 doi

  3. [11]

    Survey of the state of the art in natural language generation: Core tasks, applications and evaluation

    Albert Gatt and Emiel Krahmer. Survey of the state of the art in natural language generation: Core tasks, applications and evaluation. Journal of Artificial Intelligence Research, 61: 0 65--170, 2018. URL https://doi.org/10.1613/jair.5477

    2018 doi

  4. [12]

    Openagi: When LLM meets domain experts

    Yingqiang Ge, Wenyue Hua, Kai Mei, Jianchao Ji, Juntao Tan, Shuyuan Xu, Zelong Li, and Yongfeng Zhang. Openagi: When LLM meets domain experts. In Proceedings of the NeurIPS , 2023. URL http://papers.nips.cc/paper\_files/paper/2023/hash/1190733f217404edc8a7f4e15a57f301-Abstract...

    2023

  5. [13]

    Alex Graves, Abdel - rahman Mohamed, and Geoffrey E. Hinton. Speech recognition with deep recurrent neural networks. In Proceedings of the ICASSP , pages 6645--6649, 2013. URL https://doi.org/10.1109/ICASSP.2013.6638947

    2013

  6. [14]

    Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries

    Benjamin Heinzerling and Kentaro Inui. Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries. In Proceedings of the EACL , pages 1772--1791, 2021. URL https://doi.org/10.18653/v1/2021.eacl-main.153

    2021 doi

  7. [15]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. CoRR, abs/2311.05232, 2...

    2023 doi arXiv

  8. [16]

    Survey of hallucination in natural language generation

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys , 55 0 (12): 0 248:1--248:38, 2023. URL https://doi.org/10.1145/3571730

    2023 doi

  9. [17]

    Juraj Juraska, Panagiotis Karagiannis, Kevin Bowden, and Marilyn A. Walker. A deep ensemble model with slot alignment for sequence-to-sequence natural language generation. In Proceedings of the NAACL-HLT , pages 152--162, 2018. URL https://doi.org/10.18653/v1/n18-1014

    2018 doi

  10. [18]

    Juraj Juraska, Kevin Bowden, and Marilyn A. Walker. Viggo: A video game corpus for data-to-text generation in open-domain conversation. In Proceedings of the INLG , pages 164--172, 2019. URL https://aclanthology.org/W19-8623/

    2019

  11. [19]

    Beyond reference-based metrics: Analyzing behaviors of open llms on data-to-text generation

    Zdenek Kasner and Ondrej Dusek. Beyond reference-based metrics: Analyzing behaviors of open llms on data-to-text generation. CoRR , abs/2401.10186, 2024. URL https://doi.org/10.48550/arXiv.2401.10186

    2024 doi arXiv

  12. [20]

    Tabgenie: A toolkit for table-to-text generation

    Zdenek Kasner, Ekaterina Garanina, Ondrej Pl \' a tek, and Ondrej Dusek. Tabgenie: A toolkit for table-to-text generation. In Proceedings of the ACL , pages 444--455, 2023. URL https://doi.org/10.18653/v1/2023.acl-demo.42

    2023 doi

  13. [22]

    BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the ACL , pag...

    2020 doi

  14. [23]

    Unifying structured data as graph for data-to-text pre-training

    Shujie Li, Liang Li, Ruiying Geng, Min Yang, Binhua Li, Guanghu Yuan, Wanwei He, Shao Yuan, Can Ma, Fei Huang, et al. Unifying structured data as graph for data-to-text pre-training. Transactions of the Association for Computational Linguistics, 12: 0 210--228, 2024. ISSN 2307...

    2024 doi

  15. [24]

    Faithfulness in natural language generation: A systematic survey of analysis, evaluation and optimization methods

    Wei Li, Wenhao Wu, Moye Chen, Jiachen Liu, Xinyan Xiao, and Hua Wu. Faithfulness in natural language generation: A systematic survey of analysis, evaluation and optimization methods. CoRR , abs/2203.05227, 2022. URL https://doi.org/10.48550/arXiv.2203.05227

    2022 doi arXiv

  16. [25]

    A survey on neural data-to-text generation

    Yupian Lin, Tong Ruan, Jingping Liu, and Haofen Wang. A survey on neural data-to-text generation. IEEE Transactions on Knowledge and Data Engineering , 36 0 (4): 0 1431--1449, 2024. URL https://doi.org/10.1109/TKDE.2023.3304385

    2024

  17. [26]

    High-quality data-to-text generation for severely under-resourced languages with out-of-the-box large language models

    Michela Lorandi and Anya Belz. High-quality data-to-text generation for severely under-resourced languages with out-of-the-box large language models. In Proceedings of the EACL Findings , pages 1451--1461, 2024. URL https://aclanthology.org/2024.findings-eacl.98

    2024

  18. [27]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Proceedings of the ICLR , 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  19. [28]

    Recurrent neural network based language model

    Tom \' a s Mikolov, Martin Karafi \' a t, Luk \' a s Burget, Jan Cernock \' y , and Sanjeev Khudanpur. Recurrent neural network based language model. In Proceedings of the Interspeech , pages 1045--1048, 2010. URL https://doi.org/10.21437/Interspeech.2010-343

    2010 doi

  20. [29]

    Linyong Nan, Dragomir R. Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Che...

    2021

  21. [30]

    The refinedweb dataset for falcon LLM: outperforming curated corpora with web data only

    Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Hamza Alobeidli, Alessandro Cappelli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon LLM: outperforming curated corpora with web data only. In Proceedings of the ...

    2023

  22. [31]

    Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer

    Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Proceedings of the NAACL-HLT , pages 2227--2237, 2018. URL https://doi.org/10.18653/v1/n18-1202

    2018 doi

  23. [32]

    Data-to-text generation with macro planning

    Ratish Puduppully and Mirella Lapata. Data-to-text generation with macro planning. Transactions of the Association for Computational Linguistics, 9: 0 510--527, 2021. URL https://doi.org/10.1162/tacl\_a\_00381

    2021 doi

  24. [33]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21: 0 140:1--140:67, 2020. URL...

    2020

  25. [34]

    A structured review of the validity of BLEU

    Ehud Reiter. A structured review of the validity of BLEU . Computational Linguistics, 44 0 (3), 2018. URL https://doi.org/10.1162/coli\_a\_00322

    2018 doi

  26. [35]

    Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilic, Daniel Hesslow, Roman Castagn \' e , Alexandra Sasha Luccioni, Fran c ois Yvon, Matthias Gall \' e , Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wa...

    2022 doi arXiv

  27. [36]

    Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P. Parikh. Sticking to the facts: Confident decoding for faithful data-to-text generation. CoRR , abs/1910.08684, 2019. URL http://arxiv.org/abs/1910.08684

    1910 arXiv

  28. [37]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

    2023 doi arXiv

  29. [38]

    Tackling hallucinations in neural chart summarization

    Saad Obaid ul Islam, Iza Skrjanec, Ondrej Dusek, and Vera Demberg. Tackling hallucinations in neural chart summarization. In Proceedings of the INLG , pages 414--423, 2023. URL https://doi.org/10.18653/v1/2023.inlg-main.30

    2023 doi

  30. [39]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the NeurIPS , pages 5998--6008, 2017. URL https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91f...

    2017

  31. [40]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R \' e mi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger,...

    2020 doi

  32. [41]

    On hallucination and predictive uncertainty in conditional language generation

    Yijun Xiao and William Yang Wang. On hallucination and predictive uncertainty in conditional language generation. In Proceedings of the EACL , pages 2734--2744, 2021. URL https://doi.org/10.18653/v1/2021.eacl-main.236

    2021 doi

  33. [42]

    Biomedical data-to-text generation via fine-tuning transformers

    Ruslan Yermakov, Nicholas Drago, and Angelo Ziletti. Biomedical data-to-text generation via fine-tuning transformers. In Proceedings of the INLG , pages 364--370, 2021. URL https://doi.org/10.18653/v1/2021.inlg-1.40

    2021 doi

  34. [43]

    Alignscore: Evaluating factual consistency with A unified alignment function

    Yuheng Zha, Yichi Yang, Ruichen Li, and Zhiting Hu. Alignscore: Evaluating factual consistency with A unified alignment function. In Proceedings of the ACL , pages 11328--11348, 2023. URL https://doi.org/10.18653/v1/2023.acl-long.634

    2023 doi

  35. [44]

    Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer...

    2022 doi arXiv

  36. [45]

    Siren's song in the AI ocean: A survey on hallucination in large language models

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren's song in the AI ocean: A survey on hallucination in large language models. CoRR, abs/2309.0...

    2023 doi arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.