Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

CF-RAG: A Dataset and Method for Carbon Footprint QA Using Retrieval-Augmented Generation

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that CarbonPDF, a fine-tuned 8-billion-parameter Llama 3 model with a retrieval critic and program-based reasoning, outperforms GPT-4o and fine-tuned TAT-LLM on answering carbon-footprint questions from raw PDF-extracted…

desk verdict A useful new benchmark for numeric QA over messy PDF sustainability reports, but the headline 93.7% EM is measured on template-generated questions whose gold documents come from the same generation process, so the broad real-world claim is not yet supported. read the letter →

arxiv 2508.03489 v1 pith:S3GBAJFJ submitted 2025-08-05 cs.CL cs.AI

Kaiwen Zhao , Bharathan Balaji , Stephen Lee This is my paper
classification cs.CLcs.AI
keywords productcarbonfootprintquestionansweringretrieval-augmentedgenerationPDFparsingLLMfine-tuningprogram-basedreasoningsustainabilityreportsnumerical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Product carbon-footprint reports are published as PDFs with a messy mix of tables, text, and charts, and extracting answers from them is mostly manual. The paper introduces CarbonPDF-QA, a dataset of about 18,000 questions over 1,735 real product reports from four companies, with answers verified by humans. It also proposes CarbonPDF, a fine-tuned Llama 3 8B model that first retrieves candidate documents, uses a trained critic to pick the most relevant one, and then generates an executable Python program to compute the answer. On this dataset the full pipeline reaches 93.70% exact match, compared with 49.20% for GPT-4o and 64.13% for a fine-tuned TAT-LLM, with much lower RMSE and MAE. The paper argues that a smaller, fine-tuned model with program-based reasoning can handle the inconsistent, noisy text that PDF parsing produces.

What carries the argument

The load-bearing components are (1) CarbonPDF-QA, a dataset built by regex-parsing 1,735 product carbon-footprint PDFs into CSV records, generating questions from predefined templates, auto-generating matching Python programs, and validating both through automated checks plus manual review; and (2) the CarbonPDF inference pipeline, in which a TF-IDF retriever proposes candidate documents, a fine-tuned critic model (trained with a conditional language-modeling objective) selects the most relevant document, and a program-based reasoner -- a LoRA fine-tuned Llama 3 8B -- generates an executable Python program whose output is the answer. The program-based reasoner is what carries the numerical reasoning: instead of asking the LLM to do arithmetic, it writes code that an interpreter executes exactly.

What would settle it

Have humans rewrite a sample of the test questions in their own words without changing the answers, then run the trained CarbonPDF model on those rewrites; if exact match falls to the level of the few-shot baselines, the claimed superiority does not generalize to natural query phrasing.

Watch

Extended reading notes

Core claim

The central discovery is that a comparatively small, fine-tuned language model can answer numerical questions about carbon footprints directly from raw, inconsistently structured PDF-extracted text, provided the pipeline compensates for two failure points: noisy retrieval and arithmetic error. CarbonPDF fine-tunes Llama 3 8B with LoRA to do two jobs. A critic model takes the top-k documents returned by a TF-IDF retriever and selects the single document most relevant to the question; a program-based reasoner then writes a short Python program that computes the answer, and a standard interpreter executes it. The authors report 93.70% exact match on the CarbonPDF-QA test set, against 49.20% for GPT-4o, 65.81% for the best few-shot program-based baseline (Gemini-2.0-flashProgram), and 64.13% for TAT-LLM fine-tuned on the same data. They attribute the gains to the critic correcting TF-IDF's imperfect top-1 selection and to shifting arithmetic onto the Python interpreter rather than the language model.

Load-bearing premise

The reported accuracy rests on the assumption that real user questions will resemble the template-generated questions used for both training and testing, since the test set is drawn from the same generation pipeline.

Editorial extensions

If this is right

  • Fine-tuning a modest 8B model on domain data can beat much larger few-shot LLMs for a specialized extraction-and-reasoning task, reducing inference cost for such workloads.
  • Adding a trained critic between retriever and generator improves end-to-end exact match by roughly 10 percentage points over using the retriever's top-1 document alone, since TF-IDF top-1 is correct only about 87% of the time.
  • Having the model generate executable programs rather than final answers directly raises exact match by about 27 percentage points, because arithmetic is done by an interpreter.
  • The CarbonPDF-QA dataset gives researchers a benchmark where tables and text arrive in the noisy, unformatted state produced by real PDF parsing, unlike curated hybrid-QA benchmarks.
  • The approach as presented is limited to text-extractable content; graph and chart data, which the paper identifies as a current limitation, require multimodal extensions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the test questions come from the same template generator as the training questions, the 93.7% exact match is an in-distribution number; a hand-written test set of naturally phrased queries would likely erode the margin over GPT-4o, possibly substantially.
  • The critic's gold labels are, by construction, the exact document that generated each question, so the critic is only ever asked to pick among candidates that include a guaranteed-correct answer; in open retrieval with no guaranteed gold document, its selection accuracy would likely drop.
  • The regex-based extraction of component percentages and the template-based program generation are tuned to the structure of these four companies' PCF reports; applying the pipeline to reports from other industries or companies without retuning would be a meaningful generalization test.
  • All reported numbers describe what the reports say, not measured emissions; a QA system built on this dataset will reproduce any over- or under-reporting contained in the source disclosures.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents CarbonPDF-QA, a dataset of roughly 18,000 template-generated question-answer pairs over 1,735 product carbon footprint PDFs from HP, Dell, Acer, and Lenovo, with answers derived from CSV extraction and a multi-stage validation workflow. It also presents CarbonPDF, a fine-tuned Llama-3-8B RAG system that combines a TF-IDF retriever, a fine-tuned critic that selects among top-k retrieved documents, and a program-based reasoner that generates Python code to compute answers. On the test split, the reported exact match is 93.70%, compared with 64.13% for TAT-LLM and 49.20% for GPT-4o, with ablations attributing gains to the critic and to program-based reasoning.

Significance. The dataset addresses a real and underexplored problem: question answering over noisy, PDF-extracted sustainability data, and the program-based reasoning design is a sensible way to handle numerical arithmetic. The explicit treatment of retrieval errors via a critic is also a worthwhile direction. However, the evaluation is conducted on template-generated questions from the same pipeline used to construct the training data, and the baseline systems receive a weaker retrieval protocol than the proposed model. As a result, the headline claim that CarbonPDF outperforms current state-of-the-art QA systems is not yet established for real-world queries; the contribution is better framed as an in-distribution benchmark and a competitive method on that benchmark.

major comments (3)
  1. [§4.1 / Table 3 / Figure 4] The RAG baselines (GPT-4o, Gemini-2.0-flash, DeepSeek-R1) receive only the top-1 TF-IDF document, while CarbonPDF receives top-k documents plus a fine-tuned critic. Since Figure 4 shows a top-1 hit rate of about 87%, the baselines are forced to answer from the wrong document on roughly 13% of test questions, independent of their reasoning quality. The ablation in Table 6 attributes about 10 EM points to the critic, so part of the margin in Table 3 may be due to the retrieval setup rather than to the reasoner. To support the claim that CarbonPDF outperforms these systems, the comparison should give baselines the same top-k access, an oracle or gold document, or the same critic, or otherwise isolate the reasoner's contribution with retrieval held fixed.
  2. [§2.2 / §5 / Table 3] The test questions are generated by the same Python template and CSV pipeline as the training questions, with document-level splitting but the same four companies, the same four question types, the same template wording, and the same CSV-derived answer schema. This makes Table 3 an in-distribution evaluation on synthetic questions, not a test of handling the 'unstructured and inconsistent' real user queries claimed in the abstract. The §7 processor/mainboard example shows that the model is sensitive to lexical variation, but this limitation is not measured. Please add an out-of-distribution evaluation (for example, human-written questions, paraphrases, or unseen product terms or report layouts) and report results separately.
  3. [§3.3 / Eq. (1)] The critic's ground-truth labels are the source documents used to generate each question. Because the generated questions contain explicit product identifiers, the critic may be learning to match the product name string to the containing document rather than to judge semantic relevance. This would explain the critic's high contribution in Table 6 but would not transfer to queries without exact identifiers. Please analyze this possibility, for example by removing product tokens from critic inputs, comparing the critic against a lexical product-name matcher, or evaluating on documents that do not contain the product identifier.
minor comments (7)
  1. [Figure 1 / Table 2] The dataset name is inconsistent: Figure 1 refers to 'CarbonPCF-QA' and Table 2 spells 'CaronPDF-QA'; please use 'CarbonPDF-QA' throughout.
  2. [§5.1] The text says 'The critic helps validate program outputs,' but according to §3.3 the critic selects relevant documents; please correct this description.
  3. [Abstract / §2.2] The abstract states the dataset has 'human-annotated answers,' but §2.2 describes ground truth from CSV files with manual review only for samples that failed automated checks; please align the annotation claim with the actual procedure.
  4. [Listing 2 / §3.2] The code listings have formatting artifacts such as spaced variable names ('m a n u f a c t u r i n g _ p e r c e n t') and 'sklearn.Tfidf Vectorizer' is not in backticks; please fix the listings.
  5. [References] Reference [25] has a malformed arXiv identifier ('arXiv:2111.156647'); please correct it.
  6. [§4.1] The sentence 'Since our data lacks formatted tables, we used the reference text as a replacement within their prompt template' should specify whether the reference text is the top-1 retrieved document or the gold document, as this affects the comparison.
  7. [Abstract / Conclusion] The paper claims the dataset is open-source but provides no repository link; please include a URL or a clear availability statement.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the derivation chain is self-contained and the reported 93.70% EM is a legitimate in-distribution measurement; remaining concerns are external-validity issues, not derivation circularity.

full rationale

The claimed derivation chain is self-contained and no load-bearing step reduces to its own input. The dataset is built from PDFs via regex extraction into CSVs (§2.1); questions and auxiliary programs are generated from those CSVs by template scripts (§2.2); the critic's gold document is the document used at question-generation time (§3.3); and evaluation (§5) is on a document-level held-out split of the same pipeline. This sharing of the generation pipeline is a real external-validity limitation: the 93.70% EM may not transfer to genuinely new user queries. But it is not circularity in the defined sense: the model must still locate numerical values inside noisy, unstructured reference text and emit executable programs, and the ground-truth answers are not derivable from the questions alone. The critic's relevance label ('the document that generated this question') is a standard, if synthetic, retrieval-label definition; the critic's contribution is measured empirically and is not forced by construction, as evidenced by the non-perfect EM (93.70%) and by the paper's own §7 admission that lexical variants such as 'processor' versus 'mainboard' are not handled. The only self-citation is CaML [3], used as a baseline in Table 3 and for SBERT; it is not load-bearing for the central claim. Concerns about template-based test questions and benchmark validity belong to correctness risk, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central performance claim rests on the benchmark being representative of real PDF analysis and on the extraction pipeline being lossless. The ledger shows no invented entities and no fitted scientific parameters, but several domain assumptions about report formatting and question distribution are load-bearing.

free parameters (3)
  • Validation threshold for PCF deviation = 2 (multiples of MAE)
    Hand-chosen rule in §2.2 to flag samples for manual review; affects which 56 samples were inspected, not the final model.
  • Component sum validation range = 99% to 101%
    Hand-chosen range in §2.2 to flag samples where component percentages don't sum to 100%; affected 24 samples.
  • Retriever top-k for critic = 10 (from Figure 4)
    Number of candidate documents passed to the critic; hit rate approaches ~100% at k=10, and the critic selects among these.
assumptions (4)
  • domain assumption PDF text extraction with PyMuPDF captures all information needed to answer the questions, including table content.
    The entire pipeline operates on extracted raw text from PDFs; charts are explicitly excluded (Discussion).
  • domain assumption Component carbon footprint percentages appear as text patterns like 'Chassis 7.5%' that regex rules can reliably detect.
    Data collection in §2.1 uses regular expressions; documents that don't match are discarded.
  • domain assumption Template-generated questions are representative of real user questions about carbon footprints.
    The dataset is built from a fixed question template set; no evaluation on human-authored questions.
  • domain assumption Gold values in the CSV, after human validation, are accurate.
    Ground truth for QA is the CSV extracted by regex and human-checked in §2.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CF-RAG: A Dataset and Method for Carbon Footprint QA Using Retrieval-Augmented Generation." pith.science (2026). https://pith.science/paper/S3GBAJFJ

@misc{pith2026250803489,
  author       = {Pith},
  title        = {Pith review of: CF-RAG: A Dataset and Method for Carbon Footprint QA Using Retrieval-Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S3GBAJFJ}},
  note         = {Machine review of arXiv:2508.03489}
}
read the original abstract

Product sustainability reports provide valuable insights into the environmental impacts of a product and are often distributed in PDF format. These reports often include a combination of tables and text, which complicates their analysis. The lack of standardization and the variability in reporting formats further exacerbate the difficulty of extracting and interpreting relevant information from large volumes of documents. In this paper, we tackle the challenge of answering questions related to carbon footprints within sustainability reports available in PDF format. Unlike previous approaches, our focus is on addressing the difficulties posed by the unstructured and inconsistent nature of text extracted from PDF parsing. To facilitate this analysis, we introduce CarbonPDF-QA, an open-source dataset containing question-answer pairs for 1735 product report documents, along with human-annotated answers. Our analysis shows that GPT-4o struggles to answer questions with data inconsistencies. To address this limitation, we propose CarbonPDF, an LLM-based technique specifically designed to answer carbon footprint questions on such datasets. We develop CarbonPDF by fine-tuning Llama 3 with our training data. Our results show that our technique outperforms current state-of-the-art techniques, including question-answering (QA) systems finetuned on table and text data.

Figures

Figures reproduced from arXiv: 2508.03489 by the authors.

Figure 1
Figure 1. The CarbonPCF-QA dataset is derived from Product Carbon Footprint (PCF) reports. Examples demonstrate the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Data validation workflow. 1 total_carbon =505.0 2 manufacturing_percent =0.5 3 manufacturing_carbon = total_carbon * manufacturing_percent 4 display_percent =0.24 5 display_carbon = total_carbon * manufacturing_percent * display_percent 6 answer =[ manufacturing_carbon , display_carbon ] Listing 2: HP Calculation question program example tailored to specific question types. The templates follow a consistent structur… view at source ↗
Figure 3
Figure 3. Overview of CarbonPDF. 1 2 3 4 5 6 7 8 9 10 Top k documents 0 10 20 30 40 50 60 70 80 90 100 Hit percentage (%) TF-IDF SBERT Contriever [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of retrievers [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. GaiaFlow: Semantic-Guided Diffusion Tuning for Carbon-Frugal Search

    cs.IR 2026-02 unverdicted novelty 5.0 of 10

    GaiaFlow combines semantic-guided diffusion tuning with early-exit and quantization methods to lower carbon emissions in neural information retrieval while maintaining competitive effectiveness.

Reference graph

Works this paper leans on

51 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    Acer Inc. 2024. Acer Library Document Collection (Product Carbon Footprint). https://www.acer.com/us-en/sustainability/product-carbon-footprint Accessed: 2024-08-12

    work page 2024

  2. [2]

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-rag: Self-reflective retrieval augmented generation. InNeurIPS 2023 Workshop on Instruction Tuning and Instruction Following

    work page 2023

  3. [3]

    Bharathan Balaji, Venkata Sai Gargeya Vunnava, Geoffrey Guest, and Jared Kramer. 2023. CaML: Carbon footprinting of household products with zero- shot semantic text similarity. InProceedings of the ACM Web Conference 2023. 4004–4014

    work page 2023

  4. [4]

    Tom B Brown. 2020. Language models are few-shot learners.arXiv preprint ArXiv:2005.14165(2020)

    arXiv 2020

  5. [5]

    Tianfeng Chai and Roland R Draxler. 2014. Root mean square error (RMSE) or mean absolute error (MAE)?–Arguments against avoiding RMSE in the literature. Geoscientific model development7, 3 (2014), 1247–1250

    work page 2014

  6. [6]

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2019. Tabfact: A large-scale dataset for table-based fact verification.arXiv preprint arXiv:1909.02164(2019)

    arXiv 2019

  7. [7]

    Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Wang. 2020. Hybridqa: A dataset of multi-hop question answering over tabular and textual data.arXiv preprint arXiv:2004.07347(2020)

    arXiv 2020

  8. [8]

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, et al

Show all 51 references
  1. [9]

    Zhiyu Chen, Shiyang Li, Charese Smiley, Zhiqiang Ma, Sameena Shah, and William Yang Wang. 2022. Convfinqa: Exploring the chain of numerical reasoning in conversational finance question answering.arXiv preprint arXiv:2210.03849 (2022)

    2022 arXiv

  2. [10]

    Google DeepMind. 2024. Gemini-2.0-flash. https://deepmind.google/ technologies/gemini/flash/. Accessed: 2025-02-14

    2024

  3. [11]

    Dell Inc. 2024. Dell Library Document Collection (Product Carbon Foot- print). https://www.dell.com/en-us/dt/corporate/social-impact/advancing- sustainability/climate-action/product-carbon-footprints.htm#tab0=0 Accessed: 2024-08-12

    2024

  4. [12]

    Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.arXiv preprint arXiv:1903.00161(2019)

    2019 arXiv

  5. [13]

    Matthew Dunn, Levent Sagun, Mike Higgins, V Ugur Guney, Volkan Cirik, and Kyunghyun Cho. 2017. Searchqa: A new q&a dataset augmented with context from a search engine.arXiv preprint arXiv:1704.05179(2017)

    2017 arXiv

  6. [14]

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Pal: Program-aided language models. In International Conference on Machine Learning. PMLR, 10764–10799

    2023

  7. [15]

    Izacard Gautier, Lewis Patrick, Lomeli Maria, Hosseini Lucas, Petroni Fabio, Schick Timo, Dwivedi-Yu Jane, Joulin Armand, Riedel Sebastian, and Grave Edouard. 2022. Few-shot learning with retrieval augmented language models. arXiv preprint arXiv: 2208.03299(2022)

    2022 arXiv

  8. [16]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al . 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

    2025 arXiv

  9. [17]

    Udit Gupta, Mariam Elgamal, Gage Hills, Gu-Yeon Wei, Hsien-Hsin S Lee, David Brooks, and Carole-Jean Wu. 2022. ACT: Designing sustainable computer systems with an architectural carbon modeling tool. InProceedings of the 49th Annual International Symposium on Computer Architect...

    2022

  10. [18]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. InInternational conference on machine learning. PMLR, 3929–3938

    2020

  11. [19]

    Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. Chartllama: A multimodal llm for chart understanding and generation.arXiv preprint arXiv:2311.16483(2023)

    2023 arXiv

  12. [20]

    2018.Life cycle assessment

    Michael Z Hauschild, Ralph K Rosenbaum, Stig Irving Olsen, et al . 2018.Life cycle assessment. Vol. 2018. Springer

    2018

  13. [21]

    HP Inc. 2024. HP Library Document Collection (Product Carbon Footprint). https://h20195.www2.hp.com/v2/library.aspx?doctype=95&footer=95&filter_ doctype=no&showregionfacet=yes&filter_country=no&cc=us&lc=en&filter_ oid=no&filter_prodtype=rw&prodtype=ij&showproductcompatibility=...

    2024

  14. [22]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685(2021)

    2021 arXiv

  15. [23]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card.arXiv preprint arXiv:2410.21276(2024)

    2024 arXiv

  16. [24]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bo- janowski, Armand Joulin, and Edouard Grave. 2021. Unsupervised dense in- formation retrieval with contrastive learning.arXiv preprint arXiv:2112.09118 (2021)

    2021 arXiv

  17. [25]

    Geewook Kim, Teakgyu Hong, Moonbin Yim, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2021. Donut: Document understanding transformer without ocr.arXiv preprint arXiv:2111.156647, 15 (2021), 2

    2021 arXiv

  18. [26]

    DP Kingma. 2014. Adam: a method for stochastic optimization.arXiv preprint arXiv:1412.6980(2014)

    2014 arXiv

  19. [27]

    Lenovo Inc. 2024. Lenovo Library Document Collection (Product Carbon Foot- print). https://www.lenovo.com/us/en/compliance/eco-declaration/?orgRef= https%253A%252F%252Fwww.google.com%252F Accessed: 2024-08-12

    2024

  20. [28]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing S...

    2020

  21. [29]

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022. Solving quantitative reasoning problems with language models.Advances in Neural Information Processing System...

    2022

  22. [30]

    Moxin Li, Fuli Feng, Hanwang Zhang, Xiangnan He, Fengbin Zhu, and Tat-Seng Chua. 2022. Learning to imagine: Integrating counterfactual thinking in neural discrete reasoning. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo...

    2022

  23. [31]

    Xiao Li, Yin Zhu, Sichen Liu, Jiangzhou Ju, Yuzhong Qu, and Gong Cheng. 2023. Dyrren: A dynamic retriever-reranker-generator model for numerical reasoning over tabular and textual data. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 13139–13147

    2023

  24. [32]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. When not to trust language models: Investigat- ing effectiveness of parametric and non-parametric memories.arXiv preprint arXiv:2212.10511(2022)

    2022 arXiv

  25. [33]

    Meta AI. 2023. LLaMA 3 8B. https://llama.meta.com/ Accessed: 2024-08-10

    2023

  26. [34]

    Olivier M. Schwab. 2022. Sustainability reporting: why it’s impor- tant. https://www.weforum.org/agenda/2022/06/sustainability-reporting-why- important/ [Online; accessed 14-August-2024]

    2022

  27. [35]

    PyMuPDF Developers. 2024. PyMuPDF Documentation. https://pymupdf. readthedocs.io/en/latest/ Accessed: 2024-08-08

    2024

  28. [36]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text.arXiv preprint arXiv:1606.05250(2016)

    2016 arXiv

  29. [37]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks.arXiv preprint arXiv:1908.10084(2019)

    2019 arXiv

  30. [38]

    Rodriguez, Isabel and Caglio, Ariela. 2023. Tackling the Sustainability Reporting Challenge. A Policy Guide. https://unctad.org/system/files/official-document/ diae2022d3_en.pdf [Online; accessed 14-August-2024]

    2023

  31. [39]

    Diogo Silva, A Oliveira Nunes, Aparecida da Silva Moris, Cassiano Moro, and Thiago Oliveira Rodrigues Piekarski. 2017. How important is the LCA software tool you choose Comparative results from GaBi, openLCA, SimaPro and Umberto. InProceedings of the VII Conferencia Internacio...

    2017

  32. [40]

    Alon Talmor and Jonathan Berant. 2018. The web as a knowledge-base for answering complex questions.arXiv preprint arXiv:1803.06643(2018)

    2018 arXiv

  33. [41]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

    2022

  34. [42]

    Yifan Wei, Fangyu Lei, Yuanzhe Zhang, Jun Zhao, and Kang Liu. 2023. Multi- view graph representation learning for answering hybrid numerical reasoning question.arXiv preprint arXiv:2305.03458(2023)

    2023 arXiv

  35. [43]

    Cort J Willmott and Kenji Matsuura. 2005. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model CF-RAG: A Dataset and Method for Carbon Footprint QA Using Retrieval-Augmented Generation Conference’17, July 2017, Washingto...

    2005

  36. [44]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for di- verse, explainable multi-hop question answering.arXiv preprint arXiv:1809.09600 (2018)

    2018 arXiv

  37. [45]

    Ziqiang Yuan, Kaiyuan Wang, Shoutai Zhu, Ye Yuan, Jingya Zhou, Yanlin Zhu, and Wenqi Wei. 2024. FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models.arXiv preprint arXiv:2401.10744(2024)

    2024 arXiv

  38. [46]

    Yongwei Zhou, Junwei Bao, Chaoqun Duan, Youzheng Wu, Xiaodong He, and Tiejun Zhao. 2022. Unirpg: Unified discrete reasoning over table and text as program generation.arXiv preprint arXiv:2210.08249(2022)

    2022 arXiv

  39. [47]

    Fengbin Zhu, Wenqiang Lei, Fuli Feng, Chao Wang, Haozhou Zhang, and Tat-Seng Chua. 2022. Towards complex document understanding by discrete reasoning. In Proceedings of the 30th ACM International Conference on Multimedia. 4857–4866

    2022

  40. [48]

    Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. TAT-QA: A question an- swering benchmark on a hybrid of tabular and textual content in finance.arXiv preprint arXiv:2105.07624(2021)

    2021 arXiv

  41. [49]

    Fengbin Zhu, Ziyang Liu, Fuli Feng, Chao Wang, Moxin Li, and Tat-Seng Chua

  42. [2021]

    Finqa: A dataset of numerical reasoning over financial data.arXiv preprint arXiv:2109.00122(2021)

    2021 arXiv

  43. [2024]

    Tat-llm: A specialized language model for discrete reasoning over tabular and textual data.arXiv preprint arXiv:2401.13223(2024)

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.