Pith. sign in

REVIEW 4 major objections 5 minor 42 references

LLM-Assisted Question-Answering on Technical Documents Using Structured Data-Aware Retrieval Augmented Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A structured-data-aware RAG pipeline reports 94–96% faithfulness and 87–93% answer relevancy on technical-document QA.

desk verdict Competent RAG engineering write-up, but the train/test overlap between the RAFT corpus and the evaluation set makes the headline faithfulness/relevancy numbers unreliable. read the letter →

arxiv 2506.23136 v1 pith:6TBXGIK6 submitted 2025-06-29 cs.CL

Shadman Sobhan , Mohammad Ariful Haque This is my paper · ORCID
classification cs.CL
keywords LLMHallucinationFine-tuningRAGStructureddataRAFTQuestionansweringDocumentunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a RAG pipeline can answer questions about technical documents containing tables, images, and scanned pages with a faithfulness score of 94% (RAGas) and 96% (DeepEval), and an answer-relevancy score of 87% (RAGas) and 93% (DeepEval). The wager is that the bottleneck in technical-document QA is not the generator but the retrieval: tables lose their structure when flattened into chunks, images never enter the vector store, and vector search can pull chunks that are similar but wrong. The pipeline therefore converts scanned pages to searchable text, turns tables into descriptive sentences and figures into VLM-generated captions, and reranks retrieved chunks with an LLM fine-tuned with RAFT to tell relevant from irrelevant contexts. If the claim is right, engineers could get faithful, source-grounded answers from equipment manuals and test reports without retraining the generator for each new document. The paper further claims that this architecture beats a general RAG pipeline on table-based questions and on questions whose answers lie outside the document, where the general pipeline hallucinates.

What carries the argument

The argument is carried by two coupled mechanisms. The first is a structured-data preprocessing layer: OCR converts scanned PDF pages into searchable text; a table detector locates tables, which are converted to HTML and then paraphrased by an LLM into row-by-row descriptive sentences; and a vision-language model turns figures into detailed captions. These descriptions sit alongside the original text so the vector store can retrieve them. The second mechanism is the reranker: after first-stage semantic search over fixed 512-token chunks retrieves the top 10 contexts, a Gemma-2-9b-it model fine-tuned with RAFT scores them and keeps the top 3. RAFT (Retrieval-Augmented Fine-Tuning) matters because the training data pairs each question with an oracle chunk carrying the answer and two distractor chunks, teaching the reranker to prefer contexts that actually answer and to stay silent when none does. The generator, also fine-tuned Gemma-2-9b-it, answers from the reranked context.

What would settle it

Run the full pipeline on a technical corpus that shares no text with the three manuals used to build the RAFT training data, with a fresh set of questions built from that corpus; if faithfulness and answer relevancy fall materially below the reported 94–96% and 87–93%, or if the system starts answering out-of-context questions with fabricated content, the central claim of general technical-document capability is refuted.

Watch

Extended reading notes

Core claim

The core discovery is that making a RAG system both structured-data aware and context-aware is what moves the metrics: first, tables and images are extracted and re-expressed as natural-language descriptions so that semantic search can index them; second, a reranker fine-tuned with RAFT on oracle-plus-distractor chunks learns to reject retrieved passages that are topically adjacent but do not contain the answer. With these two changes, the system reports a faithfulness of 0.94 (RAGas) and 0.96 (DeepEval) and an answer relevancy of 0.87 (RAGas) and 0.93 (DeepEval) on a 50-question test set drawn from technical and general documents. In the head-to-head comparison, the proposed pipeline answered a table-derived question correctly where the general pipeline stumbled, and on a question outside the document it answered 'This document doesn't contain the answer' while the general pipeline fabricated a response. The authors present this as evidence that the pipeline improves retrieval and generation on technical documents without requiring the generator to be retrained on each new corpus.

Load-bearing premise

The central claim rests on the 50-question test set being an independent measure of generalization, but the technical test questions come from transformer testing documents and the RAFT training data was generated from the same three testing manuals, so content overlap rather than retrieval quality could explain the high scores and the outside-context advantage.

Editorial extensions

If this is right

  • Questions about column-and-row table data in manuals can be answered because tables are re-expressed as descriptive sentences before embedding.
  • Scanned technical manuals become usable in RAG without manual text extraction, since OCR makes their pages searchable.
  • Questions whose answers are not in the supplied documents should produce an explicit 'not found' answer instead of a hallucinated one.
  • The reranker can be fine-tuned once on mixed technical and general chunks and then applied to new corpora without retraining the generator.
  • The reported scores suggest that entirely open-source components are sufficient for a technical-document QA pipeline in a resource-constrained setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same preprocessing could be tested on engineering drawings and datasheets by pairing the table detector with a stronger vision-language model, since the paper itself flags open-source VLM accuracy as a current limit.
  • Beyond the paper, the outside-context advantage could be stress-tested on documents whose topic differs entirely from the fine-tuning manuals, which would separate the RAFT reranker's 'do not answer' behavior from topic familiarity.
  • Beyond the paper, the table-to-sentence conversion suggests a testable extension: measure whether retrieval precision degrades for tables with ambiguous headers, merged cells, or multi-row entries, and whether retaining the HTML structure would preserve more information.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a Retrieval-Augmented Generation (RAG) pipeline for technical documents that handles scanned PDFs, tables, and images. The pipeline converts scanned documents via OCR, extracts tables and images into descriptive text, performs two-stage retrieval (vector similarity search followed by a Gemma-2-9b-it reranker fine-tuned with RAFT), and generates answers with a fine-tuned Gemma model. The authors report high RAGas and DeepEval scores on a 50-question self-created test set (94% and 96% faithfulness; 87% and 93% answer relevancy) and claim superiority over general RAG pipelines on table-based and outside-context questions, supported by a 5-question comparison. The paper provides code, notebooks, and fine-tuned weights for reproducibility.

Significance. If the headline results held, the pipeline would be a practically useful contribution for question answering over industrial technical manuals, especially in settings with scanned documents and tabular content. The paper is commendably concrete about the pipeline components and makes its artifacts available. However, the central empirical claims are not supported by the evaluation as designed: the RAFT training data and the technical test questions are drawn from overlapping source manuals, the comparison with general RAG uses only five questions, and all scores are LLM-as-judge point estimates without human validation or statistical testing. The claimed quantitative advantages could reflect content memorization rather than retrieval quality, so the significance of the work is currently unverified.

major comments (4)
  1. [§4.2.2 and §5.3.1] The training and test sets overlap at the document level. The RAFT corpus is generated from the Circuit Breaker Testing Guide, Power Cable Testing, and Power Transformer Testing manuals, while the technical portion of the test set is built from transformer-testing documents; all five technical questions shown in Table 6 are transformer-manual content. Since the reranker and the generator (Section 3.3, Table 3) are fine-tuned on chunks of these same manuals, the statement that no exact question was reused does not rule out passage-level memorization of the relevant sentences, tables, and paraphrased answers. A document-level split, such as fine-tuning on two manuals and testing on a held-out manual, is necessary before the reported faithfulness, relevancy, and outside-context advantages can be attributed to the retrieval pipeline rather than to content familiarity.
  2. [§5.4, Table 10] The claim of superiority over general RAG pipelines rests on only five questions, and the two headline sub-claims (table-based questions and outside-context questions) each depend on a single instance, Q1 and Q5 respectively. No confidence intervals or significance tests are provided. In addition, context precision and context recall for Q5 are marked N/A, so the average values of 1.0 in Table 10 are computed over only four questions. A single example in each category is insufficient evidence for a general architectural claim.
  3. [§5.3.2 and §5.3.3] The headline scores are produced by an LLM-as-a-judge (Llama-3.3-70B) on a self-created 50-question set, with no human validation, no variance estimate, and no comparison against an existing benchmark. The authors themselves note in Section 5.3 that evaluation is subjective and that different evaluator LLMs give different scores. Reporting point estimates such as 94% and 96% without confidence intervals or human-audited examples does not establish that these are stable properties of the pipeline.
  4. [§5.3.2] The reported context precision (0.94) and context recall (0.97) are computed under a setup where RAGas uses the same chunk size as the proposed pipeline, and the paper's own explanation states that because the correct context and retrieved context are similar in size, these metrics become very high whenever the retrieved chunk is correct. This makes the metrics heavily dependent on the authors' chunk-level ground-truth assignments rather than being independent measures of retrieval quality, and the resulting near-perfect scores should not be presented as strong evidence of retrieval superiority.
minor comments (5)
  1. [Table 2] The two example tables are rendered identically, so the claimed loss of table structure in the vector database is not visible to the reader; showing a flattened representation with lost borders would make the illustration clearer.
  2. [§5.3.2] The statement that RAGas 'evaluates exactly word by word' is inaccurate, since RAGas faithfulness and answer relevancy are based on LLM judgments rather than exact string matching.
  3. [§5.5] The sentence 'From earlier evaluations, it is proven that our model performs better' overstates what the evidence supports; a more cautious formulation such as 'suggests' or 'indicates' would be appropriate.
  4. [§4.2.2] The composition of the 1,040 chunks should be stated explicitly; the current text requires the reader to infer that 500 of the 800 additional chunks are technical to reconcile the reported totals of 740 technical and 300 general chunks.
  5. [General] Several reproducibility links appear as '[link]' placeholders rather than full URLs or DOIs; these should be completed in the references for the artifacts to be accessible.

Circularity Check

1 steps flagged · score 6.0 of 10

The reranker is fine-tuned on chunks from the same testing manuals that later supply the technical test questions, so the reported faithfulness and relevancy scores partly measure content memorization rather than retrieval quality.

  1. fitted input called prediction [Section 4.2.2 (Dataset Preparation); Section 5.3.1 (Test Set Formation); Table 6]
    "We initially selected three publicly available technical manuals: the Circuit Breaker Testing Guide [link], Power Cable Testing [link], and Power Transformer Testing [link]. ... From each chunk, exactly 1 question was generated by LLM. That chunk was the relevant chunk or oracle in that case. ... In our work, we created a test set of 50 questions taken from different types of documents(35 from technical documents and 15 from general documents). ... The test set contains questions, ground truth answers (taken exactly from the source documents) ..."

    The RAFT fine-tuning corpus is built from the same three testing manuals, including the Power Transformer Testing manual, and every training chunk is paired with an oracle label and a chain-of-thought answer. The technical part of the 50-question test set is then drawn from transformer-testing documents, and the five displayed test questions in Table 6 concern transformer hi-pot, TTR, winding continuity, and FRA content from that same manual. Saying that no exact question was reused does not remove the overlap: the reranker has already seen the relevant sentences, tables, and paraphrased answers, so it can rank or refuse contexts by memorized lexical overlap even for a novel surface question.

full rationale

There is no mathematical derivation chain to audit; the paper's central claim is an empirical evaluation claim. Its headline numbers are the 94% RAGas / 96% DeepEval faithfulness, the 87% / 93% relevancy scores, and the claimed superiority on table-based and out-of-context questions. That claim rests on the 50-question test set being an independent measure of retrieval and generation quality. The manuscript itself shows that the fine-tuning corpus and the technical test questions are drawn from the same transformer-testing source material, so the test is not independent. The paper's only safeguard is that training questions were not reused verbatim, but that does not prevent sentence-, table-, and answer-level memorization from inflating the metrics. A document-level or corpus-level split would be required before the scores can be attributed to the retrieval pipeline. No load-bearing self-citation was found; the self-citation in the literature review is not central to the empirical claim. Because the headline evaluation partly reduces to content the reranker was fitted on, the analysis is partially circular rather than an independent validation.

Assumptions & free parameters 13 free parameters · 7 assumptions · 0 invented entities

The central claim depends on several unverified processing assumptions: OCR quality, YOLO detection coverage, LLM table summarization fidelity, VLM image description accuracy, and the validity of LLM-based evaluation metrics. The most damaging is the assumed independence of the test set from the RAFT training corpus, since the technical manuals overlap. These assumptions are not backed by quantitative checks in the paper.

free parameters (13)
  • chunk_size_tokens = 512
    Fixed-size chunking in Section 3.2.1; no comparison of chunk sizes.
  • top_k = 10
    Stage 1 retrieval count in Section 3.2.1; chosen by hand without reported tuning.
  • top_n = 3
    Stage 2 reranker output count in Section 3.2.2; justified only as 'answer might lie within two chunks', not optimized.
  • num_questions_per_chunk = 1
    RAFT dataset hyperparameter in Section 4.6; no ablation of other values.
  • num_distract_docs = 2
    RAFT dataset hyperparameter in Section 4.6; controls ratio of irrelevant contexts during fine-tuning.
  • lora_r = 8
    LoRA rank in Section 4.6; selected without ablation.
  • lora_alpha = 32
    LoRA scaling factor in Section 4.6.
  • lora_dropout = 0.05
    LoRA dropout in Section 4.6.
  • learning_rate = 2e-05
    Training learning rate in Section 4.6.
  • num_train_epochs = 3
    Fine-tuning epoch count in Section 4.6.
  • max_seq_length = 2048
    SFTTrainer sequence length in Section 4.6.
  • max_new_tokens = 500
    Generation length cap in Section 4.6.
  • generation_temperature = 0.01
    Generation temperature in Section 4.6; set to make output more deterministic.
assumptions (7)
  • domain assumption Pytesseract OCR preserves document text accurately enough for downstream embedding.
    Section 3.1.1; no OCR accuracy evaluation is reported on the scanned document.
  • domain assumption Unstructured YOLO detects every table and image in the documents.
    Section 3.1.3; the claim that all tables and images are extracted is asserted without precision or recall numbers.
  • domain assumption LLM summarization of table HTML preserves every number and relationship in the original table.
    Section 3.1.3; this is contradicted by Table 4, where k is interpreted as thermal conductivity and 'Impregnated PPP' is qualified as 'likely Polypropylene'.
  • domain assumption VLM descriptions of images are accurate enough for question answering.
    Section 3.1.4; the authors note in Section 5.1.3 that VLMs can add unnecessary or imprecise detail.
  • domain assumption The 50-question test set is independent of the RAFT training data.
    Section 4.2.2 vs Section 5.3.1; technical test questions are drawn from the same manuals used to create training chunks, so the assumption is suspect.
  • domain assumption RAGAS and DeepEval LLM-based judgments are valid proxies for answer quality.
    Section 5.3; Llama-3.3-70B is used as an evaluator, but no human agreement study is provided.
  • domain assumption RAFT fine-tuning transfers from generator training to reranker scoring.
    Section 4.2; the paper applies RAFT to train a reranker, a change from the original generator-focused method, with no demonstration that this transfer works.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM-Assisted Question-Answering on Technical Documents Using Structured Data-Aware Retrieval Augmented Generation." pith.science (2026). https://pith.science/paper/6TBXGIK6

@misc{pith2026250623136,
  author       = {Pith},
  title        = {Pith review of: LLM-Assisted Question-Answering on Technical Documents Using Structured Data-Aware Retrieval Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6TBXGIK6}},
  note         = {Machine review of arXiv:2506.23136}
}
read the original abstract

Large Language Models (LLMs) are capable of natural language understanding and generation. But they face challenges such as hallucination and outdated knowledge. Fine-tuning is one possible solution, but it is resource-intensive and must be repeated with every data update. Retrieval-Augmented Generation (RAG) offers an efficient solution by allowing LLMs to access external knowledge sources. However, traditional RAG pipelines struggle with retrieving information from complex technical documents with structured data such as tables and images. In this work, we propose a RAG pipeline, capable of handling tables and images in documents, for technical documents that support both scanned and searchable formats. Its retrieval process combines vector similarity search with a fine-tuned reranker based on Gemma-2-9b-it. The reranker is trained using RAFT (Retrieval-Augmented Fine-Tuning) on a custom dataset designed to improve context identification for question answering. Our evaluation demonstrates that the proposed pipeline achieves a high faithfulness score of 94% (RAGas) and 96% (DeepEval), and an answer relevancy score of 87% (RAGas) and 93% (DeepEval). Comparative analysis demonstrates that the proposed architecture is superior to general RAG pipelines in terms of table-based questions and handling questions outside context.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 23 canonical work pages

  1. [1]

    Advances in Neural Information Processing Systems (2017)

    Vaswani, A.: Attention is all you need. Advances in Neural Information Processing Systems (2017)

    2017

  2. [2]

    arXiv preprint arXiv:2404.08700 (2024)

    Mousavi, S.M., Alghisi, S., Riccardi, G.: Is your llm outdated? benchmark- ing llms & alignment algorithms for time-sensitive knowledge. arXiv preprint arXiv:2404.08700 (2024)

    arXiv 2024

  3. [3]

    In: 2024 IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream), pp

    Reddy, G.P., Kumar, Y.P., Prakash, K.P.: Hallucinations in large language models (llms). In: 2024 IEEE Open Conference of Electrical, Electronic and Information Sciences (eStream), pp. 1–6 (2024). IEEE

    work page 2024

  4. [4]

    arXiv preprint arXiv:2409.05746 (2024)

    Banerjee, S., Agarwal, A., Singla, S.: Llms will always hallucinate, and we need to live with this. arXiv preprint arXiv:2409.05746 (2024)

    arXiv 2024

  5. [5]

    : Factuality challenges in the era of large language models and opportunities for fact-checking

    Augenstein, I., Baldwin, T., Cha, M., Chakraborty, T., Ciampaglia, G.L., Corney, D., DiResta, R., Ferrara, E., Hale, S., Halevy, A., et al. : Factuality challenges in the era of large language models and opportunities for fact-checking. Nature Machine Intelligence 6(8), 852–863 (2024)

    work page 2024

  6. [6]

    Advances in Neural Information Processing Systems 36, 50117–50143 (2023)

    Zhuang, Y., Yu, Y., Wang, K., Sun, H., Zhang, C.: Toolqa: A dataset for llm ques- tion answering with external tools. Advances in Neural Information Processing Systems 36, 50117–50143 (2023)

    work page 2023

  7. [7]

    In: International Conference on AI-generated Content, pp

    Huang, D., Wei, Z., Yue, A., Zhao, X., Chen, Z., Li, R., Jiang, K., Chang, B., Zhang, Q., Zhang, S., et al.: Dsqa-llm: domain-specific intelligent question answer- ing based on large language model. In: International Conference on AI-generated Content, pp. 170–180 (2023). Springer

    work page 2023

  8. [8]

    In: 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE), pp

    Mohammad, A.F., Clark, B., Hegde, R.: Large language model (llm) & gpt, a monolithic study in generative ai. In: 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE), pp. 383–388 (2023). IEEE 26

    work page 2023

Show all 42 references
  1. [9]

    Gaikwad, A., Rambhia, P., Pawar, S.: An extensive analysis between different language models: gpt-3, bert and macaw (2022)

    2022

  2. [10]

    Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie...

    2019

  3. [11]

    arXiv preprint arXiv:2205.11726 (2022)

    Artetxe, M., Du, J., Goyal, N., Zettlemoyer, L., Stoyanov, V.: On the role of bidirectionality in language model pre-training. arXiv preprint arXiv:2205.11726 (2022)

    2022 arXiv

  4. [12]

    Automatika: ˇ casopis za automatiku, mjerenje, elektroniku, raˇ cunarstvo i komunikacije 62(2), 226–238 (2021)

    ¨Oz¸ cift, A., Akarsu, K., Yumuk, F., S¨ oylemez, C.: Advancing natural language processing (nlp) applications of morphologically rich languages with bidirectional encoder representations from transformers (bert): an empirical case study for turkish. Automatika: ˇ casopis za a...

    2021

  5. [13]

    Machine Learning 111(3), 917–935 (2022)

    Wu, X., Xia, Y., Zhu, J., Wu, L., Xie, S., Qin, T.: A study of bert for context-aware neural machine translation. Machine Learning 111(3), 917–935 (2022)

    2022

  6. [14]

    Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)

    2018

  7. [15]

    : Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. : Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)

    2019

  8. [16]

    : Language models are few-shot learners

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. : Language models are few-shot learners. Advances in neural information processing systems 33, 1877–1901 (2020)

    2020

  9. [17]

    arXiv preprint arXiv:2308.14149 (2023)

    Gao, K., He, S., He, Z., Lin, J., Pei, Q., Shao, J., Zhang, W.: Examining user- friendly and open-sourced large gpt models: A survey on language, multimodal, and scientific gpt models. arXiv preprint arXiv:2308.14149 (2023)

    2023 arXiv

  10. [18]

    In: Asia-Pacific Web (APWeb) and Web-Age Information Management (W AIM) Joint International Conference on Web and Big Data, pp

    Chen, C., Wang, B., Lin, Y.: A systematic mapping study of llm applications in mobile device research. In: Asia-Pacific Web (APWeb) and Web-Age Information Management (W AIM) Joint International Conference on Web and Big Data, pp. 163–174 (2024). Springer

    2024

  11. [19]

    In: 2024 IEEE 21st International Conference on Smart Communities: Improving Quality of Life Using AI, Robotics and IoT (HONET), pp

    Chkirbene, Z., Hamila, R., Gouissem, A., Devrim, U.: Large language models (llm) in industry: A survey of applications, challenges, and trends. In: 2024 IEEE 21st International Conference on Smart Communities: Improving Quality of Life Using AI, Robotics and IoT (HONET), pp. 2...

    2024

  12. [20]

    In: 2020 12th International Conference on Information Technology and Electrical Engineering (ICITEE), pp

    Yunianto, I., Permanasari, A.E., Widyawan, W.: Domain-specific contextualized embedding: a systematic literature review. In: 2020 12th International Conference on Information Technology and Electrical Engineering (ICITEE), pp. 162–167 (2020). IEEE

    2020

  13. [21]

    Fundamental Research 1(6), 831–833 (2021)

    Zhang, M., Li, J.: A commentary of gpt-3 in mit technology review 2021. Fundamental Research 1(6), 831–833 (2021)

    2021

  14. [22]

    Journal of Medical Internet Research 26, 52758 (2024)

    Matsui, K., Utsumi, T., Aoki, Y., Maruki, T., Takeshima, M., Takaesu, Y.: Human-comparable sensitivity of large language models in identifying eligible studies through title and abstract screening: 3-layer strategy using gpt-3.5 and gpt-4 for systematic reviews. Journal of Med...

    2024

  15. [23]

    arXiv preprint arXiv:1611.01578 (2016)

    Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)

    2016 arXiv

  16. [24]

    arXiv preprint arXiv:1606.01549 (2016)

    Dhingra, B., Liu, H., Yang, Z., Cohen, W.W., Salakhutdinov, R.: Gated-attention readers for text comprehension. arXiv preprint arXiv:1606.01549 (2016)

    2016 arXiv

  17. [25]

    arXiv preprint arXiv:1908.01841 (2019)

    Olabiyi, O., Mueller, E.T.: Dlgnet: a transformer-based model for dialogue response generation. arXiv preprint arXiv:1908.01841 (2019)

    2019 arXiv

  18. [26]

    Advances in neural information processing systems 33, 9459–9474 (2020)

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K¨ uttler, H., Lewis, M., Yih, W.-t., Rockt¨ aschel, T.,et al.: Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, 9459–9474 (2020)

    2020

  19. [27]

    arXiv preprint arXiv:2310.01469 (2023)

    Yao, J.-Y., Ning, K.-P., Liu, Z.-H., Ning, M.-N., Liu, Y.-Y., Yuan, L.: Llm lies: Hallucinations are not bugs, but features as adversarial examples. arXiv preprint arXiv:2310.01469 (2023)

    2023 arXiv

  20. [28]

    arXiv preprint arXiv:2407.03282 (2024)

    Ji, Z., Chen, D., Ishii, E., Cahyawijaya, S., Bang, Y., Wilie, B., Fung, P.: Llm internal states reveal hallucination risk faced with a query. arXiv preprint arXiv:2407.03282 (2024)

    2024 arXiv

  21. [29]

    In: European Semantic Web Conference, pp

    Martino, A., Iannelli, M., Truong, C.: Knowledge injection to counter large lan- guage model (llm) hallucination. In: European Semantic Web Conference, pp. 182–185 (2023). Springer

    2023

  22. [30]

    arXiv preprint arXiv:2403.10446 (2024)

    Li, J., Yuan, Y., Zhang, Z.: Enhancing llm factual accuracy with rag to counter hallucinations: A case study on domain-specific queries in private knowledge- bases. arXiv preprint arXiv:2403.10446 (2024)

    2024 arXiv

  23. [31]

    Journal of Machine Learning Research24(251), 1–43 28 (2023)

    Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi- Yu, J., Joulin, A., Riedel, S., Grave, E.: Atlas: Few-shot learning with retrieval augmented language models. Journal of Machine Learning Research24(251), 1–43 28 (2023)

    2023

  24. [32]

    arXiv preprint arXiv:2301.12652 (2023)

    Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., Yih, W.-t.: Replug: Retrieval-augmented black-box language models. arXiv preprint arXiv:2301.12652 (2023)

    2023 arXiv

  25. [33]

    Transactions of the Association for Computational Linguistics 11, 1–17 (2023)

    Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., Nanayakkara, S.: Improving the domain adaptation of retrieval augmented gen- eration (rag) models for open domain question answering. Transactions of the Association for Computational Linguistics 11, 1–17 (2023)

    2023

  26. [34]

    arXiv preprint arXiv:2004.02349 (2020)

    Herzig, J., Nowak, P.K., M¨ uller, T., Piccinno, F., Eisenschlos, J.M.: Tapas: Weakly supervised table parsing via pre-training. arXiv preprint arXiv:2004.02349 (2020)

    2020 arXiv

  27. [35]

    In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp

    Xu, Y., Li, M., Cui, L., Huang, S., Wei, F., Zhou, M.: Layoutlm: Pre-training of text and layout for document image understanding. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1192–1200 (2020)

    2020

  28. [36]

    IEEE Transactions on Neural Networks and Learning Systems 34(11), 8174–8194 (2022)

    Song, Z., Yang, X., Xu, Z., King, I.: Graph-based semi-supervised learning: A comprehensive review. IEEE Transactions on Neural Networks and Learning Systems 34(11), 8174–8194 (2022)

    2022

  29. [37]

    arXiv preprint arXiv:1801.06146 (2018)

    Howard, J., Ruder, S.: Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146 (2018)

    2018 arXiv

  30. [38]

    : Lora: Low-rank adaptation of large language models

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al. : Lora: Low-rank adaptation of large language models. ICLR 1(2), 3 (2022)

    2022

  31. [39]

    arXiv preprint arXiv:2403.14608 (2024)

    Han, Z., Gao, C., Liu, J., Zhang, J., Zhang, S.Q.: Parameter-efficient fine- tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608 (2024)

    2024 arXiv

  32. [40]

    In: International Conference on Machine Learning, pp

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Ges- mundo, A., Attariyan, M., Gelly, S.: Parameter-efficient transfer learning for nlp. In: International Conference on Machine Learning, pp. 2790–2799 (2019). PMLR

    2019

  33. [41]

    In: First Conference on Language Modeling (2024)

    Zhang, T., Patil, S.G., Jain, N., Shen, S., Zaharia, M., Stoica, I., Gonzalez, J.E.: Raft: Adapting language model to domain specific rag. In: First Conference on Language Modeling (2024)

    2024

  34. [42]

    In: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pp

    Es, S., James, J., Anke, L.E., Schockaert, S.: Ragas: Automated evaluation of retrieval augmented generation. In: Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pp. 150–158 (2024) 29

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.