Pith. sign in

REVIEW 4 major objections 7 minor 2 cited by

Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning

T0 review · 4 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LSIM claims legal LLM responses improve when retrieval uses a predicted fact-rule chain, not just semantic similarity, and reports consistent gains over embedding-based RAG baselines in automated and human evaluation.

desk verdict A coherent logical-RAG framework for legal QA, but the 'logical' training signal is actually semantic, so the central mechanism claim is not supported by the experiments. read the letter →

arxiv 2502.07912 v1 pith:P7LBQK4I submitted 2025-02-11 cs.CL

Rujing Yao , Yang Wu , Chenghao Wang , Jingwei Xiong , Fang Wang , Xiaozhong Liu This is my paper · ORCID
classification cs.CL
keywords legalquestionansweringretrieval-augmentedgenerationfact-rulechainreinforcementlearningdeepstructuredsemanticmodellogicalstructurein-contexthallucinationmitigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LSIM claims that legal question answering with LLMs gets more accurate when retrieval is driven by a predicted logical structure—a fact-rule chain linking the facts of the case to the legal rules that apply—in addition to semantic similarity. The paper builds a supervised pipeline that predicts the chain with reinforcement learning, retrieves similar lawyer-answered questions with a trainable DSSM that scores candidates on both chain and question embeddings, and then feeds question, chain, and retrieved pairs to the LLM as in-context examples. On a real-world criminal-law QA set of over 16,000 questions, the framework beats the best embedding retriever by 2.38 METEOR and 2.48 ROUGE-1 points with LLaMA-3-8B and is rated higher by three lawyers on accuracy, specificity, and adoptability. The point of the claim is practical: legal RAG systems should treat logical coherence as a retrievable signal, not a post-hoc explanation.

What carries the argument

The carrying mechanism is the fact-rule chain, a sequence of up to four nodes from a legal-domain graph in which fact nodes (for example, prior conviction, evidence, possession) and rule nodes (for example, sentencing guidelines, evidence admissibility, criminal law) alternate. The chain is extracted from each question-answer pair by an LLM using the graph; a REINFORCE-trained MLP policy then predicts the answer-side chain from the question-side chain, with a reward of 1 at each step for choosing a node that appears in the gold answer chain. The DSSM retriever embeds the question and its predicted chain with BERT, concatenates the two vectors, and scores each candidate question with an MLP; a margin ranking loss, trained on LLM-assigned 0-5 relevance labels, makes the model prefer the most relevant candidate. The top-3 retrieved question-answer pairs are then concatenated with the predicted chain into the prompt for the LLM, which generates the final answer.

What would settle it

Replace the predicted fact-rule chain in LSIM's retriever with random chains drawn from the same legal graph, keeping all other components and training labels the same; if METEOR stays near 21.00 instead of falling toward the 19.10 observed when the logical module is removed, the chains are not carrying the gain. A complementary check is to retrain the DSSM using relevance labels from independent lawyers rather than the LLM and see whether the margin over text-embedding-3-large survives.

Watch

Extended reading notes

Core claim

The paper's central claim is that logical structure can be learned and injected into the retrieval step of legal RAG, and that doing so improves answer quality beyond what semantic embeddings achieve. LSIM represents the logical structure of a legal question as a fact-rule chain of up to four nodes drawn from a legal-domain graph; the same extraction is applied to lawyer answers to build training targets. A REINFORCE-trained policy network predicts the answer-side chain from the question-side chain, and a DSSM retriever ranks candidate questions by concatenating BERT embeddings of the question and of its predicted chain, trained with margin ranking loss against LLM-generated relevance scores. In the reported experiments on 648 held-out questions, the full pipeline reaches METEOR 21.00 with LLaMA-3-8B, against 18.62 for text-embedding-3-large, and similar gains appear with LLaMA-2-13B and GPT-4o; human evaluation by three legal professionals favors LSIM on accuracy, specificity, and adoptability. The authors read these results as evidence that both modules contribute, and that the framework yields more professional, case-specific legal answers.

Load-bearing premise

The load-bearing premise, asserted in Sections 3.1.1 and 3.2, is that the LLM-extracted fact-rule chains and the LLM-generated 0-5 relevance scores are accurate enough to serve as ground truth for training; the paper reports no expert validation of either signal.

Editorial extensions

If this is right

  • If LSIM's claim holds, retrieval for legal QA should index a predicted fact-rule chain alongside the text, because chain-aware scoring outperforms every semantic embedding baseline tested by a clear margin.
  • The predicted chain becomes an inspectable artifact: users and lawyers can see which facts and rules drove the retrieval and therefore partly audit why the LLM gave the answer it did.
  • Because gains appear with three different generators, the improvement is attributed to the retrieval infrastructure rather than a lucky interaction with one LLM.
  • The paper's own limitation section warns that the framework's ceiling depends on database coverage and single-turn setup; in low-resource legal domains where few similar cases exist, the logical chain cannot compensate for absent retrieved context.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the logical chain's causal role would hold the retriever and labels fixed and swap the RL-predicted chain for a random chain sampled from the same graph; if METEOR does not drop toward the 19.10 observed without the logical module, the chains are a correlate rather than the active ingredient.
  • The authors list healthcare and finance as future domains; a natural extension is to build fact-rule graphs from regulatory texts such as tax codes and insurance rules, where the fact-rule distinction is often explicit and the extracted chains can be checked by domain experts.
  • If LLM relevance labels are the only retrieval supervision, a larger human-labeled relevance set could reveal how much of LSIM's gain comes from the logical structure versus from supervised retraining on the same corpus; the semantic-only ablation at 19.10 METEOR already suggests both factors are at work.
  • For deployment, multi-turn follow-up could re-predict or refine the fact-rule chain after the user's first answer, turning the single-shot chain into an interactive legal diagnostic.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes LSIM, a retrieval-augmented generation framework for legal question answering that combines (i) an RL-trained policy predicting a 'fact-rule chain' for each question, (ii) a DSSM-based retriever that concatenates semantic and logical features, and (iii) in-context learning with retrieved lawyer QA pairs. Experiments on a newly collected JUSTIA criminal-law QA dataset (16,190 questions) compare LSIM against BM25 and ten embedding baselines with three base LLMs (LLaMA-2-13B, LLaMA-3-8B, GPT-4o), reporting consistent improvements on ROUGE, METEOR, and BERTScore, plus a small human evaluation (three legal professionals) favoring LSIM. The central claim is that integrating logical structure into retrieval significantly enhances the accuracy and reliability of legal LLM responses.

Significance. If the central claim holds, the paper makes a useful contribution by showing that injecting a trainable logical-structure signal into retrieval can improve legal QA over semantic-only RAG. The authors provide a real-world dataset, release code and data, and include an external human evaluation, which are strengths. The main significance currently is limited by the lack of evidence that the learned 'logical structure' is in fact logical: the reward in Eq. (5) ignores order, the retrieval supervision is LLM event-similarity labels, and no human validation of the extracted chains or relevance scores is reported. The consistent improvements across three base LLMs and the ablation study are encouraging, but the mechanism attributed to logical coherence remains unsubstantiated.

major comments (4)
  1. [Sec. 3.1.2, Eq. (5)] The reward rt = 1 iff nt ∈ Cai assigns full credit for any predicted node that belongs to the gold answer chain, regardless of the order in which nodes are generated or the transitions between them. The REINFORCE objective in Eq. (4) therefore trains a policy over node membership rather than over a chain, and the text's claim that the policy learns 'logical relationships between nodes' and produces a 'logically coherent and accumulative' chain is not supported by the training signal. A policy that outputs the gold nodes in any arbitrary order receives identical cumulative reward. This is load-bearing for the paper's central claim that logical structure, not a bag of answer-chain terms, drives the gains; please either modify the reward to incorporate order/transition constraints or provide evidence that the predicted chains are order-correct.
  2. [Sec. 3.2 and Appendix A.2] The DSSM retriever is trained with a margin-ranking loss whose only supervision is the LLM-assigned score of 'the similarity of ... the events described in each legal question' (Appendix A.2). This is a semantic event-similarity label, not a legal logical-relevance label. Because the logical-structure features hCqi are concatenated into the query representation, the reported gains could arise from supervised query expansion with answer-side terms rather than from logical coherence. The ablation in Table 7 is consistent with this concern: LSIM w/o LS still improves over text-embedding-3-large by 0.48 METEOR, while LSIM w/o SI is only 0.15 above that baseline. To support the logical-integration claim, report retrieval-level metrics (e.g., Recall@K, MRR) and ablate the specific contribution of predicted-chain features versus the supervised ranking objective.
  3. [Sec. 4.3 and Sec. 4.4] The abstract states that LSIM 'significantly enhances accuracy and reliability,' and Sec. 4.4 repeatedly reports 'best performance,' but all results are given as point averages of three runs with no standard deviations, confidence intervals, or significance tests. The gaps in Table 3 (METEOR 21.00 vs. 18.62 for text-embedding-3-large) and Table 7 (21.00 vs. 19.10 for LSIM w/o LS) may be meaningful, yet without variance information the word 'significantly' is unsupported. Please report per-run results and perform paired significance tests over the 648 test instances (e.g., bootstrap or paired t-test) for the main comparisons and ablations.
  4. [Sec. 3.1.1 and Sec. 3.2] Both the ground-truth fact-rule chains and the retrieval relevance labels are generated by LLMs without expert validation: the chains are extracted via an LLM prompt over a graph inherited from Wu et al. (2024), and the relevance scores are LLM-generated on a 0-5 scale. This creates a potential circularity, because LLM-generated labels supervise an LLM-based pipeline, and any bias in those labels (e.g., toward surface similarity) is inherited by the retriever and the final generator. Please provide a human audit of a sample of extracted chains and relevance scores, or an independent validation of the logical structure (e.g., against a legal ontology), to break the loop and substantiate that the supervision signal is genuinely logical.
minor comments (7)
  1. [Sec. 4.1] The number '3,283' appears twice in the data-split description, but the sum of training (2,590) and testing (648) in Table 1 is 3,238; please correct this typo.
  2. [Sec. 4.4] The improvement percentages (e.g., 3.46% on METEOR for LLaMA-2-13B) are presented as relative gains but appear to be absolute point differences (17.09 to 20.55 is an absolute increase of 3.46 points, which is 20.2% relative); please clarify which calculation is intended.
  3. [Sec. 3.3, Eq. (12)] The logical structure passed to the LLM is written as Cqi, but Sec. 3.1.2 predicts the answer-side chain; specify whether the input is the question chain, the predicted answer chain, or both, because this affects the interpretation of the in-context prompt.
  4. [Throughout] There are several typos and grammar issues: 'stabilihed' in Sec. 2.3, 'empolyed' in Sec. 3.1.2, 'Tabel 6' in Sec. 4.5, and 'Tabel 7' in Sec. 4.6; these should be corrected.
  5. [References] The BERT citation is given as 'Kenton and Toutanova, 2019' but the standard attribution is Devlin et al. (2019); please update the reference.
  6. [Table 5] The human evaluation reports averages for three professionals but does not state how many questions were rated or provide inter-annotator agreement (e.g., Krippendorff's alpha); please add this information to support the reliability of the human scores.
  7. [Appendix A.2] The prompt instructs the LLM to score 'the similarity of ... the events described' in the legal questions; this is explicitly a semantic criterion. The paper should discuss why this is an appropriate proxy for legal logical relevance, given that the DSSM is presented as a logical-semantic integration component.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: the reported gains are measured against held-out lawyer answers and blinded expert ratings, so LLM-generated chain/relevance labels do not force the conclusion. Only a minor, non-load-bearing self-citation (Wu et al. 2024) supplies the fact-rule graph construction.

  1. other [Section 3.1.1 (Logical structure extraction)]
    "Following Wu et al. (2024), a fact-rule graph G is constructed using the LLM. Assume our training set T = {(qi, ai)}Ni=1 contains N instances, where qi is the i-th question, and ai is the real lawyer's answer to qi. For each question-answer pair (qi, ai), the LLM, guided by a tailored prompt, is employed to extract the most relevant fact-rule chain from graph G."

    Three of the six present authors (Yang Wu, Chenghao Wang, Xiaozhong Liu) are also authors of the cited Wu et al. (2024), so the fact-rule graph and the LLM extraction procedure that define the paper's central 'logical structure' are inherited from the authors' own prior work, with no expert validation of the extracted chains in this paper. This is a self-citation, but it is not load-bearing circularity: the central claim (LSIM improves answer quality) is tested externally via ROUGE/METEOR/BERTScore against held-out real lawyer answers and by three blinded legal professionals (Section 4.2), and the ablation in Table 7 independently attributes part of the gain to the logical-structure module.

full rationale

Walking the derivation chain: (1) The RL policy (Section 3.1.2) predicts answer-side fact-rule nodes for each question; its reward (Eq. 5, rt = 1 if nt is in Cai) uses chains extracted only from the training split (2,590 samples), and the policy is then applied to held-out test questions and database items, so no test answer chain enters training. (2) The DSSM (Section 3.2) is trained on margin-ranking labels that are LLM-assigned event-similarity scores computed for training queries against the disjoint 12,952-item database; at inference the trained DSSM ranks database items for unseen test queries, and the top-K results feed generation. (3) The final answer (Eq. 12) is evaluated against real lawyer answers with ROUGE/METEOR/BERTScore (Tables 2-4) and by three blinded legal experts (Table 5), criteria that appear nowhere in the training labels. Hence no prediction reduces to an input by construction, and the paper is self-contained against external benchmarks. The legitimate concerns are validity issues, not circularity: the relevance labels (Appendix A.2) measure semantic event similarity while the paper claims integration of logical features; Eq. 5's reward checks only set membership, so the 'logically coherent and accumulative' chains are not actually supervised for order or transitions; and the fact-rule graph is adopted from the authors' prior work (Wu et al. 2024) without expert validation. These undermine the 'logical structure' attribution and the word 'significantly' (no significance tests are reported for the 648 test items), but they do not make the measured improvements equivalent to the system's inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The claim rests on hand-set modeling choices (chain cap, K, margin alpha, RL horizon) and on LLM-generated labels being accepted as gold for both chain supervision and retrieval supervision. No new physical entities are introduced; the fact-rule chain is a representational construct inherited from the authors' prior work and is listed with its testable handle.

free parameters (4)
  • Margin alpha in DSSM ranking loss = not reported
    Eq. 10 defines the margin ranking loss with hyperparameter alpha, but the paper never states its value, even though the trained retriever's behavior depends on it.
  • Maximum fact-rule chain length t = 4
    Footnote 2 in Sec 3.1.1 caps the chain at four nodes based on the authors' own statistics; this bounds the logical representation and the RL action space.
  • Number of retrieved examples K = 3
    Sec 3.3 sets K to 3 citing prompt length limits; the final answer quality depends on this choice and it is not ablated.
  • RL horizon and policy architecture = not reported
    Sec 3.1.2 stops sampling when a maximum step count T is reached, but T, the MLP hidden sizes, and the action space are never specified.
assumptions (4)
  • domain assumption The fact-rule graph G from Wu et al. (2024) is a valid representation of legal reasoning, and LLM-extracted 1-4 node chains faithfully summarize question and answer logic.
    Sec 3.1.1 inherits G and the extraction prompt from the authors' prior work; no expert validation or agreement statistics are given for chain quality.
  • domain assumption LLM-generated relevance scores (0-5) are accurate enough to serve as ground truth for training the retriever.
    Sec 3.2: 'The annotated relevance scores are generated by the LLM.' This assumes the annotating LLM's pairwise relevance judgments are reliable, with no check against lawyer judgments reported.
  • ad hoc to paper Membership reward rt = 1 iff nt is in C_ai implies logical coherence of the assembled chain.
    Eq. 5 defines reward on node-set membership only; the paper claims REINFORCE training 'ensuring that the generated fact-rule chain is logically coherent' (Sec 3.1.2). The connection between membership reward and sequential coherence is asserted, not shown.
  • standard math Standard supervised learning and policy-gradient assumptions (REINFORCE convergence, margin ranking loss behavior).
    Standard ML background; not the load-bearing risk here.
invented entities (1)
  • Fact-rule chain (logical structure) independent evidence
    purpose: A 1-4 node sequence of facts and legal rules extracted from each question and answer, used to condition retrieval and generation.
    The chain is testable through the reported ablations and human evaluation, which attribute part of the gain to the logical structure, but the construct itself inherits its graph from the authors' prior work (Wu et al. 2024) and is not externally validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning." pith.science (2026). https://pith.science/paper/P7LBQK4I

@misc{pith2026250207912,
  author       = {Pith},
  title        = {Pith review of: Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P7LBQK4I}},
  note         = {Machine review of arXiv:2502.07912}
}
read the original abstract

Large Language Models (LLMs) have achieved impressive results across numerous domains, yet they experience notable deficiencies in legal question-answering tasks. LLMs often generate generalized responses that lack the logical specificity required for expert legal advice and are prone to hallucination, providing answers that appear correct but are unreliable. Retrieval-Augmented Generation (RAG) techniques offer partial solutions to address this challenge, but existing approaches typically focus only on semantic similarity, neglecting the logical structure essential to legal reasoning. In this paper, we propose the Logical-Semantic Integration Model (LSIM), a novel supervised framework that bridges semantic and logical coherence. LSIM comprises three components: reinforcement learning predicts a structured fact-rule chain for each question, a trainable Deep Structured Semantic Model (DSSM) retrieves the most relevant candidate questions by integrating semantic and logical features, and in-context learning generates the final answer using the retrieved content. Our experiments on a real-world legal QA dataset-validated through both automated metrics and human evaluation-demonstrate that LSIM significantly enhances accuracy and reliability compared to existing methods.

Figures

Figures reproduced from arXiv: 2502.07912 by the authors.

Figure 1
Figure 1. An illustration of our model. to potential legal risks and financial losses (Dahl et al., 2024). Retrieval-Augmented Generation (RAG) (Lewis et al., 2020; Li et al., 2024) has emerged as a promising approach to mitigate hal￾lucinations and improve response accuracy by re￾trieving relevant legal cases and provisions as con￾textual knowledge sources for LLMs. Nevertheless, existing RAG methods primarily focus on seman… view at source ↗
Figure 2
Figure 2. The overall framework of LSIM. The network consists of three modules: learnable fact-rule chain, [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustrative example 1. The sentences highlighted in red indicate remaining silent, while those highlighted [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Illustrative example 2. The sentences highlighted in blue indicate responses from different models [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Illustrative example 3. The sentences highlighted in blue indicate responses from different models [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Illustrative examples of the fact-rule chains. The red text represents the fact-rule chain for the question, [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DIGMAPPER: A Modular System for Automated Geologic Map Digitization

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A Dockerized, workflow-orchestrated deep-learning system automatically digitizes geologic maps into georeferenced vector features, with reported high accuracy on easy maps but degraded performance on visually complex ones.

  2. Active Domain Knowledge Acquisition with 100-Dollar Budget: Enhancing LLMs via Cost-Efficient, Expert-Involved Interaction in Sensitive Domains

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    A budget-aware framework (PU-ADKA) selects which domain expert an LLM should query under a fixed $100 budget, improving specialized-domain answers at low cost.

Reference graph

Works this paper leans on

46 extracted references · 29 canonical work pages · cited by 2 Pith papers

  1. [1]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    2005

  2. [2]

    Razieh Baradaran, Razieh Ghiasi, and Hossein Amirkhani. 2022. A survey on machine reading comprehension systems. Natural Language Engineering, 28(6):683–732

    work page 2022

  3. [3]

    Andong Chen, Feng Yao, Xinyan Zhao, Yating Zhang, Changlong Sun, Yun Liu, and Weixing Shen. 2023. Equals: A real-world dataset for legal question answering via reading chinese laws. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, pages 71--80

    work page 2023

  4. [4]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216

    arXiv 2024

  5. [5]

    Inyoung Cheong, King Xia, KJ Kevin Feng, Quan Ze Chen, and Amy X Zhang. 2024. (a) i am not a lawyer, but...: Engaging legal experts towards responsible llm policies for legal advice. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 2454--2469

    2024

  6. [6]

    Ashish Chouhan and Michael Gertz. 2024. Lexdrafter: Terminology drafting for legislative documents using retrieval augmented generation. arXiv preprint arXiv:2403.16295

    work page Pith review arXiv 2024

  7. [7]

    Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li Yuan. 2023. Chatlaw: Open-source legal large language model with integrated external knowledge bases. arXiv preprint arXiv:2306.16092

    arXiv 2023

  8. [8]

    Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. 2024. Large legal fictions: Profiling legal hallucinations in large language models. arXiv preprint arXiv:2401.01301

    arXiv 2024

Show all 46 references
  1. [9]

    Zhangyin Feng, Xiaocheng Feng, Dezhi Zhao, Maojin Yang, and Bing Qin. 2024. Retrieval-generation synergy augmented large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11661--11665. IEEE

    2024

  2. [10]

    Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor OK Li. 2018. Search engine guided neural machine translation. In Proceedings of the AAAI Conference on Artificial Intelligence

    2018

  3. [11]

    Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management, pages 2333--2338

    2013

  4. [12]

    Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen Tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP, pages 6769--6781

    2020

  5. [13]

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171--4186

    2019

  6. [14]

    Gangwoo Kim, Sungdong Kim, Byeongguk Jeon, Joonsuk Park, and Jaewoo Kang. 2023. Tree of clarifications: Answering ambiguous questions with retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, page...

    2023

  7. [15]

    Veronica Latcinnik and Jonathan Berant. 2020. Explaining question answering models through text generation. In arXiv:2004.05569

    2020 arXiv

  8. [16]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

    2020

  9. [17]

    Jiarui Li, Ye Yuan, and Zehua Zhang. 2024. Enhancing llm factual accuracy with rag to counter hallucinations: A case study on domain-specific queries in private knowledge-bases. arXiv preprint arXiv:2403.10446

    2024 arXiv

  10. [18]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  11. [19]

    Jiongnan Liu, Jiajie Jin, Zihan Wang, Jiehan Cheng, Zhicheng Dou, and Ji-Rong Wen. 2023 a . Reta-llm: A retrieval-augmented large language model toolkit. arXiv preprint arXiv:2306.05212

    2023 arXiv

  12. [20]

    Shangqing Liu, Yu Chen, Xiaofei Xie, Jing Kai Siow, and Yang Liu. 2020. Retrieval-augmented generation for code summarization via hybrid gnn. In International Conference on Learning Representations

    2020

  13. [21]

    Yifei Liu, Yiquan Wu, Yating Zhang, Changlong Sun, Weiming Lu, Fei Wu, and Kun Kuang. 2023 b . Ml-ljp: Multi-law aware legal judgment prediction. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1023--1034

    2023

  14. [22]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  15. [23]

    Antoine Louis, Gijs van Dijck, and Gerasimos Spanakis. 2024. Interpretable long-form legal question answering with retrieval-augmented large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 22266--22275

    2024

  16. [24]

    Yuning Mao, Pengcheng He, Xiaodong Liu, Yelong Shen, Jianfeng Gao, Jiawei Han, and Weizhu Chen. 2020. Generation-augmented retrieval for open-domain question answering. arXiv preprint arXiv:2009.08553

    2020 arXiv

  17. [25]

    Zhang, Hui Chia, Kabir Manandhar Shrestha, Rohit Kumar Gupta, Saket Khandelwal, Jeannie Paterson, Timothy Baldwin, and Daniel Beck

    Meladel Mistica, Geordie Z. Zhang, Hui Chia, Kabir Manandhar Shrestha, Rohit Kumar Gupta, Saket Khandelwal, Jeannie Paterson, Timothy Baldwin, and Daniel Beck. 2020. Information extraction from legal documents: A study in the context of common law court judgements. In Proceedi...

    2020

  18. [26]

    Jack Mumford, Katie Atkinson, and Trevor Bench-Capon. 2023. Combining a legal knowledge model with machine learning for reasoning with legal cases. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, pages 167--176

    2023

  19. [27]

    Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022 a . Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864--1874

    2022

  20. [28]

    Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, et al. 2022 b . Large dual encoders are generalizable retrievers. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Process...

    2022

  21. [29]

    Md Rizwan Parvez, Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2021. Retrieval augmented code generation and summarization. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2719--2734

    2021

  22. [30]

    Stephen E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR’94: Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrie...

    1994

  23. [31]

    Anna Rogers, PictureMatt Gardner, and PictureIsabelle Augenstein. 2023. Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprehension. ACM Computing Surveys, 55(10):1--45

    2023

  24. [32]

    Cheol Ryu, Seolhwa Lee, Subeen Pang, Chanyeol Choi, Hojun Choi, Myeonggee Min, and Jy-Yong Sohn. 2023. Retrieval-based evaluation for llms: A case study in korean legal qa. In Proceedings of the Natural Legal Language Processing Workshop 2023, pages 132--137

    2023

  25. [33]

    Mehmet Fatih Sert, Engin Yıldırım, and İrfan Haşlak. 2022. Using artificial intelligence to predict decisions of the turkish constitutional court. Social Science Computer Review, 40(6):1416--1435

    2022

  26. [34]

    Sanat Sharma, David Seunghyun Yoon, Franck Dernoncourt, Dewang Sultania, Karishma Bagga, Mengjiao Zhang, Trung Bui, and Varun Kotte. 2024. Retrieval augmented generation for domain-specific question answering. arXiv preprint arXiv:2404.14760

    2024 arXiv

  27. [35]

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, and Tao Yu. 2023. One embedder, any task: Instruction-finetuned text embeddings. In Findings of the Association for Computational Linguistics: ACL 2023, pa...

    2023

  28. [36]

    Zhongxiang Sun, Jun Xu, Xiao Zhang, Zhenhua Dong, and Ji-Rong Wen. 2023. Law article-enhanced legal case matching: A causal learning approach. In Proceedings of ACM SIGIR, pages 1549--1558

    2023

  29. [37]

    Yu Wang, Vijay Srinivasan, and Hongxia Jin. 2022. A new concept of knowledge based question answering ( KBQA ) system for multi-hop reasoning. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno...

    2022

  30. [38]

    Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229--256

    1992

  31. [39]

    Nirmalie Wiratunga, Ramitha Abeyratne, Lasal Jayawardena, Kyle Martin, Stewart Massie, Ikechukwu Nkisi-Orji, Ruvan Weerasinghe, Anne Liret, and Bruno Fleisch. 2024. Cbr-rag: Case-based reasoning for retrieval augmented generation in llms for legal question answering. arXiv pre...

    2024 arXiv

  32. [40]

    Yang Wu, Chenghao Wang, Ece Gumusel, and Xiaozhong Liu. 2024. Knowledge-infused legal wisdom: Navigating llm consultation through the lens of diagnostics and positive-unlabeled reinforcement learning. In Findings of the Association for Computational Linguistics: ACL 2024

    2024

  33. [41]

    Yiquan Wu, Siying Zhou, Yifei Liu, Weiming Lu, Xiaozhong Liu, Yating Zhang, Changlong Sun, Fei Wu, and Kun Kuang. 2023. Precedent-enhanced legal judgment prediction with llm and domain-model collaboration. In Proceedings of the 2023 Conference on Empirical Methods in Natural L...

    2023

  34. [42]

    Gechuan Zhang, Paul Nulty, and David Lillis. 2023. Argument mining with graph representation learning. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, page 371–380

    2023

  35. [43]

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations

    2019

  36. [44]

    Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. How does nlp benefit legal system: A summary of legal artificial intelligence. In ACL, pages 5218--5230

    2020

  37. [45]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.