Pith. sign in

REVIEW 4 major objections 5 minor 34 references

An Integrated Framework of Prompt Engineering and Multidimensional Knowledge Graphs for Legal Dispute Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Coupling a three-stage legal prompt with a three-layer knowledge graph lifts LLM citation accuracy by up to 39.7% and sensitivity by over 11% on 100 legal dispute cases, while a simpler knowledge-graph setup degrades reasoning quality.

desk verdict Plausible engineering combination, but the evaluation is confounded by selective regeneration and the internal scoring rubric overlapping with the human grading. read the letter →

arxiv 2507.07893 v4 pith:YSTTBCV6 submitted 2025-07-10 cs.AI

Mingda Zhang , Na Zhao , Jianglong Qing , Qing xu , Kaiwen Pan , Ting luo This is my paper
classification cs.AI
keywords legaldisputeanalysispromptengineeringmultidimensionalknowledgegraphenhancementworkflowlargelanguagemodelsreasoningcitationaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that large language models fail at legal dispute analysis for two fixable reasons: flat prompts give the model no professional reasoning path, and the knowledge they are handed is shallow, static, and unranked. It proposes an integrated framework coupling a three-stage hierarchical prompt (task definition, knowledge background, reasoning guidance) with a three-layer legal knowledge graph (ontology, representation, instances) served by four complementary retrieval strategies. On 100 legal dispute question-answer pairs, the complete framework lifts sensitivity by 11.1%–11.3%, specificity by 5.4%–6.0%, and citation accuracy by 29.5%–39.7% over the two unmodified baseline LLMs, and scores higher on all five human-expert QUEST dimensions. The same experiments show a simpler knowledge-graph configuration actually degrades reasoning, which the authors read as evidence that citation-oriented prompting alone causes “knowledge stacking” instead of “knowledge fusion.”

What carries the argument

Two coupled structures carry the argument. The three-stage hierarchical prompt separates task definition (a BM25-style matching formula mapping queries to legal task templates), knowledge background (a BM25+ relevance score blending text similarity, knowledge-graph path distance, case-law citation weight, and jurisdictional overlap), and reasoning guidance (professional analysis-step templates described as structured chains of thought). The three-layer knowledge graph organizes law as a classification ontology layer, a representation layer, and an instance layer, and is queried by four fused retrieval strategies: exact legal-code matching, semantic vector similarity, ontology path reasoning with a distance decay factor, and lexical-segmentation term matching weighted by legal-term importance. A dynamic prompt optimization loop — generate, score on five quality dimensions, adjust the prompt, regenerate — closes the circuit, while web-search coupling handles knowledge timeliness. The prompt supplies the reasoning structure, the graph supplies precise authority-weighted concepts, and the loop adapts the prompt when the first answer fails the quality bar.

What would settle it

Re-run the complete configuration on the same 100 cases and have the ten legal experts grade the outputs on a rubric the optimization loop has never seen, for instance dimensions measuring procedural correctness, neutrality across litigant roles, or robustness to deliberately misleading case facts. If the complete configuration's margin over baseline shrinks toward its modest BLEU and ROUGE gains, the headline sensitivity, specificity, and citation-accuracy improvements are largely an artifact of the optimizer matching the test's own scoring criteria rather than a general gain in legal analysis. A second check: disable the dynamic optimization loop and compare, since the loop's contribution should survive as an independent improvement.

Watch

Extended reading notes

Core claim

The paper's central claim is that LLM legal dispute analysis is limited less by model capability than by the structure wrapped around the model: how the task is defined, what knowledge is retrieved, and which reasoning path is prescribed. Evidence comes from a three-way comparison on two models. A basic knowledge-graph configuration emphasizing citation and format (the “traditional” condition) improves citation accuracy but lowers reasoning rationality and conclusion reliability, a pattern the authors call “knowledge stacking” rather than “knowledge fusion.” The complete configuration — three-stage prompt plus three-layer graph — beats the unmodified baseline on every reported dimension, with sensitivity rising from 0.6201 to 0.6892 (DeepSeek) and 0.6546 to 0.7285 (Qwen), specificity from 0.7841 to 0.8266 and 0.8062 to 0.8547, and citation accuracy from 0.58 to 0.81 and 0.61 to 0.79. The authors read this as a transition from “simple application” to “comprehensive integration” of knowledge acquisition and reasoning guidance, a reading they say is confirmed by ten legal experts scoring the complete configuration higher on all five QUEST dimensions.

Load-bearing premise

The reported gains assume the framework's internal response-quality scorer is not grading the model on the same criteria the human experts later use; the internal dimensions — legal accuracy, comprehensiveness, citation standardization, logical rigor, and expression — overlap heavily with the QUEST dimensions scored by the ten experts in Section 4.3, so the optimizer may be tuning the model to the exact rubric that produces the headline scores.

Editorial extensions

If this is right

  • Legal assistance systems could reach substantially more reliable citation behavior from general-purpose LLMs without fine-tuning, since the gains appear on two different base models (DeepSeek-R1-Distill-Qwen-32B and Qwen/QwQ-32B).
  • Knowledge injection without reasoning guidance is actively harmful on these metrics: the traditional configuration lowers sensitivity, specificity, precision, and reasoning-quality scores relative to baseline, so the structure wrapping the retrieved knowledge is itself load-bearing.
  • The framework's timeliness mechanisms, coupling the knowledge graph with web search through jurisdictional identification, concept timeliness marking, and change tracking, give deployed systems a path to stay current as laws and judicial interpretations evolve.
  • The dynamic prompt optimization loop turns the system into a self-correcting pipeline that can re-prompt and re-fuse knowledge until an answer passes a quality threshold, which is the mechanism the paper credits for handling cases with intertwined legal concepts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate whether the three-stage prompt or the knowledge graph carries more of the gain; a direct ablation of the two components would test its “knowledge stacking versus knowledge fusion” explanation, which the current three-configuration design only supports jointly.
  • The overlap between the internal quality scorer driving prompt optimization and the human-expert QUEST rubric means the manual-evaluation gains have not been shown to survive grading on an independent rubric; that check is not reported in the paper.
  • Because the largest reported gain is in citation accuracy and the smallest in specificity, much of the benefit may come from retrieving and formatting legal sources rather than from deeper legal reasoning; a retrieval-augmented baseline without the prompt structure could close part of the gap on citation metrics alone.
  • The response pattern in the Liebeck case suggests the framework's reasoning templates lean on a standard legal memo structure, raising an untested question the authors defer to future work: whether the templates transfer to non-English and civil-law jurisdictions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an integrated framework for legal dispute analysis that combines a three-stage hierarchical prompt structure (task definition, knowledge background, reasoning guidance) with a three-layer knowledge graph (legal ontology, representation, instance layers), plus four complementary retrieval strategies and a dynamic prompt optimization loop. The framework is evaluated on 100 question-answer pairs with two LLMs (DeepSeek-R1-Distill-Qwen-32B and Qwen/QwQ-32B) under three configurations: Baseline, Traditional (basic KG with single-dimension retrieval), and Complete. The authors report that Complete outperforms Baseline on BLEU/ROUGE, sensitivity/specificity/precision, citation accuracy, reasoning rationality, conclusion reliability, and manual QUEST-style evaluations, with gains such as 11.1%–11.3% sensitivity and 29.5%–39.7% citation accuracy.

Significance. If the reported gains are real, the paper would provide a useful integration of prompt engineering and knowledge graphs for legal dispute analysis, an area where prior work has largely treated these techniques separately. The paper is explicit about its architecture and provides equations, a workflow diagram, and a detailed case study (Liebeck v. McDonald's), which is a strength. However, the central empirical claim is currently undermined by two confounds: the Complete configuration uses an iterative quality-based regeneration loop that Baseline and Traditional do not receive, and the internal quality scorer overlaps substantially with the human evaluation rubric. The evaluation is also underpowered, with no error bars or significance tests. These issues are load-bearing for the headline numbers, so the contribution as presently stated is not yet established.

major comments (4)
  1. [§3.1.4, Fig. 2; Tables 2–3] The dynamic prompt optimization mechanism gives the Complete configuration an uncontrolled test-time computation advantage. The workflow in §3.1.4 describes a closed loop of initial generation, multi-dimensional quality assessment, prompt adjustment, legal knowledge fusion, and regeneration until a quality threshold is met. Baseline and Traditional are described as single-pass configurations with no equivalent refinement budget. The measured superiority of Complete over Baseline may therefore reflect selective regeneration under the internal scorer rather than the prompt/KG design. To support the attribution in the Abstract, the authors should either freeze first-pass outputs, provide Baseline and Traditional with the same number of regeneration attempts using neutral prompts, or report results as a function of the iteration count and show that the gains persist when the refinement budget is matched.
  2. [§3.1.3 vs §4.3, Table 3] The internal multi-dimensional response quality assessment used to drive prompt optimization overlaps with the human evaluation rubric that later grades the outputs. Internal dimensions (legal accuracy, content comprehensiveness, citation standardization, logical rigor, professional expression, §3.1.3) are close to the QUEST dimensions (information quality, understanding and reasoning, expression style, trust and confidence, §4.3) and to the Table 3 dimensions (citation accuracy, reasoning rationality, conclusion reliability). Because the optimizer is tuned against the same kind of criteria that human experts then use, the reported gains are inflated by construction unless independence is demonstrated. The authors should either show that the internal scorer is not derived from the QUEST rubric, or evaluate using outcome measures that are not part of the optimization objective (e.g., verifiable citation existence, legal correctness against an independent gold standard).
  3. [§4.1, Tables 1–2] The evaluation is underpowered and the Traditional baseline is anomalous. The test set contains only 100 question-answer pairs, and no confidence intervals, standard deviations, or significance tests are reported, so the claimed improvements (e.g., sensitivity +11.1%–11.3%) are not statistically anchored. More importantly, Traditional performs dramatically worse than Baseline on automated metrics, such as DeepSeek's BLEU-1 dropping from 0.2409 to 0.1124 (−53.3%) and ROUGE-1 from 0.2779 to 0.1713 (−38.4%). The paper attributes this to 'single prompt limitations,' but such a large degradation suggests a possible implementation problem in the Traditional retrieval or prompt construction. Since Traditional is meant to be a controlled ablation, this anomaly must be diagnosed and, if it reflects a bug, the comparison rerun.
  4. [§3.1.1–§3.2.2, Eqs. (1)–(8)] The framework's quantitative behavior relies on many free parameters—w_j, α, β, γ, δ, k1, b, λ, α1–α3, σ, and the fusion weights over the four retrieval strategies—but the manuscript does not report their fitted values, the data on which they were fit, or any sensitivity analysis. This makes the retrieval and prompt-matching contributions non-reproducible and prevents the reader from assessing whether the reported gains are robust to reasonable parameter variation. At minimum, the authors should report all parameter values, the optimization procedure, and a robustness check (e.g., perturbing each parameter by ±20% and reporting the resulting BLEU and sensitivity ranges).
minor comments (5)
  1. [§4.2.1] The metric names are misspelled: 'Bilingual Evaluation Underresearch' should be 'Bilingual Evaluation Understudy,' and 'Recall-Oriented Underresearch for Gisting Evaluation' should be 'Recall-Oriented Understudy for Gisting Evaluation.'
  2. [Abstract vs §5] The reported sensitivity improvement is inconsistent: the Abstract states 11.1%–11.3%, while the Conclusion states 7.3%–11.3%. The Conclusion also reports precision gains of 8.7%–12.5% that do not appear in the Abstract. These numbers should be reconciled.
  3. [§4.1] For the 100-pair test set, the paper does not report how the cases were stratified across legal domains, nor the inter-annotator agreement among the three senior legal experts who annotated the gold labels. Reporting Cohen's kappa or a similar measure would strengthen the evaluation.
  4. [§4.3, Table 4] The manual QUEST evaluation uses only 30 of the 100 test cases, with no explanation of the subsampling and no confidence intervals or significance tests for the expert scores. Please state the selection criterion and add variance information.
  5. [References] Several references are duplicated or inconsistently formatted: refs [1] and [7] are the same work, and refs [18] and [2] are the same LegalBench entry with different DOI fields. Please deduplicate and unify the reference list.

Circularity Check

2 steps flagged · score 6.0 of 10

Dynamic prompt optimization tunes outputs to a rubric overlapping the human evaluation dimensions, and citation accuracy is partly built into the prompt by construction.

  1. fitted input called prediction [Sections 3.1.3, 3.1.4, and 4.3]
    "The legal reasoning guidance framework adopts multi-dimensional assessment of the professional level of responses, including five key dimensions: legal accuracy, content comprehensiveness, citation standardization, logical rigor, and professional expression standardization. The assessment standards reference the accuracy dimension in the QUEST assessment framework, focusing on evaluating the consistency of AI system output content with legal authoritative standards."

    The framework's closed loop (Section 3.1.4) regenerates Complete outputs until they pass an internal quality threshold built from these dimensions, including citation standardization and legal accuracy, explicitly referencing QUEST. The manual evaluation in Section 4.3 then scores the same outputs with the QUEST indicators (information quality, understanding and reasoning, expression style and role, safety and harm, trust and confidence), and Table 3 uses closely related dimensions (citation accuracy, reasoning rationality, conclusion reliability). Baseline and Traditional are not described as receiving any equivalent iterative-refinement budget.

  2. self definitional [Sections 3.2.3, 3.1.4, and 4.2.3]
    "This mechanism first establishes a unified retrieval interface covering authoritative legal data sources, ensuring retrieved legal information meets timeliness requirements through three mechanisms: jurisdictional identification, legal concept timeliness marking, and change tracking; second, it develops a legal authority assessment model, calculating authority scores based on legal source type, publishing institution hierarchy, and citation frequency, prioritizing high-authority information."

    The complete pipeline injects the retrieved authoritative legal provisions and precedents into the prompt (legal knowledge fusion optimizer, Section 3.1.4), and Section 3.2.3 says this can enhance the legal correctness and citation standardization of model answers. Table 3's citation-accuracy metric then measures whether the model's citations match the legal provisions and precedents it was handed, so the 29.5-39.7% citation-accuracy improvement is largely mechanical: the sources being checked for citation accuracy are placed into the prompt by construction. This validates the retrieval-and-insertion component rather than independently testing the framework's reasoning contribution.

full rationale

The paper's central empirical claim is that the Complete configuration beats Baseline on Tables 2-4. The most load-bearing issue is Section 3.1.4's dynamic prompt-optimization loop: outputs are regenerated until an internal multi-dimensional quality scorer is satisfied, and that scorer (Section 3.1.3) uses dimensions that overlap the QUEST rubric used by 10 human experts in Section 4.3 and the Table 3 dimensions scored by five legal experts. Because Baseline and Traditional are not given an equivalent regeneration budget, the comparison conflates the framework's structure with a selective sampling and optimization advantage. The Traditional ablation removes the prompt/KG structure but does not neutralize the regeneration loop. The citation-accuracy gain is further by construction, since retrieved authoritative sources are inserted into the prompt and then checked as citations. These are structural reductions, not intent-based accusations, and they make the headline gains partially circular. On the other hand, there is no self-citation chain, no imported uniqueness theorem, and the BLEU/ROUGE evaluation and some retrieval components are externally grounded, so the work is not entirely reducible to its inputs. A score of 6 reflects partial circularity in the headline comparisons rather than a fully definitional derivation. Additional concerns, such as the absence of significance tests on the 100-pair test set, are correctness risks rather than circularity.

Assumptions & free parameters 11 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the construction quality of the legal knowledge graph and prompt templates, the validity of expert annotations, and the assumption that the evaluation rubric used by human graders is independent of the system's internal optimization rubric. No code, data, or parameter values are released, so these premises cannot be checked.

free parameters (11)
  • w_j (task matching feature weights)
    Eq. 1: importance weights for feature dimensions in legal task identification, determined by legal experts via Delphi method; values not reported.
  • alpha (provision-term adjustment, Eq. 1)
    Adjustment parameter for professional legal terms in task matching; value not reported.
  • alpha, beta, gamma, delta (relevance combination weights, Eq. 2)
    Weights combining text, KG path, case, and jurisdiction relevance, determined by Bayesian optimization; values not reported.
  • k1, b (BM25 parameters, Eq. 2)
    Term frequency saturation and length normalization parameters; standard BM25 values not stated.
  • delta (long-document compensation factor, Eq. 2)
    Compensation factor inside the BM25+ term; value not reported.
  • gamma (code matching balance, Eq. 3)
    Weight balancing exact vs partial legal code matching; value not reported.
  • lambda (path decay factor, Eq. 5)
    Decay factor for path length in ontology reasoning; 0<lambda<1 but value not reported.
  • alpha_1, alpha_2, alpha_3 (term matching weights, Eq. 7)
    Weights for exact, stem, and semantic matching summing to 1; values not reported.
  • sigma (smoothing factor, Eq. 8)
    Smoothing in legal-term weight ILT(t); value not reported.
  • fusion weights across the four retrieval strategies
    Final concept retrieval score weights optimized by machine learning; values and any train/test separation not reported.
  • quality assessment dimension weights and threshold
    Weights for the five response-quality dimensions (Section 3.1.3) and the preset threshold for the dynamic prompt optimizer (Section 3.1.4); values not reported.
assumptions (6)
  • domain assumption The three-layer knowledge graph and legal reasoning path templates are correctly and sufficiently constructed for the test domains.
    Section 3.2.1 describes the architecture but provides no construction data, source coverage, error rates, or validation of the ontology, representation, and instance layers.
  • domain assumption Expert annotations (three senior legal experts for judgment labels, five for content scores, ten for QUEST) are reliable and consistent ground truth.
    Section 4.2.2 and 4.3 rely on expert judgment for all primary metrics, but inter-annotator agreement is not reported.
  • standard math Standard IR formulas (BM25+, TF-IDF, cosine similarity) remain valid for legal text matching as implemented.
    Equations 1-8 in Sections 3.1 and 3.2 use standard information retrieval math; the axioms of these methods are assumed without adaptation specifics.
  • domain assumption The COLIEE-derived test set of 100 QA pairs is representative of the legal dispute analysis task.
    Section 4.1 randomly selects 100 pairs from COLIEE and unspecified court cases; representativeness across jurisdictions and legal domains is asserted, not demonstrated.
  • domain assumption The QUEST framework dimensions are suitable for grading legal LLM output quality.
    Section 4.3 adopts QUEST without calibration or validation in the legal domain.
  • domain assumption Single-run model outputs are stable enough to compare configurations.
    All tables (Tables 1-4) report single numbers per configuration with no multiple runs, so temperature stochasticity is not controlled.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Integrated Framework of Prompt Engineering and Multidimensional Knowledge Graphs for Legal Dispute Analysis." pith.science (2026). https://pith.science/paper/YSTTBCV6

@misc{pith2026250707893,
  author       = {Pith},
  title        = {Pith review of: An Integrated Framework of Prompt Engineering and Multidimensional Knowledge Graphs for Legal Dispute Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YSTTBCV6}},
  note         = {Machine review of arXiv:2507.07893}
}
read the original abstract

Legal dispute analysis is crucial for intelligent legal assistance systems. However, current LLMs face significant challenges in understanding complex legal concepts, maintaining reasoning consistency, and accurately citing legal sources. This research presents a framework combining prompt engineering with multidimensional knowledge graphs to improve LLMs' legal dispute analysis. Specifically, the framework includes a three-stage hierarchical prompt structure (task definition, knowledge background, reasoning guidance) along with a three-layer knowledge graph (legal ontology, representation, instance layers). Additionally, four supporting methods enable precise legal concept retrieval: direct code matching, semantic vector similarity, ontology path reasoning, and lexical segmentation. Through extensive testing, results show major improvements: sensitivity increased by 11.1%-11.3%, specificity by 5.4%-6.0%, and citation accuracy by 29.5%-39.7%. As a result, the framework provides better legal analysis and understanding of judicial logic, thus offering a new technical method for intelligent legal assistance systems.

Figures

Figures reproduced from arXiv: 2507.07893 by the authors.

Figure 1
Figure 1. Overall Architecture of the Legal Dispute Analysis Framework [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Dynamic Prompt Optimization Mechanism Workflow [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Legal Large Language Model Case Analysis Comparison: Baseline Version vs. Traditional Version vs. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 19 canonical work pages

  1. [2]

    Proceedings of the Conference on Neural Information Processing Systems, pp

    Guha, N., Nyarko, J., Ho, D.E., et al.: LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models. Proceedings of the Conference on Neural Information Processing Systems, pp. 44123–44279 (2023). doi:10.5555/3666122.3668037

    work page arXiv 2023

  2. [3]

    Advances in Neural Information Processing Systems 33, 1877–1901 (2020)

    Brown, T., Mann, B., Ryder, N., et al.: Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems 33, 1877–1901 (2020). doi:10.5555/3495724.3495883

    arXiv 2020

  3. [4]

    In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp

    Wu, Y ., Zhou, S., Liu, Y ., et al.: Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12060–12075 (2023). doi:10.18653/v1/2023.emnlp-main.740

    work page doi:10.18653/v1/2023.emnlp-main.740 2023

  4. [5]

    ACM Transactions on Intelligent Systems and Technology 15(2), 1–43 (2024)

    Wang, Y ., Zhang, Z., Wang, H., et al.: A Survey on Evaluation of Large Language Models. ACM Transactions on Intelligent Systems and Technology 15(2), 1–43 (2024). doi:10.1145/3641289

    doi:10.1145/3641289 2024

  5. [6]

    In: Proceedings of the 2023 Conference on Neural Information Processing Systems, pp

    Glanzer, P., Winter, F., Schwaiger, R., et al.: FACTS Grounding: A New Benchmark for Evaluating the Factuality of Large Language Models. In: Proceedings of the 2023 Conference on Neural Information Processing Systems, pp. 26583–26597 (2023). doi:10.48550/arXiv.2311.08401

    arXiv doi:10.48550/arxiv.2311.08401 2023

  6. [7]

    Information 15(2), 98–117 (2023)

    Lai, J., Gan, W., Wu, J., et al.: Large Language Models in Law: A Survey. Information 15(2), 98–117 (2023). doi:10.3390/info15020098

    work page doi:10.3390/info15020098 2023

  7. [8]

    In: Proceedings of the 2024 International Conference on Machine Intelligence and Digital Applications, pp

    Zhu, T., Zeng, M., Cai, C., et al.: Intelligent Legal Document Generation System and Method Based on Knowledge Graph. In: Proceedings of the 2024 International Conference on Machine Intelligence and Digital Applications, pp. 164–168 (2024). doi:10.1145/3662739.3669909

    work page arXiv 2024

  8. [9]

    Frontiers in Artificial Intelligence 6 (2023)

    Zubaer, A.A., Granitzer, M., Mitrovi´c, J.: Performance analysis of large language models in the domain of legal argument mining. Frontiers in Artificial Intelligence 6 (2023). doi:10.3389/frai.2023.1278796

    work page arXiv 2023

Show all 34 references
  1. [10]

    arXiv preprint arXiv:2311.15735 (2023)

    Sun, Z., Li, J., Wang, M., et al.: BetterLegal: A Comprehensive Approach to Enhancing Legal Knowledge in Large Language Models by Feedback from Legal Professionals. arXiv preprint arXiv:2311.15735 (2023). doi:10.48550/arXiv.2311.15735

    2023 doi arXiv

  2. [11]

    Advances in Neural Information Processing Systems 35, 24824–24837 (2022)

    Wei, J., Wang, X., Schuurmans, D., et al.: Chain-of-Thought Prompting Elicits Reasoning in Large Language Mod- els. Advances in Neural Information Processing Systems 35, 24824–24837 (2022). doi:10.5555/3600270.3602070

    2022

  3. [12]

    In: Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, pp

    Mumford, J., Atkinson, K., Bench-Capon, T.: Combining a Legal Knowledge Model with Machine Learning for Reasoning with Legal Cases. In: Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, pp. 224–233 (2023). doi:10.1145/3594536.3595158

    2023

  4. [13]

    ACM Computing Surveys 54(4), 1–37 (2021)

    Hogan, A., Blomqvist, E., Cochez, M., et al.: Knowledge Graphs. ACM Computing Surveys 54(4), 1–37 (2021). doi:10.1145/3447772

    2021 doi

  5. [14]

    arXiv preprint arXiv:2104.02284 (2021)

    Li, L., Fan, Y ., Tong, M., et al.: Text-guided Legal Knowledge Graph Reasoning. arXiv preprint arXiv:2104.02284 (2021). doi:10.48550/arXiv.2104.02284

    2021 doi arXiv

  6. [15]

    In: Proceedings of the 2024 9th International Conference on Intelligent Information Technology, pp

    Wan, R., Zhang, J., Yang, L., et al.: A Method for Judicial Case Knowledge Graph Construction Based on Event Extraction. In: Proceedings of the 2024 9th International Conference on Intelligent Information Technology, pp. 156–162 (2024). doi:10.1145/3654522.3654533

    2024

  7. [16]

    Journal of Institutional and Theoretical Economics (2023)

    Choi, J.H.: How to Use Large Language Models for Empirical Legal Research. Journal of Institutional and Theoretical Economics (2023). doi:10.2139/ssrn.4536852

    2023 doi

  8. [17]

    In: Huang, D.S., Si, Z., Zhang, C

    Shi, J., Guo, Q., Liao, Y ., et al.: Legal-LM: Knowledge Graph Enhanced Large Language Models for Law Con- sulting. In: Huang, D.S., Si, Z., Zhang, C. (eds.) Advanced Intelligent Computing Technology and Applications, pp. 138–149. Springer Nature Singapore (2024). doi:10.1007/...

    2024 doi

  9. [18]

    arXiv preprint arXiv:2308.11462 (2023)

    Guha, N., Nyarko, J., Ho, D.E., et al.: LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models. arXiv preprint arXiv:2308.11462 (2023). doi:10.48550/arXiv.2308.11462

    2023 doi arXiv

  10. [19]

    arXiv preprint arXiv:2107.04771 (2021)

    Dhani, J.S., Nimbal, P.K., Yashwanth, C.S.J., et al.: Similar Cases Recommendation using Legal Knowledge Graphs. arXiv preprint arXiv:2107.04771 (2021). doi:10.48550/arXiv.2107.04771 14 An Integrated Framework for Legal Dispute Analysis

    2021 doi arXiv

  11. [20]

    In: Proceedings of the Natural Legal Language Processing Workshop, pp

    Jayakumar, T., Farooqui, F., Farooqui, L.: Large Language Models are legal but they are not: Making the case for a powerful LegalLLM. In: Proceedings of the Natural Legal Language Processing Workshop, pp. 223–229 (2023). doi:10.18653/v1/2023.nllp-1.22

    2023 doi

  12. [21]

    arXiv preprint arXiv:2310.14735 (2023)

    Chen, B., Xu, H., Xiao, Z., et al.: Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review. arXiv preprint arXiv:2310.14735 (2023). doi:10.48550/arXiv.2310.14735

    2023 doi arXiv

  13. [22]

    Transactions of the Association for Computational Linguistics 12, 39–57 (2024)

    Zhang, T., Ladhak, F., Durmus, E., et al.: Benchmarking Large Language Models for News Summarization. Transactions of the Association for Computational Linguistics 12, 39–57 (2024). doi:10.1162/tacl_a_00632

    2024 doi

  14. [23]

    In: Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics, pp

    Cheng, L., Wu, F., Zhang, R., et al.: Legal Document Generation with Multi-stage Prompting. In: Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics, pp. 15874–15889 (2023). doi:10.18653/v1/2023.acl-long.15874

    2023 doi

  15. [24]

    arXiv preprint arXiv:2305.14688 (2023)

    Zhang, K., Ziems, N., Ni, M., et al.: JudgeLM: Fine-tuned Large Language Models for Legal Judgment Prediction. arXiv preprint arXiv:2305.14688 (2023). doi:10.48550/arXiv.2305.14688

    2023 doi arXiv

  16. [25]

    In: Proceedings of the 2024 8th International Conference on Computer Science and Artificial Intelligence, pp

    Wang, B., Zhou, Y ., Tang, X., et al.: A Legal Fact-Finding Model Based on the T5 and LexiLaw Large Language Models. In: Proceedings of the 2024 8th International Conference on Computer Science and Artificial Intelligence, pp. 208–215 (2024). doi:10.1145/3709026.3709070

    2024

  17. [26]

    arXiv preprint arXiv:2212.02199 (2022)

    Trautmann, D., Derczynski, L., Augenstein, I.: Legal Prompt Engineering for Multilingual Legal Judgement Prediction. arXiv preprint arXiv:2212.02199 (2022). doi:10.48550/arXiv.2212.02199

    2022 doi arXiv

  18. [27]

    arXiv preprint arXiv:2312.02856 (2023)

    Cui, J., Li, Y ., Zeng, Y ., et al.: Chain-of-Law: Semantic Structure Guided Reasoning for Legal Cases Summariza- tion with LLMs. arXiv preprint arXiv:2312.02856 (2023). doi:10.48550/arXiv.2312.02856

    2023 doi arXiv

  19. [28]

    Information 15(3), 188 (2023)

    Liu, P., Wang, Z., Chen, R., et al.: Construction of Legal Knowledge Graph Based on Knowledge-Enhanced Large Language Models. Information 15(3), 188 (2023). doi:10.3390/info15030188

    2023 doi

  20. [29]

    Information 15(11), 666 (2024)

    Li, J., Qian, L., Liu, P., Liu, T.: Construction of Legal Knowledge Graph Based on Knowledge-Enhanced Large Language Models. Information 15(11), 666 (2024). doi:10.3390/info15110666

    2024 doi

  21. [30]

    arXiv preprint arXiv:2306.08302 (2023)

    Pan, S., Luo, L., Wang, Y ., et al.: Unifying Large Language Models and Knowledge Graphs: A Roadmap. arXiv preprint arXiv:2306.08302 (2023). doi:10.48550/arXiv.2306.08302

    2023 doi arXiv

  22. [31]

    arXiv preprint arXiv:2305.09531 (2023)

    Xiao, C., Zhong, H., Guo, Z., et al.: CAIL2022-LKGAC: Legal Knowledge Graph-based Advice Classification. arXiv preprint arXiv:2305.09531 (2023). doi:10.48550/arXiv.2305.09531

    2023 doi arXiv

  23. [32]

    In: Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law, pp

    Zheng, L., Guha, N., Anderson, B.R., et al.: When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset of 53,000+ Legal Holdings. In: Proceedings of the Eighteenth International Conference on Artificial Intelligence and Law, pp. 159–168 (2...

    2021

  24. [33]

    arXiv preprint arXiv:2311.08388 (2023)

    Yang, Y ., Huang, L., Fu, L., et al.: WhatLawyers: A Legal Knowledge-Enhanced System for Legal Text Understanding. arXiv preprint arXiv:2311.08388 (2023). doi:10.48550/arXiv.2311.08388

    2023 doi arXiv

  25. [34]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, vol

    Chalkidis, I., Jana, A., Hartung, D., et al.: LexGLUE: A Benchmark Dataset for Legal Language Understanding in English. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 4310–4330 (2022). doi:10.18653/v1/2022.acl-long.297

    2022 doi

  26. [35]

    In: Proceedings of the Natural Legal Language Processing Workshop, pp

    Niklaus, J., Chalkidis, I., Stürmer, M.: Swiss-Judgment-Prediction: A Multilingual Legal Judgment Predic- tion Benchmark. In: Proceedings of the Natural Legal Language Processing Workshop, pp. 19–35 (2021). doi:10.18653/v1/2021.nllp-1.3 15

    2021 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.