Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Five large language models can perform basic IRAC legal analysis, but their responses are often brief, hedged, falsely confident, and prone to hallucination, so they are not yet reliable independent legal analysts.

desk verdict A transparent, well-scoped LLM legal-analysis evaluation that is worth refereeing, but the certainty-scoring rubric conflates justified hedging with failure to commit, so its headline limitation is partly a scoring artifact. read the letter →

arxiv 2502.03487 v1 pith:YEVHVX5N submitted 2025-02-04 cs.CY cs.AI

Lee Peoples This is my paper · ORCID
classification cs.CYcs.AI
keywords largelanguagemodelslegalanalysisIRACeducationhallucinationresearchartificialintelligenceanalogicalreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This study tests five large language models—Lexis+ AI, Claude, ChatGPT 3.5, Copilot 365, and Gemini—on seven legal-analysis exercises drawn from a law-school textbook and scored with a rubric built on the Issue-Rule-Application-Conclusion (IRAC) framework. The central claim is that current LLMs can carry out basic legal analysis in IRAC format, but their usefulness is limited by brief answers that omit detail, refusal to commit to a conclusion, false confidence in incorrect answers, and hallucinations. A legal-specific research product, Lexis+ AI, did not outperform the general-purpose models and hallucinated most often, consistent with earlier findings that legal LLMs do not necessarily beat general LLMs. The paper argues these limitations matter for legal education and practice because overreliance on AI may erode the logic, reasoning, and critical-thinking skills that underpin legal judgment.

What carries the argument

The central object is the IRAC framework—Issue, Rule, Application, Conclusion—the standard legal-analysis structure taught in American law schools, which the paper uses both as the task format and as the basis of its scoring rubric. The rubric scores six discrete tasks (following source instructions, issue identification, stating the rule, applying the rule, reaching the correct conclusion, and stating the conclusion with certainty), with additional points for responding to a chain-of-thought prompt and for not hallucinating. The test consists of seven fact scenarios adapted from a legal-analysis textbook covering rule analysis, analogical reasoning, and statutory analysis at beginning, intermediate, and skilled levels. Generation was fixed at temperature 0.0, top-p 1.0, and zero-shot prompting to maximize reproducibility and mimic a novice legal researcher.

What would settle it

Re-run the same seven exercises with the same prompting settings on the same or newer model versions and have several legal-writing instructors score the answers blind with the same rubric; if the models produce stable, detailed IRAC answers, if Claude no longer tops the ranking, or if Lexis+ AI's hallucination rate falls below the general models', the paper's central claims about LLM limitations and model ordering would be undermined.

Watch

Extended reading notes

Core claim

The paper reports that all five models—Lexis+ AI, Claude, ChatGPT 3.5, Copilot 365, and Gemini—handled legal fact patterns in IRAC format with varying success, and that Claude and Copilot generally produced the most detailed application of law to facts, while GPT-3.5 and Gemini returned shorter answers and Lexis+ AI hallucinated most often. The paper treats these results as evidence that LLMs can assist with elementary legal analysis but cannot yet be trusted as independent legal analysts, because they hallucinate, express false confidence in wrong conclusions, refuse to commit to answers, and rarely ground reasoning in principles, policies, or moral considerations.

Load-bearing premise

The paper's central claim rests on the assumption that seven hand-picked exercises from one textbook, scored by one author-developed rubric applied by a single evaluator, validly measure legal-analysis ability; if those exercises or that scoring are not representative or reliable, the model rankings and the conclusion about LLMs' legal-analysis limits would not generalize.

Editorial extensions

If this is right

  • Law students and lawyers can use LLMs as drafting and analysis assistants for basic IRAC tasks, but must verify every cited authority and conclusion before relying on the output.
  • Legal-specific AI research products should not be presumed superior to general-purpose models for legal analysis; in this study the general models outperformed Lexis+ AI, which also hallucinated most often.
  • Prompting strategy matters and is model-specific: the 'think step by step' prompt improved Claude, Copilot, and Gemini on complex exercises but did not meaningfully help ChatGPT 3.5 or Lexis+ AI.
  • Because LLMs hedge, refuse to commit, and display false confidence, legal work product produced with LLM assistance requires human judgment about the bottom line and the reasoning that supports it.
  • Legal educators should treat generative AI as a tool that can erode critical thinking if used as a crutch, and should design first-year curricula and training to build verification and reasoning skills explicitly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the single-evaluator, seven-exercise design means the numerical ranking should be read as descriptive, not as a stable benchmark; a larger multi-rater study would be needed to confirm which model leads.
  • Beyond the paper: the result that a retrieval-augmented legal LLM hallucinated more than general models suggests a testable hypothesis—that RAG-based legal assistants inherit corpus and retrieval gaps rather than curing them; one extension is to compare Lexis+ AI against general LLMs with statute text pasted directly into the prompt.
  • Beyond the paper: the observed cageyness and false confidence are calibration problems; a natural extension is to test whether requiring the model to state its conclusion first, or to output a confidence score, improves the certainty and accuracy of legal answers.
  • Beyond the paper: if AI-assisted early education erodes critical thinking as the paper warns, the effect should be measurable longitudinally—for instance, by comparing IRAC performance of law students who trained with structured AI assistance against those who trained without it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports an exploratory study of five LLMs (Lexis+ AI, Claude 3 Sonnet, ChatGPT 3.5, Copilot 365, and Gemini) performing legal analysis on seven exercises drawn from a first-year legal analysis textbook. The exercises cover rule analysis, analogical reasoning, and statutory analysis at beginning, intermediate, and skilled levels. Responses were scored manually with a six-category IRAC rubric plus two adjustment categories for chain-of-thought prompting and hallucinations. The paper finds that all models can produce basic IRAC-style analysis, that Claude and Copilot perform best on the author's scoring, that Lexis+ AI hallucinates most frequently, and that the models commonly give brief answers, hedge or refuse to commit, and sometimes express false confidence in incorrect answers. The article then discusses the theoretical and educational implications of these findings, arguing that LLMs cannot yet think like lawyers and cautioning against over-reliance on AI in legal education.

Significance. If taken as a qualitative, descriptive study rather than a benchmark, the paper is a useful contribution to the legal education and law librarian literature. It provides concrete, documented examples of LLM behavior on realistic law-school-style exercises, makes appropriate use of an external textbook rather than self-generated test items, and is transparently accompanied by appendices containing full responses. The author also explicitly disclaims statistical significance, which is appropriate given the design. The comparative legal-versus-non-legal LLM angle is timely, and the connection to prior studies (ChatGPT Goes to Law School, Legal Bench, hallucination studies) is reasonable. However, the quantitative rankings that run through the paper—including the total scores in Table 1 and the IRAC grades in Table 2—are not robust enough to support the strength of some of the phrasing in the conclusions, and at least one scoring criterion is worded in a way that undermines a headline limitation.

major comments (3)
  1. [§26, criterion 5] The scoring rubric's criterion 5, 'Reaching the Correct Conclusion,' repeats the text of criterion 4, 'Applying the Rule,' almost exactly: both award zero for re-stating the rule or incorrectly applying it, partial credit for some correct application, and full credit for completely and correctly applying the rule with all necessary inferences. As written, criterion 5 does not define how to score whether an LLM reached the correct conclusion. Because the conclusion scores in Tables 1 and 2 feed directly into the total scores and the headline model ranking, the numerical comparisons may double-count rule application and likely do not measure what they claim to measure. This criterion needs to be rewritten, and the affected totals should be recomputed.
  2. [§26, criterion 6; §46, §58] The 'Conclusion Stated with Certainty' criterion deducts points for answers that 'waivered, hedged, qualified, or equivocated,' which makes hedging categorically bad. The paper's own examples show that hedging is sometimes legally appropriate: in §58, Claude's refusal to commit in a case of first impression is described as 'understandable,' yet the rubric still lowers Claude's certainty score; in §46, Claude's refusal to predict conviction in the robbery exercise rests on a plausible larceny-versus-robbery distinction, yet it is treated as a deficiency. The headline limitation that LLMs have an 'inability to commit to answers' (Abstract, §5, §122) is therefore partly an artifact of the scoring design, because the rubric cannot distinguish warranted hedging from failure to take a position. The rubric should separate calibration from commitment, or the paper should reframe this finding as a stylistic observation relative to conventional law-school answer expectations rather than a stable LLM deficiency.
  3. [§16–17, §26; Tables 1–2] The comparative rankings rest on seven hand-selected exercises from one textbook, scored by a single evaluator with no inter-rater reliability check. The footnote disclaiming statistical significance is welcome, but the text still speaks of results that 'clearly demonstrate' LLM capabilities (§88) and states that 'Claude narrowly outperformed other LLMs' (§88). These are stronger statements than the design supports. The paper should either present the numerical scores as explicitly descriptive for these seven prompts and the author's rubric, with rankings labeled as informal, or add a second independent scorer and a sensitivity analysis exploring how the rankings change when the rubric weights are varied. Without one of those steps, the quantitative hierarchy should not be presented as a central result.
minor comments (5)
  1. [§26, criterion 8] The text says an LLM that does not hallucinate on any of the seven scenarios receives a perfect hallucination score of 8, but each no-hallucination scenario earns 1.143 points, and 7 × 1.143 = 8.001; Table 1 correctly lists 8.001 for Claude and Copilot. Please reconcile the arithmetic in the text.
  2. [§26, criterion 7] The same arithmetic inconsistency appears for chain-of-thought scoring: 1.143 points per improved answer would give a perfect score of 8.001, not 8.
  3. [¶121] 'Claud' should be 'Claude.'
  4. [Abstract; §5] The claim that LLMs are limited by an 'inability to commit to answers' is stated as a general limitation, but the certainty scores vary considerably across models (Claude scored 13.3/14; Lexis+ AI, Copilot, and Gemini scored 11.2/14). A qualifier such as 'in some exercises' would make the claim more accurate.
  5. [Footnote on page 1; §88] The disclaimer that numerical totals are descriptive only and not statistically significant appears only in the opening footnote. Repeating the caveat immediately before Tables 1 and 2 would prevent readers from over-interpreting the numerical rankings.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study is an externally grounded empirical benchmark with transparent scoring, and no central claim reduces to a fitted input or self-citation.

full rationale

This is an empirical benchmarking study rather than a derivation. The seven legal-analysis exercises are adapted from an external textbook (Hill & Vukadin, Legal Analysis: 100 Exercises for Mastery), and the results are compared against prior external studies such as “ChatGPT Goes to Law School,” Legal Bench, and the statutory-reasoning work of Blair-Stanek et al. The author-developed scoring rubric is disclosed in full before the results are presented, and it is a measurement instrument rather than a fitted parameter; none of the headline conclusions is obtained by plugging the rubric back into itself. The few self-citations (e.g., Peoples 2005 and 2012, on CD-ROM research and Westlaw Next) are historical and contextual support for the discussions of false confidence and creativity, and they are not load-bearing because the present study supplies its own direct examples of false confidence, hedging, and hallucination. The skeptical concern that the “Conclusion Stated with Certainty” criterion penalizes epistemically appropriate hedging is a construct-validity critique rather than a circularity: the paper does not define the limitation into existence by an equation, a fitted parameter, or a self-citation chain. No specific circular step can be quoted and exhibited under the required standard, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on domain assumptions about the IRAC framework, the correctness of the stated legal answers, and the reliability of requested sampling settings; no mathematical axioms or fitted equations are used. The scoring weights are hand-chosen and affect the rankings.

free parameters (1)
  • Scoring rubric point weights = 2 points per core task; 8/7 points for chain-of-thought; 8/7 points for hallucination-free
    Chosen by hand so totals sum to 100; not validated or derived from an external standard. The model rankings depend on these weights.
assumptions (3)
  • domain assumption The IRAC framework and the textbook exercises (Hill & Vukadin) provide a valid operationalization of basic legal analysis ability.
    The study measures thinking like a lawyer through IRAC performance on seven exercises; if IRAC does not capture the relevant skill, the central claim is weakened. Introduced in ¶15-17.
  • domain assumption The designated correct answers stated by the author for each exercise are accurate statements of law for the relevant jurisdictions.
    Scoring and hallucination judgments rely on these legal conclusions; the author does not provide independent legal authority for every correct answer beyond the exercise text. See ¶29, ¶33, ¶38, ¶43, ¶48, ¶56, ¶62.
  • domain assumption Setting temperature to 0.0 and top_p to 1.0 maximizes reproducibility and minimizes hallucination, and the LLM vendors honor these requested settings.
    Cited from Blair-Stanek et al.; the paper itself cites Ouyang et al. noting that temperature 0 does not guarantee determinism, so this assumption is fragile. See ¶21-22.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession." pith.science (2026). https://pith.science/paper/YEVHVX5N

@misc{pith2026250203487,
  author       = {Pith},
  title        = {Pith review of: Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YEVHVX5N}},
  note         = {Machine review of arXiv:2502.03487}
}
read the original abstract

This article reports the results of a study examining the ability of legal and non-legal Large Language Models to perform legal analysis using the Issue-Rule-Application-Conclusion framework. LLMs were tested on legal reasoning tasks involving rule analysis and analogical reasoning. The results show that LLMs can conduct basic IRAC analysis, but are limited by brief responses lacking detail, an inability to commit to answers, false confidence, and hallucinations. The study compares legal and nonlegal LLMs, identifies shortcomings, and explores traits that may hinder their ability to think like a lawyer. It also discusses the implications for legal education and practice, highlighting the need for critical thinking skills in future lawyers and the potential pitfalls of overreliance on artificial intelligence AI resulting in a loss of logic, reasoning, and critical thinking skills.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 29 canonical work pages

  1. [1]

    Choi, et al., GPT Goes to Law School, 71 J

    102 Jonathan H. Choi, et al., GPT Goes to Law School, 71 J. LEGAL EDUCATION 387 (2022). In this study, ChatGPT was prompted to answer four law school final exams. ChatGPT’s answers were mixed in with exam answers written by law students and graded anonymously. 103 Id. at

    work page 2022

  2. [2]

    think step by step was found, out of a number of prompts to maximize performance

    Prompting to Think Step by Step ¶71 A study testing ChatGPT-3’s abilities to perform statutory reasoning concluded that prompting it to “think step by step was found, out of a number of prompts to maximize performance.”112 This finding is opposite to the results of this study where the prompt “think step by step” only improved ChatGPTs’ performance on one...

    work page 2023

  3. [3]

    have clearly documented a false sense of security on the part of computer researchers

    Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 22 Law Library Journal False Confidence – When You Know, You Know ¶82 All the LLM’s tested demonstrated some degree of false confidence in responses by stating a conclusion with certainty despite the conclusion being objectively wrong.129 This false confidence ...

    work page 1994

  4. [5]

    robbery elements and arguments in Defendant’s case

    ¶74 Claude’s initial response to the intermediate statutory exercise does a decent job of analyzing the Florida Good Samaritan statute, but only superficially discusses the two cases provided in the prompt without any detailed analysis on how they might be used to make arguments for or against the application of the statute in a case of first impression. ...

    arXiv 2025

  5. [8]

    The False Promise of ChatGPT

    Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 27 Law Library Journal ¶98 Linguist Noam Chomsky considered some of the themes initially raised by Sunstein in an opinion piece titled, “The False Promise of ChatGPT.”176 Chomsky posits that “machine learning will degrade our science and debase our ethics by in...

    work page 2023

  6. [10]

    125 Varun Magesh, et al., Hallucination Free? Assessing the Reliability of Leading AI Legal Research Tools, arXiv, May 30, 2024, at 1, https://arxiv.org/pdf/2405.20362 [https://perma.cc/S4AB-5RB9]. 126 Id. at

    arXiv 2024

  7. [11]

    114 John J

    113 See Appendix 1, https://www.aallnet.org/wp-content/uploads/2025/01/LLJ_V117_No-1_Lee-Peoples_Appendices-Final.pdf. 114 John J. Nay, et al, Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence, arXiv, June 12, 2023, at 8, https://arxiv.org/pdf/2306.07075 [https://perma.cc/R4DS-P4F8]. 115 Id. Artificial Intelligence and L...

    arXiv 2025

  8. [15]

    I Learned it by Watching You

    135 Id. 136 Id. 137 Miao Xiong, et al., Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs, arXiv, Mar. 17, 2024, at 1, https://arxiv.org/pdf/2306.13063 [https://perma.cc/P9M5-NB49]. 138 Wikipedia, I Learned it by Watching You!, https://en.wikipedia.org/wiki/I_learned_it_by_watching_you (July 3, 2024). YouTube. “...

    arXiv 2024

Show all 33 references
  1. [18]

    For more discussion of how AI technology inhibits creativity and stifles legal innovation see Nicholas Mignanelli, The Legal Tech Bro Blues: Generative AI, Legal Indeterminacy, and the future of Legal Research and Writing, 8 GEORGETOWN L. TECH. REV. 298, 308 (2024). 171 Cass R...

    2024

  2. [24]

    157 Katherine Lee, et al., AI and Law: The Next Generation, Chapter 2, The Devil is in the Training Data, at 5 https://ssrn.com/ abstract=4580739 (July 3, 2024)

    156 Sean Michel Kerner, What is Generative AI? Everything You Need to Know, https://www.techtarget.com/whatis/definition/large- language-model-LLM#:~:text=While there isn’t a,used to infer new content [https://perma.cc/5BU4-P5XU]. 157 Katherine Lee, et al., AI and Law: The Nex...

    2024

  3. [25]

    legal interpretations sometimes require[ing] moral judgments, there may be a line that AI cannot cross in the foreseeable future

    205 Joshua P. Davis, Law Without Mind: AI, Ethics, and Jurisprudence, 55 CAL. W. L. REV. 165-219, 172 (2019). 206 Id. 207 Id. 208 Id. Davis’ argument relies on a distinction between describing moral beliefs (which he believes computers can do) and exercising moral judgment (wh...

    2019

  4. [29]

    225 Heather Hughes, Educating Deal Lawyers for the Digital Age, 92 FORDHAM L

    224 Examples include Spellbook, https://www.spellbook.legal/ and Motionize, https://motionize.io/. 225 Heather Hughes, Educating Deal Lawyers for the Digital Age, 92 FORDHAM L. REV. 1855-1865, 1858 (2024). Bill Tomlinson, ChatGPT and Works Scholarly: Best Practices and Legal P...

    2024

  5. [31]

    provide case law supporting his position

    Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 30 Law Library Journal ¶109 The blame and embarrassment for this mess rests squarely on the shoulders of the lawyer and his supervisor who tried to pass the fake cases off as legiti...

    2019

  6. [32]

    legal research tasks for which generative AI tools are more helpful than others

    230 Id. 231 Id. Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 34 Law Library Journal ¶125 This study describes some general traits of all LLMs potentially hindering their usefulness for legal research and abilities to think lik...

    2024

  7. [38]

    All LLMs’ reached the correct conclusion and formatted it into a legal argument that was stated with certainty

    These LLMs’ received a minor reduction in points for missing this significant rule. All LLMs’ reached the correct conclusion and formatted it into a legal argument that was stated with certainty. Intermediate Statutory Analysis ¶52 The intermediate statutory analysis exercise ...

    2024

  8. [66]

    the conduct does not squarely fit the robbery elements based on the cases cited

    Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 14 Law Library Journal intimidation. Claude refused to commit to an answer about D’s conviction for robbery. When pressed using iterative questions, Claude responded that “the condu...

    1959

  9. [81]

    to fairness, justice, and social policy concerns

    Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 28 Law Library Journal used by courts when appealing “to fairness, justice, and social policy concerns.”185 Only Lexis+ AI mentioned the possibility of an exception to the statute o...

    2008

  10. [86]

    92 Id. 93 Id. 94 Lexis+AI’s initial response was to restate the language of the statute without substantially applying it to the facts. Gemini’s initial response was to throw up its hands and state “I do not have enough information about that person to help with your request. ...

    2017

  11. [88]

    cited and quoted from purported judicial decisions that were said to [be] published in the Federal Reporter, the Federal Supplement, and Westlaw

    Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 29 Law Library Journal Opposing Party and Counsel), and Rule 5 (Lawyer Responsibility for Supervising Nonlawyer Assistants).192 A complete examination of the rules implicated by the...

    2023

  12. [97]

    ChatGPT Goes to Law School

    100 28 USCS § 2401(b) (2024). Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 18 Law Library Journal ¶64 Copilot’s performance was similar to Lexis+ AI’s. Copilot ignored the cases until it was instructed to think step by step an...

    2024

  13. [131]

    Commonwealth, 112 S.E

    66 Green v. Commonwealth, 112 S.E. 562 (Va. 1922)., Beard v. Commonwealth, 451 S.E.2d 698 (Va. Ct. App. 1994)., Mason v. Commonwealth, 105 S.E.2d 149 (Va. 1958). 67 Supra note

    1922

  14. [212]

    harm[s] students when it lulls educators into thinking students need no additional training in technology to be prepared for the workforce

    209 Iantha M. Haight, Digital Natives, Techno-Transplants: Framing Minimum Technology Standards for Law School Graduates, 44 J. OF THE LEGAL PROF. 175-221, 194 (2020). Casey Flaherty was general counsel of Kia Motors at the time he administered the technology assessment. The a...

    2020

  15. [360]

    235 Sean Harrington, Evaluating Generative AI for Legal Research: A Benchmarking Project, AI Law Librarians (May 24,

    234 Choi, et al., supra note 5, at 38-39 (2024). 235 Sean Harrington, Evaluating Generative AI for Legal Research: A Benchmarking Project, AI Law Librarians (May 24,

    2024

  16. [391]

    thinking like a lawyer

    149 Id. 150 Id. 151 Whether there is such a thing as “thinking like a lawyer” and what it involves has long been a topic of debate. For a full discussion see: FREDERICK SCHAUER, THINKING LIKE A LAWYER: A NEW INTRODUCTION TO LEGAL REASONING (2009), KARL LLEWELLYN, THE BRAMBLE B...

    2009

  17. [395]

    ChatGPT scored marginally better at 11.9 with Claude attaining the highest score at 13.3 out of a possible 14 points for stating a conclusion with certainty

    111 Lexis+AI, Copilot, and Gemini each scored 11.2 out of a possible 14 points on stating a conclusion with certainty. ChatGPT scored marginally better at 11.9 with Claude attaining the highest score at 13.3 out of a possible 14 points for stating a conclusion with certainty. ...

    2025

  18. [1124]

    187 Joshua P

    186 Lexis+AI response to prompt of exercise 87 on file with author. 187 Joshua P. Davis, Artificial Wisdom? A Potential Limit on AI in Law (And Elsewhere), 72 OKLA. L. REV. 51-89, 53 (2019). 188 Id. at

    2019

  19. [1312]

    (relying on Gatto v

    63 Id. (relying on Gatto v. Publix Supermarket, Inc., 387 So.2d 377 (Fla. Dist. Ct. App. 1980)). 64 Gatto, supra note

    1980

  20. [1858]

    sort of through osmosis’ by conducting document review

    227 Comments of Casetext Co-Founder Pablo Adrredondo about learning something about corporations and litigation “sort of through osmosis’ by conducting document review.” 15:26, Geek in Review Podcast, Feb. 29, 2024, 15:26, https://www.geeklawblog.com/2024/ 02/pablo-arredondo-o...

    2024

  21. [1959]

    Elliott, 88 U.S

    (citing Edward v. Elliott, 88 U.S. 532 (1874)). 79 GPT response on file with author. 80 Ind. R. Trial P. 38 (2024). 81 Id. 82 Hayworth v. Bromwell, 158 N.E.2d 285 (Ind. 1959). Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession 15 La...

    2024

  22. [2012]

    more than half of the nation’s primary and secondary school students—more than 30 million children—use Google education apps like Gmail and Docs

    213 By 2017 “more than half of the nation’s primary and secondary school students—more than 30 million children—use Google education apps like Gmail and Docs.”214 Google hardware and software is also “the productivity tool of choice”215 for many higher education students, facu...

    2017

  23. [2017]

    conducted the first randomized controlled trial to study the effect of AI assistance on human legal analysis

    214 Id. 215 Greg Lambert, Next-Gen Bar Exam Must Tackle Google Schools and the Digital Native Myth by Testing Basic Tech Skills for Practice, 3 Geeks and a Law Blog, (March 14, 2023), https://www.geeklawblog.com/2023/03/next-gen-bar-exam-must-tackle-google-schools- and-the-dig...

    2024

  24. [2023]

    act as an advocate

    177 Id. 178 Id. 179 There is no shortage of examples of LLMs failing to work within the legal profession’s rules of ethics. The numerous stories of LLMs hallucinating when asked to “act as an advocate” play this out. The infamous lawyer in the Avianca v. Matta hallucination in...

    2024

  25. [2024]

    https://www.ailawlibrarians.com/2024/05/24/new-project-evaluating-genai/ [https://perma.cc/BWC3-R3V6]

    2024

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.