Evaluating AI for Law: Bridging the Gap with Open-Source Solutions

Jonathan Li; Rohan Bhambhoria; Samuel Dahan; Xiaodan Zhu

arxiv: 2404.12349 · v1 · pith:DTTT5P2Bnew · submitted 2024-04-18 · 💻 cs.AI · cs.HC

Evaluating AI for Law: Bridging the Gap with Open-Source Solutions

Rohan Bhambhoria , Samuel Dahan , Jonathan Li , Xiaodan Zhu This is my paper

classification 💻 cs.AI cs.HC

keywords legalopen-sourceaccuracyaddressingadvocatesbridgingchatgptclients

0 comments

read the original abstract

This study evaluates the performance of general-purpose AI, like ChatGPT, in legal question-answering tasks, highlighting significant risks to legal professionals and clients. It suggests leveraging foundational models enhanced by domain-specific knowledge to overcome these issues. The paper advocates for creating open-source legal AI systems to improve accuracy, transparency, and narrative diversity, addressing general AI's shortcomings in legal contexts.

This paper has not been read by Pith yet.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

How Can AI Augment Access to Justice? Public Defenders' Perspectives on AI Adoption
cs.CY 2025-10 accept novelty 7.0

Public defenders view AI as most useful for evidence investigation but limited in courtroom work and strategy, with adoption blocked by costs, confidentiality risks, and norms, requiring human oversight and open development.