Pith. sign in

REVIEW 4 major objections 5 minor 37 references

NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysis

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a corpus of 702,945 Indian judgments spanning all court levels, paired with a legal-domain-tuned LLaMa-2 model, achieves roughly 90% macro F1 on judgment prediction while generating expert-rated explanations.

desk verdict Large and genuinely new Indian legal corpus, but the ~90% F1 is inflated by target leakage since labels come from the final 750 words of the same full judgment fed to the model. read the letter →

arxiv 2412.08385 v1 pith:LGHMM3QF submitted 2024-12-11 cs.CL cs.AIcs.IRcs.LG

Shubham Kumar Nigam , Balaramamahanthi Deepak Patnaik , Shivam Mishra , Noel Shallum , Kripabandhu Ghosh , Arnab Bhattacharya This is my paper · ORCID
classification cs.CLcs.AIcs.IRcs.LG
keywords legaljudgmentpredictionIndiancorpusNyayaAnumanaINLegalLlamadomain-specificlanguagemodelcontinuedpretrainingexplainableAIcourt-leveldiversity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

NyayaAnumana is a corpus of 702,945 Indian court judgments spanning the Supreme Court, High Courts, tribunals, district courts, and daily orders, assembled from a raw collection of over 2.2 million documents. The paper's central claim is that this court-level diversity, not model size alone, drives a legal judgment prediction baseline to roughly 90% macro F1, a jump over the prior ~79% results on Indian data. On top of the corpus, the paper builds INLegalLlama, a LLaMa-2-based generative model that is first continued-pretrained on Indian judgments and then supervised-finetuned to produce both a binary or ternary outcome and a rationale. The authors argue that this combination yields predictions and explanations that legal experts rate as accurate and relevant, making the work a step toward AI assistance for a legal system with a large case backlog.

What carries the argument

The load-bearing object is the NyayaAnumana corpus itself, produced by cleaning and labeling 7,02,945 judgments collected from a public legal search engine. Labels come from a keyword heuristic applied to the final 750 words of each judgment: words like "allowed" and "granted" signal acceptance, "rejected" and "dismissed" signal rejection, and negation near these keywords flips the label; cases with mixed outcomes become the "partial" class. The model machinery is INLegalLlama, a LLaMa-2-7B base trained in two phases: continued pretraining on 38,321 Supreme Court and 100,000 High Court judgments, then LoRA-based supervised finetuning on 12,178 PredEx cases with 16 prediction/explanation instruction sets. Ablation tables that train on progressively larger court subsets are what carry the diversity claim, with the full corpus configuration reaching the reported ~90% F1.

What would settle it

Take a random sample of, say, 500 NyayaAnumana test judgments, have legal experts assign labels independently, and also run the trained models on versions of the same judgments with the final 750 words removed. If expert labels disagree with the heuristic labels, or if the redacted-input F1 falls far below the reported ~90%, the headline result is measuring pattern recognition of the decision paragraph rather than prediction from case reasoning.

Watch

Extended reading notes

Core claim

The central discovery is that adding lower-court and daily-order data to Supreme Court and High Court judgments improves macro F1 from about 0.83 to about 0.90 in binary prediction, and that this improvement holds on temporally later test cases from 2020–2024. The authors also find that INLegalLlama's prediction-with-explanation model, trained with 16 instruction templates on expert-annotated PredEx cases, produces explanations whose expert ratings average near 4 on a 1–5 scale, occasionally matching or exceeding the human reference. The paper presents the NyayaAnumana corpus as the resource that makes these gains possible: it is roughly twenty times larger than ILDC and the first Indian LJP corpus to include tribunals, daily orders, and district courts.

Load-bearing premise

The evaluation assumes that labels scraped by keyword heuristics from the final 750 words of each judgment are correct ground truth, and that the full judgment text the model reads does not already contain the outcome.

Editorial extensions

If this is right

  • If the diversity claim is correct, adding more court levels to a legal corpus is a reliable lever for improving LJP accuracy, so future dataset efforts should prioritize coverage over raw size.
  • Trained classifiers keep macro F1 above 0.85 on temporal test sets from 2020–2024, suggesting the gains are not an artifact of the training era.
  • INLegalLlama's joint prediction-and-explanation training yields expert Likert ratings that average near 4 out of 5, supporting the use of generative models for explainable legal AI.
  • Ternary prediction works on Supreme Court cases, but the partial-acceptance class collapses when broader court data is added, so the reliable setting remains binary.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the 90% F1 likely overstates substantive outcome prediction, because the model sees the full judgment text including its final decision; rerunning with the final 750 words removed would measure how much of the score is legal reasoning versus verdict-pattern recognition.
  • Editorial inference: court-level diversity may help partly because each court type has distinctive phrasing and label base rates; a cross-court transfer test (train on High Court only, test on tribunals) would separate vocabulary effects from substantive generalization.
  • Editorial inference: the same keyword-plus-negation labeling recipe could be ported to other common-law judgment archives to create large LJP corpora, but any such corpus should be validated against human labels before being used to benchmark models.
  • Editorial inference: explanation quality rests on a small expert sample (50 and 54 cases), so a larger annotation study with agreement statistics would be needed to confirm that the ratings reflect stable model behavior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces NyayaAnumana, a large corpus of Indian legal judgments (about 703k preprocessed cases) with binary and ternary outcome labels, and INLegalLlama, a LLaMA-2-7B model adapted via continued pretraining and supervised finetuning for judgment prediction and explanation. The authors report roughly 90% F1 for binary prediction on several court-level configurations, claim that diverse court data improves accuracy, and provide expert-based evaluation of generated explanations. The dataset and code are released for reproducibility.

Significance. If the reported performance reflected genuine legal judgment prediction from case facts, the dataset would be a substantial resource for Indian legal NLP, and the findings on data diversity and domain adaptation would be useful to the community. The paper also ships a large, publicly released corpus and a reproducible training pipeline, which are real contributions. However, the central quantitative claims are compromised by a label-generation procedure that makes the prediction target recoverable from the input text, and the temporal and ternary results raise additional validity concerns that must be addressed before the main claims can be accepted.

major comments (4)
  1. [§4.2.3 / §6.1.1 / Appendix E.1 / Figure 3] The evaluation setup has a fundamental target-leakage problem. Appendix E.1 states that labels are generated by keyword heuristics applied to the last 750 words of each judgment, where the decision is summarized, with a context window around terms like 'appeal,' 'petition,' or 'case' and a label-flipping rule for negations. The model input, however, is the full judgment text: Section 6.1.1 chunks each document into 512-token windows with overlap, and the training procedure (Appendix C.1) uses the full train set. Figure 3 illustrates that the label is extracted from the same judgment text that is fed to the model. Consequently, a model can achieve high F1 by locating the operative clause (e.g., 'appeal dismissed' or 'appeal allowed') rather than by predicting the outcome from case facts. The reported ~90% F1 in Table 5 and Appendix A therefore does not measure predictive ability in the intended sense; it measures the model's ability to copy or recognize the concluding decision. This invalidates the headline claim that 'incorporating diverse court data significantly boosts model accuracy' in judgment prediction.
  2. [§4.2.3 / Table 4] The temporal test set is not explicitly shown to be disjoint from the training data. Section 4.2.3 says the 'single' dataset is split into train/validation/test in a 70:10:20 ratio and then adds that the authors 'also tested the model performance on temporal data... from January 2020 to April 2024,' but the paper does not state that documents dated after January 2020 were excluded from the training split. Without such a statement, the temporal results in Table 4 and Tables 5, 15-21 could be contaminated by training-set overlap, making the claimed 'future or unseen data' evaluation uninterpretable. Even if the split is temporally exclusive, the final-750-word leakage described above persists, because a future judgment still contains its own concluding decision in its last 750 words. The authors should clarify the exact construction of the temporal split and, more importantly, re-evaluate on a setting where the input excludes the decision-containing final section.
  3. [Tables 24-26 / Section 8.1 / Appendix A] The ternary classification results contradict the paper's general claim that adding diverse court data boosts accuracy. In Tables 24-26, the 'partially accepted' class (class 2) has macro F1 of 0.00 for all three models when Tribunals and Daily Orders/District Courts are included, and at most 0.31 for the SCI-only setting. This means the models never (or almost never) predict partial acceptance in the larger configurations, despite the dataset containing roughly 1% partial-acceptance examples in those settings (Table 3). The abstract and Appendix A claim 'approximately 90% F1-score in prediction tasks' and that diversity and volume of training data are crucial for high accuracy; these statements are not supported for the three-way task. The paper should either restrict the headline claim to binary classification or provide an analysis of why the ternary model fails on class 2 and what the limits of the proposed dataset are for partial-outcome prediction.
  4. [Section 8.2.3 / Table 8 / Table 28] The expert evaluation is too small and too narrow to support the strength of the claims made. Section 8.2.3 states that the instruction-tuned model 'in some instances even surpasses the quality of explanations provided by human legal experts, achieving an impressive rating score of 4.' However, Table 28 shows that this conclusion rests on 27 (PredEx) and 28 (ILDC_expert) ratings of 4 out of 50 and 54 total cases, respectively, with no rating of 5 on PredEx and only 4 on ILDC_expert. With such small samples, the difference between models (e.g., INLegalLlama versus LLaMa-2 SFT on ILDC_expert: 3.67 versus 3.30) is not shown to be statistically reliable, and the claim that the model surpasses human experts is not justified. The authors should report variance or significance tests, or temper the claim to a qualitative observation.
minor comments (5)
  1. [Title] The title contains a typo: 'INL EGAL LLAMA' should be 'INLegalLlama.'
  2. [Table 1] The description 'largest Indian legal corpus for judgment prediction' is potentially misleading because Table 1 lists CAIL as having 2.67 million cases; the qualifiers 'Indian' and 'English-language' should be stated more explicitly in the abstract and introduction.
  3. [Section 8.1] The sentence 'Suggests that the inclusion of domain-specific data significantly enhances performance' is a fragment; it should be integrated with the preceding sentence.
  4. [Section 4.2] The distinction between the 'single' and 'multi' splits is used throughout the results but defined only implicitly in Appendix E.1; a clearer definition in Section 4.2 would improve readability.
  5. [Section 5.1] The continued pretraining is described as using 'a comprehensive Indian legal corpus' in the abstract, but Section 5.1 (and Section 4.2.1) explains that only a subset (38,321 SCI cases and 100,000 High Court cases) was used; the text should be consistent in describing the scale of the CPT corpus.

Circularity Check

2 steps flagged · score 7.0 of 10

The ~90% F1 prediction claim reduces to locating the operative clause: labels are extracted from the final 750 words of the same judgment text that is fed to the model as input.

  1. self definitional [Appendix E.1 (Label Making) with Section 3 (Prediction Task) and Section 6.1.1 (Language Model based)]
    "To ensure accurate labeling, we focused on the last 750 words of each document, typically where decisions are summarized. Special attention was given to a context window around key terms like 'appeal,' 'petition,' or 'case' to accurately determine the judgment nature."

    The target y is defined by a keyword heuristic applied to the final 750 words of document D, and Section 3 defines the task as 'Given a document D, the task is to predict the decision y.' Section 6.1.1 then feeds the complete judgment to the classifiers by splitting it into 512-token chunks with overlap, so the exact concluding text from which y was extracted is part of the model input. A model can therefore reach the reported ~90% F1 by copying or locating the final operative clause (e.g., 'appeal is allowed' or 'appeal is dismissed') rather than by predicting an unseen legal outcome. The 'prediction' is, by construction, a deterministic function of the input document, so the headline accuracy claim does not measure genuine judgment prediction.

  2. other [Section 4.2.3 and Table 4 (temporal evaluation)]
    "We also tested the model performance on temporal data, assessing its effectiveness on future or unseen data from January 2020 to April 2024 to ensure robustness and generalization capabilities over time, as detailed in Table 4."

    The temporal test cases are processed with the same Appendix E.1 labeling procedure that scans the last 750 words of each judgment, and the same full-text chunking is used during evaluation. Future judgments from 2020-2024 still contain their own final decisions inside the input text. Thus the temporal F1 values (e.g., HCs 2020-24, ~0.89) measure the model's ability to extract the conclusion from the document, not its ability to predict an outcome that is not already present in the input. The temporal split changes the time of the documents but does not remove the constructional identity between input and target.

full rationale

The central circular step is the dataset labeling protocol. NyayaAnumana's labels are not expert annotations of a separate outcome; they are generated automatically from the final 750 words of each judgment, and the models are trained and evaluated on the full judgment text, including those final words. Figure 3 illustrates label extraction directly from the concluding paragraphs, confirming that the 'prediction target' is a substring of the model input. Consequently, the abstract's claim that 'incorporating diverse court data significantly boosts model accuracy, achieving approximately 90% F1' cannot be interpreted as evidence of legal prediction ability; it is evidence of operative-clause extraction, and the diversity comparisons in Tables 5, 15-23 inherit this leakage. The temporal-split evaluation does not cure the problem because future judgments still contain their decisional text in the last 750 words. The reuse of PredEx (Nigam et al., 2024b) for SFT and for the 50-case expert evaluation is a self-citation by the same first author, but it is not load-bearing for the main 90% claim, and the explanation ratings are given by independent legal experts, so I do not count PredEx reuse as a separate circularity. The ternary results are not part of the 90% claim and in fact show class-2 F1 collapsing to 0.00-0.31, which indicates the auto-labeled partial-acceptance category is unstable rather than circular. Overall, the paper contains a real, construction-level leakage that undermines its headline result: the target is derived from the same text supplied as the predictive input.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the validity of automatically generated labels and the absence of target leakage. The hand-chosen label extraction window and keyword lists act as free parameters that shape the dataset, and the input-format assumption is a domain assumption that is not justified.

free parameters (2)
  • Label extraction window = Last 750 words of each judgment
    The final 750 words are assumed to contain the decision; this hand-chosen constant defines the dataset labels and directly determines the prediction target.
  • Keyword lists for outcome classification = allowed, granted, approved vs rejected, dismissed, etc.
    These keywords are chosen ad hoc to assign labels; no validation against human annotations is reported, so the label quality is a free design choice.
assumptions (3)
  • domain assumption Heuristic labels from final 750 words are reliable ground truth
    The entire evaluation rests on the assumption that the automatic keyword-based labels are correct. Appendix E.1 describes the heuristics but provides no human validation for NyayaAnumana.
  • domain assumption Full judgment text is a valid input for prediction
    The paper does not use a facts-only input as in prior ILDC work, and it does not address that the input includes the final order containing the outcome. This assumption is needed for the accuracy claim to be meaningful.
  • domain assumption Temporal test split is truly future data
    The paper does not state that cases from January 2020 to April 2024 were excluded from the training set. If they were not held out, the temporal evaluation is invalid.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysis." pith.science (2026). https://pith.science/paper/LGHMM3QF

@misc{pith2026241208385,
  author       = {Pith},
  title        = {Pith review of: NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LGHMM3QF}},
  note         = {Machine review of arXiv:2412.08385}
}
read the original abstract

The integration of artificial intelligence (AI) in legal judgment prediction (LJP) has the potential to transform the legal landscape, particularly in jurisdictions like India, where a significant backlog of cases burdens the legal system. This paper introduces NyayaAnumana, the largest and most diverse corpus of Indian legal cases compiled for LJP, encompassing a total of 7,02,945 preprocessed cases. NyayaAnumana, which combines the words "Nyay" (judgment) and "Anuman" (prediction or inference) respectively for most major Indian languages, includes a wide range of cases from the Supreme Court, High Courts, Tribunal Courts, District Courts, and Daily Orders and, thus, provides unparalleled diversity and coverage. Our dataset surpasses existing datasets like PredEx and ILDC, offering a comprehensive foundation for advanced AI research in the legal domain. In addition to the dataset, we present INLegalLlama, a domain-specific generative large language model (LLM) tailored to the intricacies of the Indian legal system. It is developed through a two-phase training approach over a base LLaMa model. First, Indian legal documents are injected using continual pretraining. Second, task-specific supervised finetuning is done. This method allows the model to achieve a deeper understanding of legal contexts. Our experiments demonstrate that incorporating diverse court data significantly boosts model accuracy, achieving approximately 90% F1-score in prediction tasks. INLegalLlama not only improves prediction accuracy but also offers comprehensible explanations, addressing the need for explainability in AI-assisted legal decisions.

Figures

Figures reproduced from arXiv: 2412.08385 by the authors.

Figure 1
Figure 1. INLegalLlama flow diagram 12 in the Appendix. In addition, for instruction tuning, we adopted prompts from (Nigam et al., 2024b) for prediction tasks, as listed in [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Distribution of cases in different courts. [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the LJP Task Framework [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 12 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Nikolaos Aletras, Dimitrios Tsarapatsanis, Daniel Preo t iuc-Pietro, and Vasileios Lampos. 2016. Predicting judicial decisions of the european court of human rights: A natural language processing perspective. PeerJ computer science, 2:e93

    work page 2016

  4. [4]

    Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909 METEOR : An automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pages 65--72, Ann Arbor, Michigan. Association for Comput...

    2005

  5. [5]

    Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. Neural legal judgment prediction in english. Association for Computational Linguistics (ACL)

    work page 2019

  6. [6]

    Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, and Prodromos Malakasiotis. 2021. https://doi.org/10.18653/v1/2021.naacl-main.22 Paragraph-level rationale extraction through regularization: A case study on E uropean court of human rights cases . In Proceedings of the 2021 Conference of the North American...

    doi:10.18653/v1/2021.naacl-main.22 2021

  7. [7]

    Yi Feng, Chuanyi Li, Jidong Ge, Bin Luo, and Vincent Ng. 2021. Recommending statutes: A portable method based on neural networks. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(2):1--22

    work page 2021

  8. [8]

    Debasis Ganguly, Jack G Conrad, Kripabandhu Ghosh, Saptarshi Ghosh, Pawan Goyal, Paheli Bhattacharya, Shubham Kumar Nigam, and Shounak Paul. 2023. Legal ir and nlp: the history, challenges, and state-of-the-art. In European Conference on Information Retrieval, pages 331--340. Springer

    work page 2023

Show all 37 references
  1. [9]

    Saptarshi Ghosh, Kripabandhu Ghosh, Debasis Ganguly, Arnab Bhattacharya, Partha Pratim Chakrabarti, Shouvik Guha, Arindam Pal, Koustav Rudra, Prasenjit Majumder, Dwaipayan Roy, et al. 2023. Report on the 2nd symposium on artificial intelligence and law (sail) 2022. In ACM SIGI...

    2023

  2. [10]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  3. [11]

    Quzhe Huang, Mingxu Tao, Chen Zhang, Zhenwei An, Cong Jiang, Zhibin Chen, Zirui Wu, and Yansong Feng. 2023. Lawyer llama technical report. arXiv preprint arXiv:2305.15062

    2023 arXiv

  4. [12]

    Arnav Kapoor, Mudit Dhawan, Anmol Goel, Arjun T H, Akshala Bhatnagar, Vibhu Agrawal, Amul Agrawal, Arnab Bhattacharya, Ponnurangam Kumaraguru, and Ashutosh Modi. 2022. https://doi.org/10.18653/v1/2022.findings-acl.278 HLDC : H indi legal documents corpus . In Findings of the A...

    2022 doi

  5. [13]

    Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  6. [14]

    Andr \'e Lage-Freitas, H \'e ctor Allende-Cid, Orivaldo Santana, and L \' via Oliveira-Lage. 2022. Predicting brazilian court decisions. PeerJ Computer Science, 8:e904

    2022

  7. [15]

    Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013 ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  8. [16]

    Vijit Malik, Rishabh Sanjay, Shouvik Kumar Guha, Angshuman Hazarika, Shubham Nigam, Arnab Bhattacharya, and Ashutosh Modi. 2021 a . Semantic segmentation of legal documents via rhetorical roles. arXiv preprint arXiv:2112.01836

    2021 arXiv

  9. [17]

    Vijit Malik, Rishabh Sanjay, Shubham Kumar Nigam, Kripabandhu Ghosh, Shouvik Kumar Guha, Arnab Bhattacharya, and Ashutosh Modi. 2021 b . https://doi.org/10.18653/v1/2021.acl-long.313 ILDC for CJPE : I ndian legal documents corpus for court judgment prediction and explanation ....

    2021 doi

  10. [18]

    Masha Medvedeva, Michel Vols, and Martijn Wieling. 2020. Using machine learning to predict decisions of the european court of human rights. Artificial Intelligence and Law, 28:237--266

    2020

  11. [19]

    Shubham Kumar Nigam and Aniket Deroy. 2024. https://doi.org/10.1145/3632754.3632765 Fact-based court judgment prediction . In Proceedings of the 15th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE '23, page 78–82, New York, NY, USA. Association for Comp...

    2024

  12. [20]

    Shubham Kumar Nigam, Aniket Deroy, Subhankar Maity, and Arnab Bhattacharya. 2024 a . https://doi.org/10.18653/v1/2024.nllp-1.6 Rethinking legal judgement prediction in a realistic scenario in the era of large language models . In Proceedings of the Natural Legal Language Proce...

    2024 doi

  13. [21]

    Shubham Kumar Nigam, Aniket Deroy, Noel Shallum, Ayush Kumar Mishra, Anup Roy, Shubham Kumar Mishra, Arnab Bhattacharya, Saptarshi Ghosh, and Kripabandhu Ghosh. 2023 a . https://doi.org/10.18653/v1/2023.semeval-1.180 Nonet at S em E val-2023 task 6: Methodologies for legal eva...

    2023 doi

  14. [22]

    Shubham Kumar Nigam, Navansh Goel, and Arnab Bhattacharya. 2022. nigam@ coliee-22: Legal case retrieval and entailment using cascading of lexical and semantic-based models. In JSAI International Symposium on Artificial Intelligence, pages 96--108. Springer

    2022

  15. [23]

    Shubham Kumar Nigam, Shubham Kumar Mishra, Ayush Kumar Mishra, Noel Shallum, and Arnab Bhattacharya. 2023 b . Legal question-answering in the indian context: Efficacy, challenges, and potential of modern ai models. arXiv preprint arXiv:2309.14735

    2023 arXiv

  16. [24]

    Shubham Kumar Nigam, Anurag Sharma, Danush Khanna, Noel Shallum, Kripabandhu Ghosh, and Arnab Bhattacharya. 2024 b . https://arxiv.org/abs/2406.04136 Legal judgment reimagined: Predex and the rise of intelligent ai interpretation in indian courts . Preprint, arXiv:2406.04136

    2024 arXiv

  17. [25]

    Joel Niklaus, Ilias Chalkidis, and Matthias St \"u rmer. 2021. Swiss-judgment-prediction: A multilingual legal judgment prediction benchmark. arXiv preprint arXiv:2110.00806

    2021 arXiv

  18. [26]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  19. [27]

    Shounak Paul, Arpan Mandal, Pawan Goyal, and Saptarshi Ghosh. 2023. https://arxiv.org/abs/2209.06049 Pre-trained language models for the legal domain: A case study on indian law . In Proceedings of 19th International Conference on Artificial Intelligence and Law - ICAIL 2023

    2023 arXiv

  20. [28]

    Octavia-Maria Sulea, Marcos Zampieri, Shervin Malmasi, Mihaela Vela, Liviu P Dinu, and Josef Van Genabith. 2017. Exploring the use of text classification in the legal domain. arXiv preprint arXiv:1710.09306

    2017 arXiv

  21. [29]

    Aman Tiwari, Prathamesh Kalamkar, Atreyo Banerjee, Saurabh Karn, Varun Hemachandran, and Smita Gupta. 2024. Aalap: Ai assistant for legal & paralegal functions in india. arXiv preprint arXiv:2402.01758

    2024 arXiv

  22. [30]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  23. [31]

    Vasilyev, Vedant Dharnidharka, and John Bohannon

    Oleg V. Vasilyev, Vedant Dharnidharka, and John Bohannon. 2020. https://arxiv.org/abs/2002.09836 Fill in the BLANC: human-free quality estimation of document summaries . CoRR, abs/2002.09836

    2020 arXiv

  24. [32]

    Shaurya Vats, Atharva Zope, Somsubhra De, Anurag Sharma, Upal Bhattacharya, Shubham Nigam, Shouvik Guha, Koustav Rudra, and Kripabandhu Ghosh. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.831 LLM s -- the good, the bad or the indispensable?: A use case on legal statut...

    2023 doi

  25. [33]

    Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, et al. 2018. Cail2018: A large-scale legal dataset for judgment prediction. arXiv preprint arXiv:1807.02478

    2018 arXiv

  26. [34]

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32

    2019

  27. [35]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with BERT . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April...

    2020

  28. [36]

    Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. https://doi.org/10.18653/v1/D18-1521 Learning gender-neutral word embeddings . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847--4853, Brussels, Belgium. A...

    2018 doi

  29. [37]

    Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. https://doi.org/10.18653/v1/2020.acl-main.466 How does NLP benefit legal system: A summary of legal artificial intelligence . In Proceedings of the 58th Annual Meeting of the Association...

    2020 doi

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.