Pith. sign in

REVIEW 3 major objections 5 minor 39 references

Aligning Language Models for Icelandic Legal Text Summarization

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that preference training (DPO and RLHF) improves the legal accuracy of Icelandic legal summaries without improving the quality of the Icelandic language itself, and that ROUGE scores can rank models opposite to…

desk verdict The abstract overclaims: preference training improves legal accuracy in only one of three pairwise comparisons, but the paper is an honest, useful low-resource study. read the letter →

arxiv 2504.18180 v1 pith:L62WZAIE submitted 2025-04-25 cs.CL cs.AIcs.LG

{TH}órir Hrafn Har{ḥarson , Hrafn Loftsson , Stefán Ólafsson This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords IcelandiclegaltextsummarizationDirectPreferenceOptimizationReinforcementLearningfromHumanFeedbacklanguagemodelsROUGEevaluationlow-resourceNLPdomain-specificpre-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether preference-based training—RLHF and DPO—can make language models generate Icelandic legal summaries that legal professionals prefer over summaries from plain supervised fine-tuning. Using two open models further pre-trained on Icelandic court rulings and then instruction-tuned on lawyer-written summaries, the authors find that preference training improves the legal accuracy of generated summaries but does not consistently improve the quality of the Icelandic itself. The paper also reports a sharp split between automated ROUGE scores and expert rankings: the model with the highest ROUGE was ranked last by a legal expert, while the expert's top choices had lower ROUGE. The upshot is that for low-resource legal language, human qualitative evaluation is not merely a supplement to automated metrics but a necessary check, and language-specific pre-training remains the main lever for language quality.

What carries the argument

The load-bearing mechanism is a three-stage training pipeline. Stage one further pre-trains a base model on Icelandic court rulings only (next-token prediction on packed 512-token blocks); stage two is supervised instruction fine-tuning on court rulings paired with lawyer-written summaries; stage three is preference training on a pairwise dataset built from the stage-two model's highest-ROUGE outputs. DPO converts reward maximization into a classification loss over preferred and rejected pairs, while RLHF trains a reward model on the same pairs and optimizes the policy with PPO, with reward normalization (zero mean, unit variance) needed to stop the policy from exploiting the reward model. Perplexity on legal text and ROUGE on the test set measure progress at each stage, and a legal expert's 1-to-5 scores for legal accuracy and Icelandic quality provide the final judgement.

What would settle it

Take the same trained models and have a panel of at least five legal experts rate a random sample of 100 or more summaries, blind to model identity, on separate legal-accuracy and Icelandic-quality scales; if DPO or RLHF models do not systematically beat the supervised-only baselines on legal accuracy, or if ROUGE rankings and expert rankings agree once more summaries are rated, the paper's central claim would fail.

Watch

Extended reading notes

Core claim

The central claim is that applying DPO or RLHF on top of domain-specific pre-training and supervised instruction fine-tuning improves the legal accuracy of Icelandic court-summary generation without a corresponding improvement in general Icelandic language quality. In the authors' own evaluation, the RLHF-tuned GPT-SW3 model and the supervised-only GPT-SW3 model were the expert's most-preferred outputs, both clearly ahead of the DPO variant; meanwhile GPT-SW3-DPO achieved the highest ROUGE score yet received the lowest expert legal-accuracy score, and Llama2-DPO improved legal accuracy over Llama2-SFT while leaving Icelandic quality essentially unchanged. The paper interprets this as evidence that preference training can add domain precision, but that ROUGE is not a trustworthy proxy for legal-expert preference and that language-specific pre-training, not preference optimization, is what determines written Icelandic quality.

Load-bearing premise

The paper's conclusion rests on one legal expert's ratings of 25 court-ruling summaries, with only five rulings independently checked by two other experts; if those sparse ratings are not representative of the 300-ruling test set, the reported ROUGE-versus-expert discrepancy may reflect rater idiosyncrasy rather than a real property of the models.

Editorial extensions

If this is right

  • In Icelandic legal summarization, DPO or RLHF should be treated as targeted additions for legal accuracy, not as fixes for language quality.
  • ROUGE alone is insufficient for model selection in this setting; expert preference can invert ROUGE rankings.
  • Language-specific pre-training sets the ceiling on Icelandic quality: GPT-SW3's outputs outranked the larger Llama2 variants on expert language scores.
  • DPO is cheaper and easier to stabilize but prone to overfitting; RLHF is harder and compute-heavy yet produced the expert's top-ranked outputs.
  • All model outputs remained far below human-written summaries, particularly on legal accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the preference pairs had been selected by expert preference instead of by ROUGE at stage three, the DPO and RLHF gains in legal accuracy might be larger; the authors themselves suggest this but did not test it.
  • The ROUGE-expert gap may be intrinsic to court summaries, which are concise outcome descriptions with low n-gram overlap with the ruling; under such conditions ROUGE measures lexical similarity rather than legal correctness.
  • The same pattern may hold in other low-resource languages with strict register requirements: preference training helps only after a strong language-model base, so language-specific pre-training investment should precede alignment work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper investigates whether preference-based training (RLHF and DPO) improves the quality of Icelandic legal text summarization over standard supervised fine-tuning (SFT). The authors take two base models (GPT-SW3-1.3B and Llama2-7B), further pre-train them on domain-specific Icelandic legal text, instruction-tune them on court rulings and their human-written summaries, and then apply DPO or RLHF. Evaluation uses perplexity, ROUGE scores on a 300-item test set, and a human expert evaluation of 25 rulings (with two additional experts for five rulings) covering ranked preferences, Icelandic language quality, and legal accuracy. The abstract claims that preference training improves legal accuracy over SFT but does not improve overall Icelandic quality, and that automated metrics diverge from human judgment. The paper's own Section 6 more cautiously states that improvements are 'not consistently' observed. The core empirical contribution is the head-to-head comparison of DPO and RLHF in a low-resource language with expert human evaluation, but the reported evidence only partially supports the stated headline conclusion.

Significance. If the results are interpreted carefully, the paper makes a useful empirical contribution to legal NLP for low-resource languages. Strengths include the use of professional legal experts for evaluation, the direct comparison of DPO and RLHF on the same task, and the demonstration that ROUGE improvements need not correspond to expert-preferred legal accuracy (Tables 9 and 10). The study also provides a practical example of the instability of PPO training and the need for reward normalization. However, the significance is limited by the small human-evaluation sample, the lack of statistical testing, and a headline claim that outruns the data. The paper would be more valuable if the conclusions were explicitly restricted to the observed pairwise comparisons and presented as a case study rather than as a general finding about preference training for legal summarization.

major comments (3)
  1. [Abstract and Table 8] The abstract states that 'preference training improves the legal accuracy of generated summaries over standard fine-tuning', but Table 8 does not support this as a general claim. In the three pairwise comparisons, legal accuracy improves only for Llama2-DPO over Llama2-SFT (2.52 vs. 2.04); it decreases for GPT-SW3-RLHF (2.56 vs. 2.68) and for GPT-SW3-DPO (1.96 vs. 2.68). The pooled mean legal accuracy across preference-trained variants (2.35) is essentially equal to the SFT mean (2.36). The paper's own Section 6 acknowledges that the effect is 'not consistently' observed. The abstract and Section 1 need to be revised to reflect the actual pattern, for instance by reporting the inconsistent direction and noting that the only positive result is for Llama2.
  2. [Sections 3.4 and 6] The preference dataset was constructed by selecting responses with top ROUGE scores after SFT, not by human preference judgments, as acknowledged in Section 6. This creates a self-reinforcing loop between the DPO training signal and the ROUGE evaluation metric, and it weakens the claim that the study evaluates alignment with 'user preferences.' The paper should either label this as 'metric-based preference training' or provide a human-preference benchmark for the preference dataset itself. This is a load-bearing issue because the central ROUGE-vs-expert discrepancy is partly explained by the fact that the training signal was already ROUGE-oriented.
  3. [Section 3.5 and Tables 6-8] The human evaluation is based on a single primary expert scoring 25 rulings, with only five rulings scored by two additional experts. No inter-annotator agreement statistic (e.g., Cohen's kappa) is reported, and Table 7 shows nontrivial disagreement (e.g., GPT-SW3-DPO average rank 4.2 for the primary expert vs. 3.2 for the other experts). With this small, partly inconsistent sample, the paper's claims about 'legal accuracy' and 'Icelandic language usage' are fragile. The authors should report per-ruling variance, agreement measures, and ideally confidence intervals or at least acknowledge the limited reliability of the ground truth. This concern applies directly to the abstract's second claim that preference training 'does not significantly enhance' Icelandic quality, as an absence of a significant difference in a 25-item sample cannot be interpreted as evidence of no effect.
minor comments (5)
  1. [Section 4] The text says the model was 'pre-trained on the ICG sub-corpus' but the corpus is the Icelandic Gigaword Corpus (IGC); this appears to be a typo.
  2. [Section 5.2] The sentence 'The same limitations can also be observed in in Table 10' contains a duplicated 'in'.
  3. [Table 7] The column heading 'Comparison' is unclear; it should specify whether it is the average of the two additional experts or some other pooled measure.
  4. [Figure 1] Figure 1 is referenced in Section 4.2.2 but is not present in the provided manuscript text; please ensure the figure is included in the final submission.
  5. [Tables 3-5 and 8] No confidence intervals or significance tests are reported for the ROUGE differences or the expert score differences. Given the small evaluation sets, adding bootstrap confidence intervals or at least standard deviations would help readers judge the stability of the reported differences.

Circularity Check

1 steps flagged · score 2.0 of 10

Mild self-referential ROUGE loop in DPO evaluation, but the central human-evaluation claim is independent.

  1. fitted input called prediction [Section 6, Conclusions (dataset construction admission); Section 4.2.1, DPO ROUGE evaluation]
    "The dataset for pairwise comparison was based on responses with top ROUGE scores post fine-tuning. A more effective approach might involve gathering human feedback at this stage to identify which model is best suited for generating data for further training."

    The preference-training input was constructed by ranking SFT model outputs with ROUGE, and the resulting DPO models were then evaluated with ROUGE in Table 4, reporting 'significant improvements for GPT-SW3-1.3B.' Because the training preference signal and the headline automatic metric are the same n-gram-overlap measure, the claimed ROUGE gain is partly a re-measurement of the training criterion rather than an independent prediction. However, the paper's central legal-accuracy and Icelandic-quality claims come from expert scoring in Tables 6-8, which is independent of ROUGE, so the circularity is limited to the automatic-metric narrative.

full rationale

This is an empirical paper with no mathematical derivation chain. The only potentially circular element is that the DPO preference dataset was generated by ranking SFT outputs with ROUGE, and the DPO models were then evaluated with ROUGE; since both the training preference signal and the reported automatic metric are ROUGE, the DPO ROUGE improvement in Table 4 is a partly self-referential result. The paper explicitly admits this in Section 6. This does not affect the central human-evaluation findings: the legal-accuracy and Icelandic-quality conclusions are based on independent expert scores in Tables 6-8, and those scores actually diverge from ROUGE. Therefore the circularity is minor and localized. The abstract's generalization that preference training 'improves legal accuracy' is not well supported by Table 8 (only one of three preference variants beats its SFT baseline), but that is an internal-consistency and correctness concern, not circularity. No load-bearing self-citation or imported uniqueness theorem was found.

Assumptions & free parameters 8 free parameters · 3 assumptions · 0 invented entities

The free parameters are training hyperparameters tuned on validation data; none are physical constants. The axioms are domain assumptions about the gold summaries and expert ratings. No invented entities are introduced.

free parameters (8)
  • LoRA rank for Ice-Llama2 pre-training on IGC sub-corpus = 1024
    Chosen based on finding that increasing rank consistently lowered loss during phase 1; not justified by theory.
  • LoRA rank for phase 1 legal pre-training = 256
    Set lower than 1024 for Llama2 on IGC; hand-picked without reported ablation.
  • LoRA rank for phase 2 summarization fine-tuning = 128
    Hand-picked without reported ablation.
  • Supervised fine-tuning epochs = 5
    Selected because GPT-SW3-1.3B achieved highest ROUGE after 5 epochs on validation set.
  • DPO epochs for GPT-SW3 = 2 after 5 SFT epochs
    Tuned by ROUGE; performance plateaued after 3 epochs.
  • DPO epochs for Llama2-7B = 2 after 1 SFT epoch
    Tuned by ROUGE; overfitting risk with larger model.
  • RLHF training epochs = 1
    Necessary to avoid overfitting of reward model.
  • PPO reward normalization = mean=0, std=1
    Required to stabilize PPO training after divergence.
assumptions (3)
  • domain assumption Human-written summaries from the Icelandic Supreme Court website are valid gold references for summary quality.
    Used as training targets and as baseline in Table 8; if these summaries are not representative, the SFT and evaluation are misaligned.
  • domain assumption A single primary legal expert's scores on 25 rulings provide a reliable measure of legal accuracy and Icelandic quality.
    The central discrepancy finding relies on these scores; only 5 rulings were double-scored and agreement was imperfect.
  • domain assumption ROUGE is a meaningful proxy for summary quality when selecting preference pairs and tuning hyperparameters.
    The paper uses ROUGE to select the SFT model for generating preference data and to pick epochs; later it shows ROUGE disagrees with human judgment, so this assumption is partly undermined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning Language Models for Icelandic Legal Text Summarization." pith.science (2026). https://pith.science/paper/L62WZAIE

@misc{pith2026250418180,
  author       = {Pith},
  title        = {Pith review of: Aligning Language Models for Icelandic Legal Text Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L62WZAIE}},
  note         = {Machine review of arXiv:2504.18180}
}
read the original abstract

The integration of language models in the legal domain holds considerable promise for streamlining processes and improving efficiency in managing extensive workloads. However, the specialized terminology, nuanced language, and formal style of legal texts can present substantial challenges. This study examines whether preference-based training techniques, specifically Reinforcement Learning from Human Feedback and Direct Preference Optimization, can enhance models' performance in generating Icelandic legal summaries that align with domain-specific language standards and user preferences. We compare models fine-tuned with preference training to those using conventional supervised learning. Results indicate that preference training improves the legal accuracy of generated summaries over standard fine-tuning but does not significantly enhance the overall quality of Icelandic language usage. Discrepancies between automated metrics and human evaluations further underscore the importance of qualitative assessment in developing language models for the legal domain.

Figures

Figures reproduced from arXiv: 2504.18180 by the authors.

Figure 1
Figure 1. Mean reward and KL-divergence for GPT-SW3 1.3B after 20 epochs of training using the PPO reinforcement learning algorithm. Due to the substantial GPU resources required for RLHF, this training method was only applied to the smaller GPT-SW3 model. As with previ￾ous evaluations, the model’s performance was as￾sessed by calculating the ROUGE score. How￾ever, in contrast to its DPO-trained counterpart, the RLHF model di… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 19 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Starkaður Barkarson, Steinþór Steingrímsson, and Hildur Hafsteinsdóttir. 2022. Evolving Large Text Corpora: Four Versions of the Icelandic Gigaword Corpus . In Proceedings of the Language Resources and Evaluation Conference, page 11, Marseille, France

    work page 2022

  4. [4]

    Andrew Blair-Stanek, Nils Holzenberger, and Benjamin Van Durme. 2023. http://arxiv.org/abs/2302.06100 Can GPT-3 Perform Statutory Reasoning? In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, ICAIL '23, page 22–31. Association for Computing Machinery

    arXiv 2023

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    work page 2020

  6. [6]

    Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. http://arxiv.org/abs/2010.02559 LEGAL-BERT: The Muppets straight out of Law School . In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2898--2904. Association for Computational Linguistics

    arXiv 2020

  7. [7]

    Choi, Kristin E

    Jonathan H. Choi, Kristin E. Hickman, Amy Monahan, and Daniel Schwarcz. 2022. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4335905 ChatGPT Goes to Law School . Journal of Legal Education, 71:387

    work page 2022

  8. [8]

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf Deep Reinforcement Learning from Human Preferences . In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc

    work page 2017

Show all 39 references
  1. [9]

    Corinna Coupette, Janis Beckedorf, Dirk Hartung, Michael James Bommarito, and Daniel Martin Katz. 2021. Measuring Law Over Time: A Network Analytical Framework with an Application to Statutes and Regulations in the United States and Germany . Frontiers in Physics 9

    2021

  2. [10]

    Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk, Joey \"O hman, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, Judit Casademont, and Magnus Sahlgren. 2024. https://aclanthology.org/2024.lrec-main.695 GPT-SW3: An Autoregressive Language Model for the Scandinavian La...

    2024

  3. [11]

    Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig...

    2022 arXiv

  4. [12]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 LoRA: Low-Rank Adaptation of Large Language Models . In International Conference on Learning Representations

    2022

  5. [13]

    Deepali Jain, Malaya Dutta Borah, and Anupam Biswas. 2021. https://doi.org/https://doi.org/10.1016/j.cosrev.2021.100388 Summarization of legal documents: Where are we now and the way forward . Computer Science Review, 40:100388

    2021

  6. [14]

    Eva Hrönn Jónsdóttir. 2023. Helmingur hefur velt fyrir sér að skipta um starfsvettvang. Lögmannablaðið, 03/23:16--18

    2023

  7. [15]

    Daniel Martin Katz, Corinna Coupette, Janis Beckedorf, and Dirk Hartung. 2020. https://doi.org/10.1038/s41598-020-73623-x Complex societies and the growth of the law . Scientific Reports, 10(1)

    2020 doi

  8. [16]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f78...

    2020

  9. [17]

    Daniele Licari, Praveen Bushipaka, Gabriele Marino, Giovanni Comand\' e , and Tommaso Cucinotta. 2023. https://doi.org/10.1145/3594536.3595177 Legal holding extraction from italian case documents using italian-legal-bert text summarization . In Proceedings of the Nineteenth In...

    2023

  10. [18]

    Daniele Licari and Giovanni Comandè. 2024. https://www.sciencedirect.com/science/article/pii/S0267364923001188 ITALIAN-LEGAL-BERT models for improving natural language processing tasks in the Italian legal domain . Computer Law & Security Review, 52:105908

    2024

  11. [19]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  12. [20]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. http://arxiv.org/abs/1907.11692 RoBERTa: A Robustly Optimized BERT Pretraining Approach . ArXiv

    2019 arXiv

  13. [21]

    Marion Nickum and Pascale Desrumaux. 2023. https://doi.org/10.1080/13218719.2022.2035840 Burnout among lawyers: effects of workload, latitude and mediation via engagement and over-engagement . Psychiatry, Psychology and Law, 30(3):349--361

    2023

  14. [22]

    Anna Nikul \'a sd \'o ttir, J \'o n Gu nason, Anton Karl Ingason, Hrafn Loftsson, Eir \' kur R \"o gnvaldsson, Einar Freyr Sigur sson, and Stein \'o r Steingr \' msson. 2020. https://aclanthology.org/2020.lrec-1.418 Language Technology Programme for Icelandic 2019--2023 . In P...

    2020

  15. [23]

    Tammy Pettinato Oltz. 2023. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4347630 Chatgpt, professor of law . University of Illinois Journal of Law, Technology & Policy, page 207

    2023

  16. [24]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022 arXiv

  17. [25]

    Giuseppe Pisano, Alessia Fidelangeli, Federico Galli, Andrea Loreggia, Riccardo Rovatti, Piera Santin, and Giovanni Sartor. 2024. https://doi.org/10.6092/issn.1825-1927/19618 Summarization of tax rulings in the PRODIGIT projec . i-lex, 17(1):1–26

    2024 doi

  18. [26]

    Alec Radford and Karthik Narasimhan. 2018. https://api.semanticscholar.org/CorpusID:49313245 Improving Language Understanding by Generative Pre-Training

    2018

  19. [27]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. Direct preference optimization: your language model is secretly a reward model . In Proceedings of the 37th International Conference on Neural Information Processing S...

    2024

  20. [28]

    Marijn Schraagen, Floris Bex, Nick Van De Luijtgaarden, and Dani \"e l Prijs. 2022. https://doi.org/10.18653/v1/2022.nllp-1.7 Abstractive Summarization of Dutch Court Verdicts Using Sequence-to-sequence Models . In Proceedings of the Natural Legal Language Processing Workshop ...

    2022 doi

  21. [29]

    V \'e steinn Sn bjarnarson, Haukur Barri S \' monarson, P \'e tur Orri Ragnarsson, Svanhv \' t Lilja Ing \'o lfsd \'o ttir, Haukur J \'o nsson, Vilhjalmur Thorsteinsson, and Hafsteinn Einarsson. 2022. https://aclanthology.org/2022.lrec-1.464 A Warm Start and a Clean Crawled Co...

    2022

  22. [30]

    Steinþór Steingrímsson, Sigrún Helgadóttir, Eiríkur Rögnvaldsson, Starkaður Barkarson, and Jón Guðnason. 2018. Risamálheild: A Very Large Icelandic Text Corpus . In Proceedings of LREC 2018, Myazaki, Japan

    2018

  23. [31]

    Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020. Learning to summarize from human feedback. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIP...

    2020

  24. [32]

    Peter M. Tiersma. 1999. Legal Language , 1 edition. University of Chicago Press

    1999

  25. [33]

    Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M

    Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cant \'o n Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes,...

    2023 arXiv

  26. [34]

    Dietrich Trautmann, Alina Petrova, and Frank Schilder. 2022. http://arxiv.org/abs/2212.02199 Legal Prompt Engineering for Multilingual Legal Judgement Prediction . ArXiv

    2022 arXiv

  27. [35]

    Rush, and Thomas Wolf

    Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2024. http://arxiv.org/abs/2310.16944 Zephy...

    2024 arXiv

  28. [36]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. http://arxiv.org/abs/1706.03762 Attention is all you need . In Advances in Neural Information Processing Systems, pages 5998--6008

    2017 arXiv

  29. [37]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  30. [38]

    Fangyi Yu, Lee Quartey, and Frank Schilder. 2022. http://arxiv.org/abs/2212.01326 Legal Prompting: Teaching a Language Model to Think Like a Lawyer . ArXiv

    2022 arXiv

  31. [39]

    Joey Öhman, Severine Verlinden, Ariel Ekgren, Amaru Cuba Gyllensten, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, and Magnus Sahlgren. 2023. http://arxiv.org/abs/2303.17183 The Nordic Pile: A 1.2TB Nordic Dataset for Language Modeling . ArXiv

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.