Pith. sign in

REVIEW 4 major objections 4 minor 40 references

A Multi-Encoder Frozen-Decoder Approach for Fine-Tuning Large Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Freezing a decoder during fine-tuning is not merely a cost saver: it preserves generation quality, prevents multilingual forgetting, and, with a larger decoder, improves structured and QA performance.

desk verdict Useful breadth on frozen-decoder fine-tuning, but the 2B 'freezing recovers structured/QA' claim is missing the trainable-2B control that would support it. read the letter →

arxiv 2501.07818 v1 pith:KOHOH3BB submitted 2025-01-14 cs.CL cs.AIcs.LG

Kaustubh D. Dhole This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords frozendecoderparameter-efficientfine-tuningcatastrophicforgettingmultilingualnaturallanguagegenerationsemanticparsingquestionansweringAlexaTM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that in fine-tuning a pretrained encoder-decoder language model, you can freeze the decoder and train only the encoder without paying a performance penalty, and in several settings you actually gain. The experiments cover seven task types on AlexaTM, a multilingual 511M-parameter model, plus a larger 2B variant with a frozen decoder. The results show that frozen decoders match or improve natural-language generation tasks, and they sharply reduce catastrophic forgetting when a model fine-tuned on English is evaluated on other languages. For structured and QA outputs, the same-size frozen decoder loses ground, but the larger frozen decoder recovers or exceeds the fully trainable baseline. The paper therefore proposes freezing as a viable parameter-efficient fine-tuning strategy, especially when one shared decoder serves multiple task-specific encoders.

What carries the argument

The central object is the frozen decoder: a pretrained decoder whose weights stay fixed throughout fine-tuning, so gradients never touch it and only encoder parameters are updated. The paper pairs this with a size contrast, testing a 511M-parameter AlexaTM with a 12-layer encoder and 12-layer decoder against a 2B-parameter version with the decoder also frozen, to see whether extra decoder capacity can compensate for the loss of trainability. The argument is carried by three stacked configurations on each task, a trainable decoder, a same-size frozen decoder, and a larger frozen decoder, evaluated on single-task and mixed-task fine-tuning with beam search. The title's implied architecture, a set of task-specific encoders feeding a single shared frozen decoder, is the deployment consequence the paper motivates but does not directly build.

What would settle it

Train the same 2B AlexaTM with a trainable decoder on SQuAD and MTOP and compare exact-match scores; if the trainable 2B model reaches or exceeds the frozen-decoder scores, the claim that freezing rather than model scale drives the structured-task gains is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that the decoder of a pretrained encoder-decoder model can be frozen during task fine-tuning without losing downstream performance, and that freezing is not just a way to save computation. On natural-language generation tasks, the frozen decoder matches or slightly improves the fully trainable baseline: WebNLG BLEU rises from 29.76 to 32.01, and the 2B frozen decoder reaches 35.14, while XSUM ROUGE-1 falls from 0.39 to 0.37 and recovers most of the gap with the larger decoder. On multilingual intent classification (MASSIVE), freezing lifts non-English accuracy by 8 to 10 percentage points over the trainable model, evidence that keeping the decoder fixed prevents catastrophic forgetting of pretrained multilingual knowledge. On the structured and QA tasks where the same-size frozen decoder loses ground, such as MTOP exact match falling from 66% to 52%, the larger 2B frozen decoder overcomes the loss, scoring 68% on MTOP and 74.15% exact match on SQuAD versus 72.33% with the fully trainable model. The paper concludes that frozen decoders are a viable parameter-efficient strategy across task types, particularly when tasks with similar output formats share one decoder.

Load-bearing premise

The load-bearing premise is that the 2B model's gains come from its larger frozen decoder, because the paper never trains a 2B model with a trainable decoder; if the larger encoder or different pretraining is actually responsible, the claim that a bigger frozen decoder rescues structured tasks collapses.

Editorial extensions

If this is right

  • Fine-tuning only the encoder is a viable default for tasks with natural-language targets, since WebNLG BLEU rises and CommonGen scores stay flat when the decoder is frozen.
  • Freezing the decoder is a concrete remedy for catastrophic forgetting in multilingual assistants, with non-English MASSIVE intent accuracy climbing 8 to 10 points over the trainable baseline while English also improves.
  • Structured and QA tasks can keep pace with fully trainable models if the frozen decoder is larger, as MTOP and SQuAD both post their best exact-match scores with the 2B frozen decoder.
  • Mixed-task fine-tuning still suffers cross-task interference, so the practical win is to group tasks with similar output formats and give each group a frozen decoder.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extending the paper, the shared-decoder deployment suggested by the title can be tested directly by training several small task-specific encoders against one frozen decoder and measuring per-task accuracy and throughput.
  • The paper's results imply a testable predictor: the benefit of decoder freezing should track how similar a task's target sequences are to the decoder's pretraining text, measurable by the frozen decoder's perplexity on target strings before fine-tuning.
  • The MASSIVE gains suggest that freezing acts as a regularizer on the shared multilingual representation, which implies that freezing other modules such as the embedding layer could yield similar catastrophic-forgetting relief, although the paper does not test this.
  • The mixed-task numbers, with MTOP exact match falling to 28%, indicate that task grouping rather than freezing alone will determine multi-task success; a natural follow-up is to cluster tasks by output format and freeze one decoder per cluster.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper studies whether freezing the decoder of an encoder-decoder language model during fine-tuning is an effective parameter-efficient strategy. Using AlexaTM (511M) and a larger AlexaTM 2B model, it compares trainable decoder, frozen decoder at the same scale, and frozen decoder with the larger model across single-task and mixed-task settings on NLG (XSUM, WebNLG, CommonGen), semantic parsing/QA (MTOP, SQuAD), multilingual intent classification (MASSIVE), and XNLI. The authors report that freezing the decoder preserves or improves performance on natural-language-output tasks, mitigates catastrophic forgetting in multilingual tasks, and that a larger frozen decoder can retain or improve structured and QA performance. The paper does not provide code, checkpoints, repeated-run statistics, or a trainable 2B baseline.

Significance. If fully supported, the central claim would be practically useful: frozen decoders could reduce fine-tuning and deployment cost while preserving generation quality and preventing multilingual forgetting. The task coverage is broad and the inclusion of both single-task and mixed-task conditions is a strength. The strongest evidence is the same-scale MASSIVE comparison, where the frozen 511M decoder clearly outperforms the trainable 511M decoder across nearly all languages. The weakest part is the larger-decoder claim: because no trainable AlexaTM 2B condition is run, the observed gains on MTOP, SQuAD, and XNLI cannot be attributed to decoder capacity. The paper is an empirical report without reproducibility artifacts or error analysis, so its conclusions rest entirely on the presented tables.

major comments (4)
  1. [§3.1, Tables 3 and 5] The abstract and conclusion claim that 'pairing frozen decoders with a larger model can effectively maintain or even enhance performance in structured and QA tasks,' but this is not established. The only larger-model condition is introduced in §3.1 as 'We keep the decoder here frozen too,' with no trainable AlexaTM 2B baseline. Differences between AlexaTM-2B (frozen) and AlexaTM (trainable) on MTOP, SQuAD, and XNLI confound decoder size with encoder size, parameter count, and possibly pretraining scale/recipe. The conclusion that decoder capacity drives the improvement requires a trainable 2B baseline, or a reformulation of the claim to refer to 'a larger frozen model' rather than 'a larger frozen decoder.'
  2. [§3.3 and all result tables] No variance information or repeated-seed experiments are reported, and checkpoint selection is inconsistent: §3.3 says checkpoints were 'saved either based on the perplexity or exact match scores over the validation set.' It is not stated which criterion was used for each task or run. Without error bars or multiple runs, differences such as the 2-point exact-match gains on MTOP and SQuAD in Table 3 and small ROUGE differences in Table 2 are not interpretable as reliable effects.
  3. [§4.1 and Table 2] The prose misreports the table for WebNLG and XSUM. It says 'When fine-tuned on WebNLG, freezing results in a 2% drop in performance,' but Table 2 shows AlexaTM (frozen) with BLEU 32.01 compared with AlexaTM 29.76, i.e., improvement, and later text says 'freezing improves the performance in the single task setting.' For XSUM, the claim that performance 'is improved with the larger frozen decoder' is also not supported: AlexaTM-2B (frozen) ties or slightly trails the trainable AlexaTM on ROUGE-1, ROUGE-L, and BLEU. These contradictions make the analysis unreliable.
  4. [§4.1 and Table 5] The XNLI discussion is unclear and partly contradicted by Table 5. The column headings 'Accuracy (3)' and 'Accuracy (20)' are undefined; if these refer to decoding beam sizes, that should be stated explicitly. The text says 'Freezing the parameters improves performance,' but in the mixed setting AlexaTM-mix (84.29) outperforms AlexaTM-mix (frozen) (83.91), so the claim holds only for the single-task comparison. The additional statement that increasing beams from 3 to 5, 10, and 20 decreases the gap is not supported by any table, since only 3 and 20 are reported.
minor comments (4)
  1. [§3.3] The learning-rate schedule is stated as 'a learning rate of 10 6 and a linear decay to lr = 5x106 over 100k updates,' which appears to contain missing superscripts or exponents; please correct the notation so the schedule is unambiguous.
  2. [§4.1] The paper repeatedly writes 'ROGUE' where it means 'ROUGE' (e.g., '3 scores are computed – the ROGUE scores'), and the metric names should be normalized throughout.
  3. [Table 3 and §4.1] For MTOP, the text describes 'almost a 14% drop' and later 'a 2% increase' from the larger frozen decoder, but the exact-match numbers (0.66, 0.52, 0.68) correspond to relative drops/gains of about 21% and 31% respectively from the frozen baseline; please clarify whether percentages are absolute or relative.
  4. [§6 Future Work] The sentence 'Performances on mix tasks can be highly variable being dictated by the dataset proportion and the choice of other datasets' is grammatically incomplete; a connector such as 'and can be' would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical benchmark comparisons against external datasets, with no fitted inputs, equations, or load-bearing self-citation chain.

full rationale

The paper contains no mathematical derivation whose output could reduce to its input. All central claims are empirical comparisons of frozen versus trainable decoders on public benchmarks (XSUM, WebNLG, CommonGen, MTOP, SQuAD, MASSIVE, XNLI), evaluated with standard external metrics (ROUGE, BLEU, NIST, exact match). The frozen-decoder results are measured, not predicted from a fitted parameter, so there is no self-definitional or fitted-input-called-prediction circularity. The AlexaTM-2B frozen condition is a separate experimental condition rather than a quantity derived from the smaller-model results. Its interpretation may be confounded by the absence of a trainable 2B baseline, but a missing control is an experimental-design concern, not circularity. Self-citations in the references (e.g., GEM metrics, BIG-bench, NL-Augmenter) are used for evaluation tooling or context and are not load-bearing for the freezing conclusion. The paper is self-contained in the sense that its empirical claims stand or fall on the reported comparisons against external test sets.

Assumptions & free parameters 4 free parameters · 2 assumptions · 0 invented entities

No mathematical derivation is attempted. The ledger captures hand-chosen training and evaluation hyperparameters plus the unstated attribution assumptions behind the model-size comparison.

free parameters (4)
  • learning rate schedule = 1e-6 decaying to 5e-6 over 100k updates (paper text has typo)
    Hand-chosen optimizer setting that affects all results.
  • batch size = 128
    Hand-chosen batch size that affects convergence and final scores.
  • beam size = 3, with additional runs at 5, 10, 20 for one analysis
    Generation hyperparameter; the paper states the gap between frozen and trainable decoders shrinks with more beams, so reported rankings depend on this choice.
  • dataset sampling proportions = Square-root proportions shown in Table 1, e.g. XNLI 31.15%, XSUM 22.45%
    Multi-task mixture proportions are chosen by hand and directly affect mixed-task results.
assumptions (2)
  • domain assumption Freezing a well-pretrained decoder does not lose much representational capacity for tasks whose outputs resemble pretraining text.
    Used in Section 4.1 to explain WebNLG and XSUM results and in the conclusion to predict when freezing works.
  • ad hoc to paper Performance differences between AlexaTM and AlexaTM-2B are attributable to decoder size rather than encoder size or pretraining scale.
    Section 3.1 adds the 2B model only in frozen form and no trainable 2B condition is run. The central fix for structured and QA tasks, a larger frozen decoder, depends on this attribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Multi-Encoder Frozen-Decoder Approach for Fine-Tuning Large Language Models." pith.science (2026). https://pith.science/paper/KOHOH3BB

@misc{pith2026250107818,
  author       = {Pith},
  title        = {Pith review of: A Multi-Encoder Frozen-Decoder Approach for Fine-Tuning Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KOHOH3BB}},
  note         = {Machine review of arXiv:2501.07818}
}
read the original abstract

Among parameter-efficient fine-tuning methods, freezing has emerged as a popular strategy for speeding up training, reducing catastrophic forgetting, and improving downstream performance. We investigate the impact of freezing the decoder in a multi-task setup comprising diverse natural language tasks, aiming to reduce deployment overhead and enhance portability to novel tasks. Our experiments, conducted by fine-tuning both individual and multi-task setups on the AlexaTM model, reveal that freezing decoders is highly effective for tasks with natural language outputs and mitigates catastrophic forgetting in multilingual tasks. However, we find that pairing frozen decoders with a larger model can effectively maintain or even enhance performance in structured and QA tasks, making it a viable strategy for a broader range of task types.

Figures

Figures reproduced from arXiv: 2501.07818 by the authors.

Figure 1
Figure 1. Freezing the decoder helps train faster, re￾duces deployment overhead, improves portability, and possibly increases task performance. been a popular strategy adopted to train faster, re￾duce catastrophic forgetting, and increase down￾stream performance. We seek to investigate the benefits and drawbacks of specifically freezing the parameters of the decoder while fine-tuning indi￾vidual task-specific encoders ( [PIT… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 9 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Emanuele Bastianelli, Andrea Vanzo, Pawel Swietojanski, and Verena Rieser. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.588 SLURP : A spoken language understanding resource package . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7252--7262, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.588 2020

  4. [4]

    Andrew Brock, Theodore Lim, J. M. Ritchie, and Nick Weston. 2017. http://arxiv.org/abs/1706.04983 Freezeout: Accelerate training by progressively freezing layers

    arXiv 2017

  5. [5]

    Ronan Collobert and Jason Weston. 2008. https://doi.org/10.1145/1390156.1390177 A unified architecture for natural language processing: Deep neural networks with multitask learning . In Proceedings of the 25th International Conference on Machine Learning, ICML '08, page 160–167, New York, NY, USA. Association for Computing Machinery

    arXiv 2008

  6. [6]

    Bowman, Holger Schwenk, and Veselin Stoyanov

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2018

  7. [7]

    Asa Cooper Stickland, Xian Li, and Marjan Ghazvininejad. 2021. https://doi.org/10.18653/v1/2021.eacl-main.301 Recipes for adapting pre-trained monolingual and multilingual models to machine translation . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3440--3453, Online. As...

    work page doi:10.18653/v1/2021.eacl-main.301 2021

  8. [8]

    Kaustubh Dhole. 2023. Large language models as sociotechnical systems. In Proceedings of the Big Picture Workshop, pages 66--79

    work page 2023

Show all 40 references
  1. [9]

    Kaustubh Dhole, Varun Gangal, Sebastian Gehrmann, Aadesh Gupta, Zhenhao Li, Saad Mahamood, Abinaya Mahadiran, Simon Mille, Ashish Shrivastava, Samson Tan, Tongshang Wu, Jascha Sohl-Dickstein, Jinho Choi, Eduard Hovy, Ond r ej Du s ek, Sebastian Ruder, Sajant Anand, Nagender An...

    2023

  2. [10]

    George Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference on Human Language Technology Research, pages 138--145

    2002

  3. [11]

    Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, and Chelsea Finn. 2021. Efficiently identifying task groupings for multi-task learning. Advances in Neural Information Processing Systems, 34:27503--27516

    2021

  4. [12]

    Jack FitzGerald, Shankar Ananthakrishnan, Konstantine Arkoudas, Davide Bernardi, Abhishek Bhagia, Claudio Delli Bovi, Jin Cao, RAKESH CHADA, Amit Chauhan, Luoxin Chen, Anurag Dwarakanath, Satyam Dwivedi, Turan Gojayev, Karthik Gopalakrishnan, Thomas Gueudre, Dilek Hakkani-Tur,...

    2022

  5. [13]

    Jack FitzGerald, Christopher Hench, Charith Peris, Scott Mackie, Kay Rottmann, Ana Sanchez, Aaron Nash, Liam Urbach, Vishesh Kakarala, Richa Singh, Swetha Ranganath, Laurie Crist, Misha Britan, Wouter Leeuwis, Gokhan Tur, and Prem Natarajan. 2022 b . http://arxiv.org/abs/2204....

    2022 arXiv

  6. [14]

    Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, Pawan Sasanka Ammanamanchi, Anuoluwapo Aremu, Antoine Bosselut, Khyathi Raghavi Chandu, Miruna-Adriana Clinciu, Dipanjan Das, Kaustubh Dhole, Wanyu Du, Esin Durmus, Ond r ej Du s ek, Chris Chinenye Emezue, Varun Gangal, Cri...

    2021

  7. [15]

    Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina McMillan-Major, Anna Shvets, Ashish Upadhyay, Bingsheng Yao, et al. 2022. Gemv2: Multilingual nlg benchmarking in a single line of code. arXiv preprint arXiv:2206.11249

    2022 arXiv

  8. [16]

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. https://openreview.net/forum?id=lIsCS8b6zj Parameter-efficient fine-tuning for large models: A comprehensive survey . Transactions on Machine Learning Research

    2024

  9. [17]

    Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, and Luo Si. 2021. https://doi.org/10.18653/v1/2021.acl-long.172 On the effectiveness of adapter-based tuning for pretrained language model adaptation . In Proceedings of the 59th An...

    2021 doi

  10. [18]

    Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R

    Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. 2012. https://doi.org/10.48550/ARXIV.1207.0580 Improving neural networks by preventing co-adaptation of feature detectors

    2012 doi arXiv

  11. [19]

    Weinberger

    Gao Huang, Yu Sun, Zhuang Liu, Daniel Sedra, and Kilian Q. Weinberger. 2016. Deep networks with stochastic depth. In Computer Vision -- ECCV 2016, pages 646--661, Cham. Springer International Publishing

    2016

  12. [20]

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171--4186

    2019

  13. [21]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2014. http://arxiv.org/abs/1412.6980 Adam: A method for stochastic optimization

    2014 arXiv

  14. [22]

    Jaejun Lee, Raphael Tang, and Jimmy Lin. 2019. http://arxiv.org/abs/1911.03090 What would elsa do? freezing layers during transformer fine-tuning

    2019 arXiv

  15. [23]

    Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. 2021. https://doi.org/10.18653/v1/2021.eacl-main.257 MTOP : A comprehensive multilingual task-oriented semantic parsing benchmark . In Proceedings of the 16th Conference of the European Chapt...

    2021 doi

  16. [24]

    Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. 2020. https://doi.org/10.18653/v1/2020.findings-emnlp.165 C ommon G en: A constrained text generation challenge for generative commonsense reasoning . In Findings of the Ass...

    2020 doi

  17. [25]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  18. [26]

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. https://doi.org/10.1162/tacl_a_00343 Multilingual denoising pre-training for neural machine translation . Transactions of the Association for Computational...

    2020 doi

  19. [27]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. http://arxiv.org/abs/1907.11692 Roberta: A robustly optimized bert pretraining approach

    2019 arXiv

  20. [28]

    Simon Mille, Kaustubh Dhole, Saad Mahamood, Laura Perez-Beltrachini, Varun Gangal, Mihir Kale, Emiel van Miltenburg, and Sebastian Gehrmann. 2021. Automatic construction of evaluation suites for natural language generation datasets. In Thirty-fifth Conference on Neural Informa...

    2021

  21. [29]

    Cohen, and Mirella Lapata

    Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. ArXiv, abs/1808.08745

    2018 arXiv

  22. [30]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  23. [31]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC '20. IEEE Press

    2020

  24. [32]

    Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. https://doi.org/10.18653/v1/P18-2124 Know what you don ' t know: Unanswerable questions for SQ u AD . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages ...

    2018 doi

  25. [33]

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. https://doi.org/10.1145/3394486.3406703 Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters . In Proceedings of the 26th ACM SIGKDD International Conferenc...

    2020

  26. [34]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  27. [35]

    Saleh Soltan, Shankar Ananthakrishnan, Jack FitzGerald, Rahul Gupta, Wael Hamza, Haidar Khan, Charith Peris, Stephen Rawls, Andy Rosenbaum, Anna Rumshisky, Chandana Satya Prakash, Mukund Sridhar, Fabian Triefenbach, Apurv Verma, Gokhan Tur, and Prem Natarajan. 2022. https://ww...

    2022

  28. [36]

    Brown, Adam Santoro, Aditya Gupta, Adri \`a Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adri \`a Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, ...

    2023

  29. [37]

    McCarthy, Kevin Duh, Rebecca Marvin, Paul McNamee, Jeremy Gwinnup, Tim Anderson, and Philipp Koehn

    Brian Thompson, Huda Khayrallah, Antonios Anastasopoulos, Arya D. McCarthy, Kevin Duh, Rebecca Marvin, Paul McNamee, Jeremy Gwinnup, Tim Anderson, and Philipp Koehn. 2018. https://doi.org/10.18653/v1/W18-6313 Freezing subnetworks to analyze domain adaptation in neural machine ...

    2018 doi

  30. [38]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  31. [39]

    Giulio Zhou and Gerasimos Lampouras. 2020. https://aclanthology.org/2020.webnlg-1.22 W eb NLG challenge 2020: Language agnostic delexicalisation for multilingual RDF -to-text generation . In Proceedings of the 3rd International Workshop on Natural Language Generation from the ...

    2020

  32. [40]

    Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. https://doi.org/10.18653/v1/D16-1163 Transfer learning for low-resource neural machine translation . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1568--1575, Aus...

    2016 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.