Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a dynamically weighted mixture of per-domain adapters outperforms single-model approaches to cross-domain lay paraphrasing.

desk verdict A credible modular system for cross-domain lay paraphrasing with a wide but statistically unsupported evaluation; deserves serious review, needs a missing ablation and significance tests. read the letter →

arxiv 2505.18867 v1 pith:VTXE5SE6 submitted 2025-05-24 cs.CL cs.LG

Ming Cheng , Jiaying Gong , Hoda Eldardiry This is my paper · ORCID
classification cs.CLcs.LG
keywords layparaphrasingcross-domaintextsimplificationmixtureofLoRAsadapterweightgenerationcontrastivelearningdomainadaptationparameter-efficientfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the best way to simplify scientific text for a general audience across many fields is not one general-purpose model, but a weighted mixture of small domain-specific adapters attached to a single large language model. The method, Sci-LoRA, trains one low-rank adapter per scientific domain, then uses a contrastively trained text encoder to decide, for each input, how much weight each adapter should receive, without ever being told the input's domain. Experiments across twelve domains on five public datasets report that this mixture outperforms both large generalist models and a fully fine-tuned domain-specific model on most of ten automatic metrics and in human evaluation. The claim matters because interdisciplinary research increasingly asks readers to digest text from several technical fields at once, and a modular adapter mixture can grow as new domains are added.

What carries the argument

The load-bearing mechanism is the adapter weight generator plus the dynamic fusion step. The weight generator fine-tunes a sentence encoder with contrastive learning so that texts from the same domain are pulled together and texts from different domains are pushed apart, then represents each domain adapter by the average of the k-means cluster centroids of its training embeddings. During inference, a text's weight for each adapter is the inverse distance to that domain representation, so a biology-heavy abstract activates the biology adapter more than the business adapter. The dynamic fusion module then combines the weighted domain-specific adapter mixture with a single adapter trained on all domains, balancing specialized accuracy with general fluency.

What would settle it

Take a set of test inputs whose true domain is known and run Sci-LoRA with its learned weights, then again with weights deliberately scrambled or set uniform; if the scrambled-weight version performs equally well on the automatic metrics, the weighting mechanism is not what produces the reported gains. A second check: remove one domain's adapter and renormalize the remaining weights; a correctly functioning weight generator should make outputs for that domain's inputs degrade noticeably.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that dynamic adapter fusion, rather than a bigger model or a single fine-tune, is what unlocks cross-domain lay paraphrasing. Sci-LoRA trains twelve low-rank adapters, one per domain, on a 7-billion-parameter instruction-tuned base language model. For each input text it computes a weight per adapter from the inverse distance between the input's embedding and a cluster-centroid representation of each domain, then fuses the weighted adapters with a single general-purpose adapter trained on all domains. The authors report that this configuration beats all baselines they compare against, including large closed and open-weight models and per-domain full fine-tuning, across most of the ten automatic metrics and on human ratings for comprehensiveness, meaning preservation, and fluency.

Load-bearing premise

The whole advantage rests on the assumption that the contrastively trained embedding space measures 'which domain is this text closest to' well enough that the inverse-distance weights pick the right mixture; the paper's own visualization shows that some domains, such as biomedical corpora and news-like text, remain intermingled after contrastive training.

Editorial extensions

If this is right

  • Without domain labels at inference, the approach can be deployed where inputs arrive from unknown or mixed fields, such as science news feeds or interdisciplinary abstracts.
  • New scientific domains can be added by training one new adapter and updating the cluster centroids, avoiding a full retrain of the model.
  • The ablation results indicate that each component, separate per-domain adapters, k-means centroid representations, contrastive encoder training, and the fusion module, contributes to the final gain.
  • The reported gains hold across the twelve tested domains on five public datasets, on both automatic metrics and human ratings.
  • The authors state that scaling to hundreds of domains is limited by inference latency, unseen domains require training data, and the results are demonstrated on a single base model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the weighting is label-free, the same routing idea could be reused for other multi-style text transformations, such as simplifying text for different reading levels or adapting tone, where the 'domains' are styles rather than scientific fields.
  • The contrastive encoder's residual overlap between biomedical corpora suggests the weight generator may behave like a soft mixture for inputs that genuinely span domains; a testable extension would compare inverse-distance routing against a learned router that predicts a sparse top-k mixture.
  • If the method's advantage holds across more base models, modular adapter mixtures of this kind could become a cheaper alternative to training ever-larger generalist models for domain-heavy tasks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Sci-LoRA, a multi-LoRA framework for cross-domain lay paraphrasing. Twelve domain-specific LoRA adapters are fine-tuned on Qwen2.5-7B-Instruct over five datasets; a contrastively trained Sentence-BERT encoder together with k-means centroids produces per-input adapter weights via inverse-distance similarity (Eq. 6), and a dynamic fusion module (Eqs. 7-9) combines the weighted domain LoRAs with a generalist LoRA trained on all domains. Experiments compare Sci-LoRA with ten baselines across twelve domains using ten automatic metrics and a small human evaluation, with ablations over the adapter weight generator and fusion components.

Significance. If the empirical claims hold, the framework is a useful practical contribution: it is parameter-efficient, requires no domain labels at inference, and is evaluated broadly across twelve domains, five public datasets, and human ratings. The authors also release code, and the evaluation is grounded in public datasets and baselines rather than a proprietary benchmark. However, the central claim that the contrastive weight generator is the source of the gains is not cleanly isolated by the current ablations, and the abstract's 'significantly outperforms' is not backed by any significance testing. Both issues are addressable in revision.

major comments (3)
  1. [Sec. 3.2.2/3.2.3, Tables 5/8/9] The ablation study lacks a control condition that keeps the dynamic fusion module (Eqs. 7-9, beta = 0.5) while fixing all adapter weights to be uniform, e.g., alpha_i = 1/n for all i. The three AWG variants compared in Tables 5, 8, and 9 all compute alpha_i from Eq. 6, so every condition that includes the weight generator uses a nonuniform inverse-distance weighting. Because the central novelty is the contrastive weight generator, and because Section C reports that a single generalist LoRA outperforms Multi-LoRAs on eLife and SciTechNews while Figure 2 shows that PLOS/CELLS and eLife/SciTechNews embeddings remain intermingled after contrastive training, it is possible that the gains attributed to adaptive routing are actually due to the fusion with the generalist LoRA or to a constant, non-contextual change in the weighting scheme. Please add an 'AWGUniform' row (same contrastive encoder, same fusion, alpha_i = 1/n) across all 12 domains in Tables 5, 8, and 9.
  2. [Tables 3-4 and 8-9] No standard deviations, confidence intervals, or significance tests are reported for any of the ten automatic metrics, and many differences are small; for example, CELLS BERTScore is 83.00 for Sci-LoRA versus 82.64 for Phi-4, and SciTechNews d-BLEU is 4.61 versus 4.26 for Qwen2.5. The abstract's statement that Sci-LoRA 'significantly outperforms' state-of-the-art LLMs is therefore not supported by the evidence as presented. Report variability (multiple fine-tuning seeds or bootstrap resampling) and, if the word 'significantly' is retained, provide a statistical test at least on the headline metrics.
  3. [Eq. (6), Sec. 3.2.2] The weights alpha_i defined in Eq. 6 are inverse distances and are not normalized; as written, sum_i alpha_i can vary substantially with the absolute distances between the input embedding and the adapter representations, and can exceed one. Since these weights are used directly in the weighted LoRA merge in Eq. 7, the effective magnitude of the merged LoRA then depends on the arbitrary scale of the embedding distances. Please either state explicitly that the weights are not normalized and justify this choice, or add a normalization step (e.g., softmax over the inverse distances) and update Eq. 7 accordingly.
minor comments (4)
  1. [Eq. (3), Sec. 3.2.2] Equation (3) defines a softmax probability, not a loss; the standard InfoNCE formulation includes a negative logarithm and the text should state whether the encoder is trained by minimizing -L. As written, the optimization objective is ambiguous.
  2. [Throughout] There are numerous typos and inconsistencies: 'knolwedge' in the Introduction, 'generalizd' in the contributions, 'techical' in Sec. 3.1.2, 'vairous' in Sec. 4.1.3, 'paraphraph' in Sec. 2.1, 'SciTechNew' in Appendix B, and inconsistent capitalization of CELLS/CELLs. Please proofread the manuscript.
  3. [Sec. 4.2.3] The human evaluation reports 15 abstracts and 120 total abstracts but does not state the number of raters or whether all raters evaluated all outputs; please report the rater count and the rating procedure so that the ICC value can be interpreted.
  4. [Appendix B] The resplitting of SciTechNews from the original validation and testing sets is disclosed, but it means the SciTechNews test results are not directly comparable to results on the official test split; please state this limitation in the main text where SciTechNews results are discussed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Sci-LoRA's central claims rest on external, public benchmark evaluations and stated training/validation splits, not on definitions or self-citations.

full rationale

The paper's derivation chain is a standard supervised pipeline: per-domain LoRA adapters are trained on training splits of five public datasets; a text encoder is contrastively fine-tuned on a subset of training data; k-means representations and inverse-distance weights (Eqs. 4-6) are computed from training embeddings; fusion parameters (K, beta, sampling size) are grid-searched on the validation set (Sec. 4.1.3). The headline comparisons (Tables 3-4) are against external baselines including GPT-4o, and the test splits are the original public splits of PLOS, eLife, CELLS, SciTechNews, and VTechAGP, with SciTechNews resplit as stated. No fitted parameter is renamed as a prediction: the learned components (LoRA weights, contrastive encoder) are trained on train/validation and evaluated on held-out test data. The paper cites the authors' prior VTechAGP dataset and DSPT5 baseline, but these are public benchmark resources used for comparison, not the justification of Sci-LoRA's mechanism; the central novelty (dynamic weighting and fusion) is not defined in terms of the evaluation outcomes. The absence of a 'fusion + uniform weights' ablation is a limitation in isolating the contribution of Eq. 6, but an omitted ablation is an evidentiary gap, not circularity. The paper's own Section C observation that a single LoRA can beat Multi-LoRAs on eLife and SciTechNews actually acknowledges boundary conditions and does not make the reported gains tautological.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a set of modeling assumptions about LoRA composition, embedding-space representativeness, and domain overlap. There are no invented physical entities, but there are five hand-chosen hyperparameters that directly shape the mechanism, and the method's effectiveness depends on the contrastive encoder separating domains well enough for the weights to be informative.

free parameters (5)
  • Number of clusters K = 10
    Chosen by grid search {5,10,15,20} on the validation set; affects domain anchor representations in Eq. 5 and the resulting weights in Eq. 6.
  • Fusion weight beta = 0.5
    Set to 0.5 to balance specialized and generalized representations in Eq. 9; no tuning procedure reported.
  • Sampling size for weight generator training = 500
    Used to sample training data for the contrastive encoder and anchor computation; chosen by hand.
  • LoRA rank r = 8
    Rank of the low-rank adapters, chosen by hand; affects the capacity of each domain adapter.
  • Contrastive temperature tau = not reported
    Temperature in the contrastive loss (Eq. 3); the value is not stated, so the default is presumably used.
assumptions (5)
  • domain assumption Low-rank adaptation represents domain-specific knowledge as additive low-rank updates that can be combined linearly.
    Sci-LoRA merges LoRA weights with a weighted sum in Eqs. 6-9, assuming linear composition preserves and combines domain skills; previous MoE/LoRA literature supports this but it is an assumption.
  • domain assumption K-means cluster centroids of contrastive embeddings represent the semantic center of each domain LoRA.
    The adapter representation r in Eq. 5 is built from k-means centroids, assuming the embedding space is Euclidean and cluster coverage is representative.
  • domain assumption The test distribution matches the training domains and does not shift to unseen domains.
    The method is evaluated only on the twelve domains used to train LoRAs; the Limitations section explicitly states that unseen domains are not supported.
  • domain assumption Contrastive fine-tuning yields separable domain representations.
    The weight generator depends on domain separation; the paper shows overlap persists between CELLS/PLOS and eLife/SciTechNews after contrastive training (Figure 2, Section C).
  • standard math Standard backpropagation and optimization of the contrastive loss are correct.
    The method relies on ordinary supervised learning procedures; no formal verification is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing." pith.science (2026). https://pith.science/paper/VTXE5SE6

@misc{pith2026250518867,
  author       = {Pith},
  title        = {Pith review of: Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VTXE5SE6}},
  note         = {Machine review of arXiv:2505.18867}
}
read the original abstract

Lay paraphrasing aims to make scientific information accessible to audiences without technical backgrounds. However, most existing studies focus on a single domain, such as biomedicine. With the rise of interdisciplinary research, it is increasingly necessary to comprehend knowledge spanning multiple technical fields. To address this, we propose Sci-LoRA, a model that leverages a mixture of LoRAs fine-tuned on multiple scientific domains. In particular, Sci-LoRA dynamically generates and applies weights for each LoRA, enabling it to adjust the impact of different domains based on the input text, without requiring explicit domain labels. To balance domain-specific knowledge and generalization across various domains, Sci-LoRA integrates information at both the data and model levels. This dynamic fusion enhances the adaptability and performance across various domains. Experimental results across twelve domains on five public datasets show that Sci-LoRA significantly outperforms state-of-the-art large language models and demonstrates flexible generalization and adaptability in cross-domain lay paraphrasing.

Figures

Figures reproduced from arXiv: 2505.18867 by the authors.

Figure 1
Figure 1. The Sci-LoRA framework including the adapter weight generator and dynamic LoRA fusion. All text [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. t-SNE visualization (before contrastive learning and after contrastive learning) of text embeddings from [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Human evaluation guideline [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Spanning the Visual Analogy Space with a Weight Basis of LoRAs

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A learnable basis of LoRA adapters, mixed by an encoder at inference time, applies visual analogies to new images and beats single-adapter baselines on a custom benchmark.

Reference graph

Works this paper leans on

61 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

    arXiv 2024

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  3. [3]

    Suha S Al-Thanyyan and Aqil M Azmi. 2021. Automated text simplification: a survey. ACM Computing Surveys (CSUR), 54(2):1--36

    work page 2021

  4. [4]

    Kush Attal, Brian Ondov, and Dina Demner-Fushman. 2023. A dataset for plain language adaptation of biomedical abstracts. Scientific Data, 10(1):8

    work page 2023

  5. [5]

    Satanjeev Banerjee and Alon Lavie. 2005. METEOR : An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pages 65--72, Ann Arbor, Michigan . Association for Computational Linguistics

    work page 2005

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  7. [7]

    Buehler and Markus J

    Eric L. Buehler and Markus J. Buehler. 2024. https://doi.org/10.1063/5.0203126 X-lora: Mixture of low-rank adapter experts, a flexible framework for large language models with applications in protein mechanics and molecular design . APL Machine Learning, 2(2):026119

    doi:10.1063/5.0203126 2024

  8. [8]

    Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. 2024. A survey on mixture of experts. arXiv preprint arXiv:2407.06204

    arXiv 2024

Show all 61 references
  1. [9]

    Ronald Cardenas, Bingsheng Yao, Dakuo Wang, and Yufang Hou. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.76 ` don ' t get too technical with me ' : A discourse structure-based framework for automatic science journalism . In Proceedings of the 2023 Conference on Empirical ...

    2023 doi

  2. [10]

    Ming Cheng, Jiaying Gong, Chenhan Yuan, William A Ingram, Edward Fox, and Hoda Eldardiry. 2025. https://aclanthology.org/2025.naacl-long.311/ VT ech AGP : An academic-to-general-audience text paraphrase dataset and benchmark models . In Proceedings of the 2025 Conference of th...

    2025

  3. [11]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  4. [12]

    Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang. 2024. https://aclanthology.org/2024.lrec-main.994 Mixture-of- L o RA s: An efficient multitask tuning method for large language models . In Proceedings of the 2024 Joint International Conference on Computational Li...

    2024

  5. [13]

    Rudolf Flesch. 1979. How to write plain English . University of Canterbury. Available at http://www. mang. canterbury. ac. nz/writing\_guide/writing/flesch. shtml.[Retrieved 5 February 2016]

    1979

  6. [14]

    Marcio Fonseca and Shay Cohen. 2024. https://doi.org/10.18653/v1/2024.findings-acl.508 Can large language model summarizers adapt to diverse scientific communication goals? In Findings of the Association for Computational Linguistics: ACL 2024, pages 8599--8618, Bangkok, Thail...

    2024 doi

  7. [15]

    Polydoros Giannouris, Theodoros Myridis, Tatiana Passali, and Grigorios Tsoumakas. 2024. https://aclanthology.org/2024.determit-1.6 Plain language summarization of clinical trials . In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Contex...

    2024

  8. [16]

    Tomas Goldsack, Carolina Scarton, Matthew Shardlow, and Chenghua Lin. 2024. https://doi.org/10.18653/v1/2024.bionlp-1.10 Overview of the B io L ay S umm 2024 shared task on the lay summarization of biomedical research articles . In Proceedings of the 23rd Workshop on Biomedica...

    2024 doi

  9. [17]

    Tomas Goldsack, Zhihao Zhang, Chenghua Lin, and Carolina Scarton. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.724 Making science simple: Corpora for the lay summarisation of scientific literature . In Proceedings of the 2022 Conference on Empirical Methods in Natural Lan...

    2022 doi

  10. [18]

    Jiang Guo, Darsh Shah, and Regina Barzilay. 2018. https://doi.org/10.18653/v1/D18-1498 Multi-source domain adaptation with mixture of experts . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4694--4703, Brussels, Belgium. Assoc...

    2018 doi

  11. [19]

    Yue Guo, Wei Qiu, Gondy Leroy, Sheng Wang, and Trevor Cohen. 2024. Retrieval augmentation of large language models for lay language generation. Journal of Biomedical Informatics, 149:104580

    2024

  12. [20]

    Yue Guo, Wei Qiu, Yizhong Wang, and Trevor Cohen. 2021. Automated lay language summarization of biomedical scientific reviews. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 160--168

    2021

  13. [21]

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608

    2024 arXiv

  14. [22]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790--2799. PMLR

    2019

  15. [23]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  16. [24]

    Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. 2023. Lorahub: Efficient cross-task generalization via dynamic lora composition. arXiv preprint arXiv:2307.13269

    2023 arXiv

  17. [25]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  18. [26]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088

    2024 arXiv

  19. [27]

    Yuchen Jiang, Tianyu Liu, Shuming Ma, Dongdong Zhang, Jian Yang, Haoyang Huang, Rico Sennrich, Ryan Cotterell, Mrinmaya Sachan, and Ming Zhou. 2022. https://doi.org/10.18653/v1/2022.naacl-main.111 BlonDe : An automatic evaluation metric for document-level machine translation ....

    2022 doi

  20. [28]

    Hwanmun Kim, Kamal Raj Kanakarajan, and Malaikannan Sankarasubbu. 2024. Saama technologies at biolaysumm: Abstract based fine-tuned models with lora. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 786--792

    2024

  21. [29]

    Hao Li, Yuping Wu, Viktor Schlegel, Riza Batista-Navarro, Tharindu Madusanka, Iqra Zahid, Jiayan Zeng, Xiaochi Wang, Xinran He, Yizhi Li, and Goran Nenadic. 2024. https://doi.org/10.18653/v1/2024.findings-acl.9 Which side are you on? a multi-task dataset for end-to-end argumen...

    2024 doi

  22. [30]

    Chin-Yew Lin. 2004. ROUGE : A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74--81, Barcelona, Spain . Association for Computational Linguistics

    2004

  23. [31]

    Chin-Yew Lin and Franz Josef Och. 2004. ORANGE : A method for evaluating automatic evaluation metrics for machine translation. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics , pages 501--507, Geneva, Switzerland . COLING

    2004

  24. [32]

    Dongqi Liu, Yifan Wang, and Vera Demberg. 2023. https://doi.org/10.18653/v1/2023.acl-long.306 Incorporating distributions of discourse structure for long document abstractive summarization . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguis...

    2023 doi

  25. [33]

    Jialin Liu, Jianhua Wu, Jie Liu, and Yutai Duan. 2024 a . Learning attentional mixture of loras for language model continual learning. arXiv preprint arXiv:2409.19611

    2024 arXiv

  26. [34]

    Ran Liu, Ming Liu, Min Yu, He Zhang, Jianguo Jiang, Gang Li, and Weiqing Huang. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.574 S um S urvey: An abstractive dataset of scientific survey papers for long document summarization . In Findings of the Association for Comp...

    2024 doi

  27. [35]

    Yuhan Liu, Shangbin Feng, Xiaochuang Han, Vidhisha Balachandran, Chan Young Park, Sachin Kumar, and Yulia Tsvetkov. 2024 c . https://doi.org/10.18653/v1/2024.naacl-long.119 P ^3 S um: Preserving author ' s perspective in news summarization with diffusion language models . In P...

    2024 doi

  28. [36]

    Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu. 2024. Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models. arXiv preprint arXiv:2402.12851

    2024 arXiv

  29. [37]

    Chuancheng Lv, Lei Li, Shitou Zhang, Gang Chen, Fanchao Qi, Ningyu Zhang, and Hai-Tao Zheng. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.956 H yper L o RA : Efficient cross-task generalization via constrained low-rank adapters generation . In Findings of the Associat...

    2024 doi

  30. [38]

    Hemang Malik, Gaurav Pradeep, and Pratinav Seth. 2024. https://doi.org/10.18653/v1/2024.bionlp-1.78 HGP - NLP at B io L ay S umm: Leveraging L o RA for lay summarization of biomedical research articles using S eq2 S eq transformers . In Proceedings of the 23rd Workshop on Biom...

    2024 doi

  31. [39]

    Mohammed Muqeeth, Haokun Liu, Yufan Liu, and Colin Raffel. 2024. Learning to route among specialized experts for zero-shot generalization. arXiv preprint arXiv:2402.05859

    2024 arXiv

  32. [40]

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

    2018 arXiv

  33. [41]

    Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, and Samy Jelassi. 2024. Lora soups: Merging loras for practical skill composition tasks. arXiv preprint arXiv:2410.13025

    2024 arXiv

  34. [42]

    de Souza , Pedro G

    Ricardo Rei, Ana C Farinha, Jos \'e G.C. de Souza , Pedro G. Ramos, Andr \'e F.T. Martins, Luisa Coheur, and Alon Lavie. 2022. Searching for COMETINHO : The little metric that could. In Proceedings of the 23rd Annual Conference of the European Association for Machine Translati...

    2022

  35. [43]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.18653/v1/D19-1410 Sentence- BERT : Sentence embeddings using S iamese BERT -networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference...

    2019 doi

  36. [44]

    Hwanjun Song, Hang Su, Igor Shalyminov, Jason Cai, and Saab Mansour. 2024. https://doi.org/10.18653/v1/2024.acl-long.51 F ine S ur E : Fine-grained summarization evaluation using LLM s . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics...

    2024 doi

  37. [45]

    Chen Tang, Shun Wang, Tomas Goldsack, and Chenghua Lin. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.40 Improving biomedical abstractive summarisation with knowledge aggregation from citation papers . In Proceedings of the 2023 Conference on Empirical Methods in Natural L...

    2023 doi

  38. [46]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  39. [47]

    Xun Wu, Shaohan Huang, and Furu Wei. 2024. https://openreview.net/forum?id=uWvKBCYh4S Mixture of lo RA experts . In The Twelfth International Conference on Learning Representations

    2024

  40. [48]

    Jingwei Xu, Junyu Lai, and Yunpeng Huang. 2024. Meteora: Multiple-tasks embedded lora for large language models. arXiv preprint arXiv:2405.13053

    2024 arXiv

  41. [49]

    Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch . 2016. https://doi.org/10.1162/tacl_a_00107 Optimizing statistical machine translation for text simplification . Transactions of the Association for Computational Linguistics, 4:401--415

    2016 doi

  42. [50]

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2024. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36

    2024

  43. [51]

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning

    2024

  44. [52]

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068

    2022 arXiv

  45. [53]

    Weinberger, and Yoav Artzi

    Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore : Evaluating text generation with BERT . In International Conference on Learning Representations

    2020

  46. [54]

    Yuxuan Zhang and Ruizhe Li. 2024. Dlp-lora: Efficient task-specific lora fusion with a dynamic, lightweight plugin for large language models. arXiv preprint arXiv:2410.01497

    2024 arXiv

  47. [55]

    Lulu Zhao, Weihao Zeng, Xiaofeng Shi, and Hua Zhou. 2024 a . Mosld: An extremely parameter-efficient mixture-of-shared loras for multi-task learning. arXiv preprint arXiv:2412.08946

    2024 arXiv

  48. [56]

    Ziyu Zhao, Leilei Gan, Guoyin Wang, Wangchunshu Zhou, Hongxia Yang, Kun Kuang, and Fei Wu. 2024 b . https://doi.org/10.18653/v1/2024.findings-acl.263 L ora R etriever: Input-aware L o RA retrieval and composition for mixed tasks in the wild . In Findings of the Association for...

    2024 doi

  49. [57]

    Ziyu Zhao, Tao Shen, Didi Zhu, Zexi Li, Jing Su, Xuwu Wang, Kun Kuang, and Fei Wu. 2024 c . Merging loras like playing lego: Pushing the modularity of lora to extremes through rank-wise clustering. arXiv preprint arXiv:2409.16167

    2024 arXiv

  50. [58]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. http://arxiv.org/abs/2403.13372 Llamafactory: Unified efficient fine-tuning of 100+ language models . In Proceedings of the 62nd Annual Meeting of the Association for Compu...

    2024 arXiv

  51. [59]

    Jianing Zhou and Suma Bhat. 2021. Paraphrase generation: A survey of the state of the art. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 5075--5086

    2021

  52. [60]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  53. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.