Pith. sign in

REVIEW 4 major objections 6 minor 50 references

Flipping Knowledge Distillation: Leveraging Small Models' Expertise to Enhance LLMs in Text Matching

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reinterpreting a decoder-only LLM as an encoder-decoder via LoRA and aligning its pairwise similarity scores with a small domain-specific teacher lets it beat supervised fine-tuning and classic distillation on text matching.

desk verdict A genuinely flipped-KD method for text matching with consistent offline gains and a live deployment, but the paper never checks whether the LoRA projection actually reproduces the teacher's similarity geometry, leaving the central mechanism inferred rather than shown. read the letter →

arxiv 2507.05617 v1 pith:E4HBRODJ submitted 2025-07-08 cs.CL

Mingzhe Li , Jing Xiang , Qishen Zhang , Kaiyang Wan , Xiuying Chen This is my paper
classification cs.CL
keywords flippedknowledgedistillationtextmatchingLoRAencoder-decoderreinterpretationmargin-awarecontrastivelearningrepresentationsimilarityalignmentsmall-to-largemodeldomain-specificembeddingsFAQretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the usual direction of knowledge distillation can be reversed for text matching: a small, domain-specialized embedding model is a better teacher of pairwise similarity than a general-purpose LLM. The authors reinterpret a decoder-only LLM's LoRA adapters as an encoder-decoder pair, so the LLM emits per-text vectors in the same dimension as the small teacher. Training aligns the student's pairwise cosine scores with the teacher's via a squared-distance loss and a margin-aware contrastive loss, with a threshold filter discarding teacher outputs that contradict labels. On financial, medical, and real-world FAQ retrieval data, this flipped distillation improves accuracy, F1, and AUC over the base LLM, supervised fine-tuning, parameter-efficient fine-tuning, and standard distillation, and an online A/B test reports higher user satisfaction.

What carries the argument

The load-bearing mechanism is the reinterpretation of LoRA as an encoder-decoder: the low-rank matrix A compresses a text's token embeddings into a vector whose dimension is set equal to the teacher SLM's hidden dimension, and the matrix B maps that compressed representation back to the output space. On top of this, a threshold-aware matching matrix filters teacher cosine similarities that contradict the ground-truth label, the distillation loss matches student and teacher cosine similarities on the surviving pairs, and a margin-aware contrastive loss compares the angular distances of positive and negative pairs, using the teacher's angular distance as an adaptive margin. Together these pieces transfer a pairwise-similarity structure rather than raw embeddings.

What would settle it

A decisive test would be to shuffle the teacher similarity scores within training batches while keeping ground-truth labels and the supervised loss unchanged; if the text-matching gains persist, the pairwise-similarity distillation signal is not what drives the improvement.

Watch

Extended reading notes

Core claim

The central claim is that an LLM can absorb a small model's text-matching expertise by aligning the geometry of pairwise similarity rather than by copying vectors or labels. LoRA's down-projection matrix is reinterpreted as an encoder that compresses each text into a representation whose dimension equals the teacher's hidden dimension; the up-projection decodes the compressed representation into the normal LLM output space. The student's cosine similarities are aligned to the teacher's through a filtered squared-distance loss, and a margin-aware contrastive loss on angular distances sharpens both the separation between positive and negative pairs and the relative distances within each group. The paper reports that this recipe lets a 0.5B and a 10B decoder-only LLM outperform their unfinetuned and LoRA-tuned versions, as well as supervised fine-tuning, classic distillation from a larger teacher, and baseline flipped distillation, on all three benchmarks.

Load-bearing premise

The core assumption is that the compressed vectors the LLM produces for each text live in the same similarity space as the teacher's embeddings, so their cosine scores can be aligned, and the paper does not verify this alignment independently of final task performance.

Editorial extensions

If this is right

  • Domain-specific small models can be reused as teachers for much larger LLMs, reducing the need for large amounts of domain instruction data in text-matching tasks.
  • Because only LoRA parameters are trained (22% of parameters for the 0.5B student, 7.54% for the 10B student), the flipped distillation direction is compatible with parameter-efficient fine-tuning and shorter training time.
  • The margin-aware contrastive loss is load-bearing: ablations show that removing either the similarity-distillation loss or the margin-aware contrastive loss lowers F1 and AUC on all datasets, and removing the noise filter also hurts.
  • The transfer works even when the teacher underperforms the student on the same dataset, suggesting that the teacher's pairwise-similarity knowledge, not its absolute quality, is what improves the LLM.
  • Deployed in an online FAQ retrieval system, the flipped-distillation model improved user satisfaction across insurance, loan, payment, and subscription scenes relative to the production baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: The same LoRA-as-encoder reinterpretation could be tested for other pairwise scoring tasks, such as dense retrieval, entailment, or citation matching, where small cross-encoders already outperform large decoders.
  • Extension: If the projected student space faithfully mirrors the teacher's, frozen LLMs could be upgraded to new domain specialists by re-running only the distillation step whenever a better small embedding model is released, without architectural changes.
  • Extension: A direct test of the mechanism would be to randomize or permute the teacher similarity scores during training; if gains persist, the pairwise alignment is not what carries the improvement.
  • Extension: Using an ensemble of small teachers and weighting by their agreement could make the distillation signal more robust to individual teacher blind spots; the paper does not explore this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a 'flipped' knowledge-distillation method for text matching in which a decoder-only LLM is fine-tuned to learn from a smaller, domain-specific encoder model. The authors reinterpret LoRA as an encoder-decoder mechanism: the A-projection compresses the concatenated LLM input, mean-pooled vectors provide student text representations, and the B-projection decodes back to the output space. The student's pairwise cosine similarities are aligned with the teacher's via a threshold-filtered distillation loss and a margin-aware contrastive loss. Experiments are reported on ATEC, NFCorpus, and an in-house ByteDance FAQ dataset, with offline gains over base LLMs, LoRA-only fine-tuning, and two baseline flipped-distillation methods, plus positive online A/B results in four ByteDance scenarios.

Significance. If the central claims hold, the paper is a useful contribution: it demonstrates a parameter-efficient way to inject representation-level expertise from a smaller specialist model into a larger LLM for text matching, and it backs the approach with ablations and a deployed online system. The direction is genuinely 'flipped' relative to standard LLM-to-SLM distillation, and the margin-aware contrastive objective is a reasonable mechanism for transferring relative similarity structure. The reported gains are directionally consistent across datasets and model scales, and the ablation study supports the contributions of both loss terms and the noise filter. However, the empirical support has important gaps: no variance or confidence intervals are reported, the comparison with classic distillation baselines is confounded by different backbone models and training data, and the assumed mechanism linking the LoRA projection to the teacher's representation space is never directly validated.

major comments (4)
  1. [Section 3.3, Eq. (2) and Appendix B] The method's mechanism assumes that the mean-pooled vectors r_i^l and r_j^l obtained from the LoRA A-projection live in a space whose pairwise cosine structure can track the teacher SLM's r_i^s and r_j^s. This is load-bearing because L_dist and L_MCL align α^l with α^s, but the paper never reports a direct check of this alignment: e.g., the correlation between α^l and α^s on held-out pairs, the agreement in retrieval ordering, or whether the projected vectors recover the teacher's pair geometry. As reported, the end-task gains could come from a useful task-specific projection acting as a regularizer while the distillation losses align to an uninformative signal. Please add a direct alignment analysis, or substantially soften the claim that the LLM learns the SLM's representational expertise.
  2. [Section 3.4, Eq. (6)] The MCL loss as printed is mathematically ill-defined for filtered positive pairs. If φ^s_{i,j}=0, the numerator is zero, but the denominator can still be positive because of the negative-pair sum, so the term becomes -log(0). The formula also appears to have a typo in the denominator: the angular margin in the negative term should presumably use θ^s_{i,j'} rather than θ^s_{i,j}, and the summation symbol is garbled. Please correct the formula and clarify how the φ masks are meant to zero out filtered examples without producing infinities.
  3. [Tables 1 and 2] The central empirical claims rest on Table 1 and Table 2, but the reported numbers are averages over five checkpoints with no standard deviation or confidence intervals, and the significance annotation only indicates improvement over the backbone model, not over the other compared methods. The online A/B results in Table 2 report relative improvements without confidence intervals, sample sizes, or significance tests. Because the paper's headline contribution is empirical, please report variance and proper pairwise significance tests for the method comparisons, and provide intervals for the online results.
  4. [Section 4.2 and Table 1] The comparison to 'classic distillation' is confounded. Upaya and PMC-Llama use different backbone LLMs (Llama-7b and Llama-13b) and different teacher/training data than the GLM-10b and Qwen-0.5b students used for the proposed method, so the table does not establish that the proposed flip-distillation outperforms classic distillation within a matched setting. To support the claimed advantage over classic distillation, run classic distillation with the same student backbones and same data, or restrict the claim to an apples-to-apples comparison.
minor comments (6)
  1. [Section 3.3 and Section 3.4] The symbol θ is used both for the noise-filtering threshold in Eq. (3) and for the angular distance in Eq. (5). This notation clash makes the equations harder to parse; please use a different symbol, e.g., τ for the threshold.
  2. [Appendix B] The implementation details state that 'the rank and alpha of LoRA is determined based on the hidden dimension of the teacher model,' but the body text says the rank matches the teacher hidden dimension while alpha is not specified. Please clarify the actual values of rank and alpha used.
  3. [Figure 5 caption] The caption contains the typo 'Qweb-0.5b-flip'; it should read 'Qwen-0.5b-flip'.
  4. [Section 5.2 and Implementation Details] The margin coefficient m_c is tuned on the ByteDance validation set and then used for ATEC and NFCorpus, while θ=0.5 is chosen by convention. Please report sensitivity for the other datasets or justify why tuning on one dataset transfers.
  5. [References] Several references are incomplete: the Hu et al. LoRA citation and the Devlin (2018) BERT citation lack full publication details, and some entries are missing venue/year information.
  6. [General] No code or data are released, and the third dataset is proprietary. Given the claims of practical deployment, adding a reproducibility statement or indicating where model parameters can be obtained would strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are empirical outcomes of explicitly defined distillation losses, not consequences of a self-referential definition.

full rationale

I traced the claimed derivation chain. The method defines student representations r^l = x_l A^T and teacher representations r^s from an SLM, computes cosine similarities via Eq. (2), and trains with L_sup, L_dist = (1/n)Σ φ^s_i,j (α^s_i,j − α^l_i,j)^2, and the margin-aware contrastive loss. These are explicit objective functions that define what it means for the student to align with the teacher; they do not by construction entail the accuracy/F1/AUC improvements reported in Tables 1 and 2, since the student also keeps its own LLM knowledge and is supervised by labels. The LoRA encoder-decoder reinterpretation is presented as an analogy inspired by the external LoRA formulation, not as a theorem, and no uniqueness result or load-bearing self-citation is invoked. Self-citations (Chen et al. 2024a, Li et al. 2024, Song et al. 2025) are contextual related-work references and do not justify the method's premises. The choices of mc and θ are hyperparameter selections; choosing them on a validation split of the target benchmark is standard model selection, not a fitted parameter renamed as a prediction. The unvalidated assumption that the LoRA projection preserves the teacher's pair-similarity geometry is an empirical validation gap, not a reduction by construction. Therefore no circular step is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on established mathematical tools and two domain assumptions about teacher quality and projection fidelity. The fitted numbers are mc, theta, loss weights, and LoRA rank; none of these constitutes a circular derivation, but they are hand-tuned and affect the reported gains.

free parameters (4)
  • margin coefficient mc = 0.06
    Tuned on the ByteDance validation set in Figure 4 and Section 5.2, where F1 peaks at mc=0.06, then reused for all reported experiments.
  • noise filtering threshold theta = 0.5
    Set as the balanced midpoint for binary classification in Section 4.3 and Appendix B; no sensitivity analysis is reported.
  • distillation loss weights = 0.1 for Ldist and 0.1 for LMCL
    Chosen to scale components to the same magnitude in Appendix B; no search or ablation is reported.
  • LoRA rank = teacher model hidden dimension
    Set equal to the teacher's representation size so student and teacher vectors can be aligned, as described in Section 4.3 and Appendix B; this choice is load-bearing for the alignment claim.
assumptions (5)
  • standard math Cosine similarity and arccosine angular distance capture matching relationships.
    Used in Eqs. (2) and (5) to define teacher and student similarities; assumes these geometric quantities are meaningful training targets.
  • domain assumption Fine-tuned SLMs provide better matching knowledge than LLM direct prediction.
    This is the premise of the flipped paradigm in Section 1; if false, aligning to the SLM would degrade rather than improve the LLM.
  • domain assumption The LoRA A-projection produces a representation space comparable to the teacher embedding space.
    Sections 3.2 and 3.3 set the LoRA rank to the teacher's hidden dimension and compare cosine similarities; this is the load-bearing architectural assumption.
  • domain assumption Threshold filtering leaves a reliable distillation set.
    Eq. (3) removes pairs where teacher similarity disagrees with ground truth; assumes the remaining pairs are informative and sufficient.
  • domain assumption Mean-pooling token vectors for each text in the concatenated input yields comparable sentence representations.
    Section 3.3 averages the first m and last n dimensions to obtain r_l_i and r_l_j; no evidence is given that this pooling aligns with the teacher's pooling.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Flipping Knowledge Distillation: Leveraging Small Models' Expertise to Enhance LLMs in Text Matching." pith.science (2026). https://pith.science/paper/E4HBRODJ

@misc{pith2026250705617,
  author       = {Pith},
  title        = {Pith review of: Flipping Knowledge Distillation: Leveraging Small Models' Expertise to Enhance LLMs in Text Matching},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E4HBRODJ}},
  note         = {Machine review of arXiv:2507.05617}
}
read the original abstract

Knowledge distillation typically involves transferring knowledge from a Large Language Model (LLM) to a Smaller Language Model (SLM). However, in tasks such as text matching, fine-tuned smaller models often yield more effective domain-specific representations, as they focus on optimizing the similarity of input pairs. To leverage both the specialized strengths of small models and the rich semantic understanding of LLMs, we introduce a flipped knowledge distillation paradigm, where LLM learns from SLM. Specifically, we address the architectural gap between decoder-only LLMs and smaller encoder-based models by reinterpreting LLMs in an encoder-decoder manner using LoRA. The encoder generates compressed representations, while the decoder maps them to the output space. During training, the encoder produces representations and their similarities, which are then aligned with the similarity scores produced by the teacher, using our proposed Margin-aware Contrastive Learning (MCL) approach. The MCL ensures accurate similarity for both positive and negative pairs, and adaptively handles the internal differences within positive and negative samples. Our paradigm requires only a reasonably good-performing SLM, allowing the LLM to achieve improved performance. Experiments on financial and healthcare benchmarks, as well as real-world applications, confirm its effectiveness, and the model has been fully deployed in an online environment.

Figures

Figures reproduced from arXiv: 2507.05617 by the authors.

Figure 1
Figure 1. (a) Comparison of traditional knowledge distillation from LLM to SLM, and our flipping knowledge [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of SLM, traditional LLM, and our reinterpreted LLM for text matching. We reinterpret the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of our framework. The SLM and LLM output the representations for each text, forming a matrix. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Impact of changing angular margin mc in LMCL. We take the F1 results of Qwen-0.5-flip (GTE) on Bytedance as example. (a) Qwen-0.5b Output Projection (b) Qwen-0.5b-flip Output Projection Samples with positive labels Samples with negative labels P r e dic t e d S c o r e…
Figure 5
Figure 5. Figure 5: Distribution of positive and negative sam [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 26 canonical work pages

  1. [1]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint arXiv:2309.16609

    arXiv 2023

  2. [2]

    Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. 2016. http://www.cl.uni-heidelberg.de/ riezler/publications/papers/ECIR2016.pdf A full-text learning to rank dataset for medical information retrieval

    work page 2016

  3. [3]

    Xiuying Chen, Mingzhe Li, Shen Gao, Xin Cheng, Qingqing Zhu, Rui Yan, Xin Gao, and Xiangliang Zhang. 2024 a . Flexible and adaptable summarization via expertise separation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2018--2027

    work page 2024

  4. [4]

    Xiuying Chen, Tairan Wang, Taicheng Guo, Kehan Guo, Juexiao Zhou, Haoyang Li, Zirui Song, Xin Gao, and Xiangliang Zhang. 2025. Unveiling the power of language models in chemical research question answering. Communications Chemistry, 8(1):4

    work page 2025

  5. [5]

    Xiuying Chen, Daorui Xiao, Shen Gao, Guojun Liu, Wei Lin, Bo Zheng, Dongyan Zhao, and Rui Yan. 2019. Rpm-oriented query rewriting framework for e-commerce keyword-based sponsored search. AAAI

    work page 2019

  6. [6]

    Xuanting Chen, Junjie Ye, Can Zu, Nuo Xu, Rui Zheng, Minlong Peng, Jie Zhou, Tao Gui, Qi Zhang, and Xuanjing Huang. 2023. How robust is gpt-3.5 to predecessors? a comprehensive study on language understanding tasks. arXiv preprint arXiv:2303.00293

    arXiv 2023

  7. [7]

    Zhipeng Chen, Kun Zhou, Wayne Xin Zhao, Junchen Wan, Fuzheng Zhang, Di Zhang, and Ji-Rong Wen. 2024 b . Improving large language models via fine-grained reinforcement learning with minimum editing constraint. arXiv preprint arXiv:2401.06081

    arXiv 2024

  8. [8]

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6

    2023

Show all 50 references
  1. [9]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. NAACL

    2018

  2. [10]

    Shizhe Diao, Tianyang Xu, Ruijia Xu, Jiawei Wang, and Tong Zhang. 2023. Mixture-of-domain-adapters: Decoupling and injecting domain knowledge to pre-trained language models’ memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V...

    2023

  3. [11]

    Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, 5(3):220--235

    2023

  4. [12]

    Lu Fan, Qimai Li, Bo Liu, Xiao-Ming Wu, Xiaotong Zhang, Fuyu Lv, Guli Lin, Sen Li, Taiwei Jin, and Keping Yang. 2022. Modeling user behavior with graph convolution for personalized product search. In Proceedings of the ACM Web Conference 2022, pages 203--212

    2022

  5. [13]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793

    2024 arXiv

  6. [14]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2023. Knowledge distillation of large language models. arXiv preprint arXiv:2306.08543

    2023 arXiv

  7. [15]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. Minillm: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations

    2024

  8. [16]

    Lars Hillebrand, Armin Berger, Tobias Deu er, Tim Dilmaghani, Mohamed Khaled, Bernd Kliem, R \"u diger Loitz, Maren Pielka, David Leonhard, Christian Bauckhage, et al. 2023. Improving zero-shot text matching for financial auditing with large language models. In Proceedings of ...

    2023

  9. [17]

    Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, and Peng Qi. 2024. Bad actor, good advisor: Exploring the role of large language models in fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 22105--22113

    2024

  10. [18]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations

  11. [19]

    Ashvini Kumar Jindal, Pawan Kumar Rajpoot, and Ankur Parikh. 2024. Upaya at the finllm challenge task 1 and 2: Distfin: Distillation based fine-tuning for financial tasks. In Proceedings of the Eighth Financial Technology and Natural Language Processing and the 1st Agent AI fo...

    2024

  12. [20]

    Mingzhe Li, Xiuying Chen, Jing Xiang, Qishen Zhang, Changsheng Ma, Chenchen Dai, Jinxiong Chang, Zhongyi Liu, and Guannan Zhang. 2024. Multi-intent attribute-aware text matching in searching. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining...

    2024

  13. [21]

    Xuanlin Li, Yunhao Fang, Minghua Liu, Zhan Ling, Zhuowen Tu, and Hao Su. 2023 a . Distilling large vision-language model with out-of-distribution generalizability. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2492--2503

    2023

  14. [22]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023 b . Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281

    2023 arXiv

  15. [23]

    Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, and Tuo Zhao. 2023. Less is more: Task-aware layer-wise distillation for language model compression. In International Conference on Machine Learning, pages 20852--20867. PMLR

    2023

  16. [24]

    Yubo Ma, Yixin Cao, Yong Hong, and Aixin Sun. 2023. Large language model is not a good few-shot information extractor, but a good reranker for hard samples! In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10572--10601

    2023

  17. [25]

    Cong-Duy Nguyen, Thong Nguyen, Xiaobao Wu, and Luu Anh Tuan. 2024. Kdmcse: Knowledge distillation multimodal sentence embeddings with adaptive angular margin contrastive learning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Comput...

    2024

  18. [26]

    Bhavani Shankar, Preethi Jyothi, and Pushpak Bhattacharyya. 2024. In-context mixing (icm): Code-mixed prompts for multilingual llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4162--4176

    2024

  19. [27]

    Zirui Song, Bin Yan, Yuhan Liu, Miao Fang, Mingzhe Li, Rui Yan, and Xiuying Chen. 2025. Injecting domain-specific knowledge into large language models: a comprehensive survey. arXiv preprint arXiv:2502.10708

    2025 arXiv

  20. [28]

    Michael Stewart, Melinda Hodkiewicz, and Sirui Li. 2023. Large language models for failure mode classification: an investigation. arXiv preprint arXiv:2309.08181

    2023 arXiv

  21. [29]

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford alpaca: An instruction-following llama model

    2023

  22. [30]

    Inar Timiryasov and Jean-Loup Tastet. 2023. Baby llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty. arXiv preprint arXiv:2308.02019

    2023 arXiv

  23. [31]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  24. [32]

    Charangan Vasantharajan, Kyaw Zin Tun, Ho Thi-Nga, Sparsh Jain, Tong Rong, and Chng Eng Siong. 2022. https://doi.org/10.23919/APSIPAASC55919.2022.9980157 Medbert: A pre-trained language model for biomedical named entity recognition . In 2022 Asia-Pacific Signal and Information...

    2022

  25. [33]

    Zheng Wang, Shu Xian Teo, Jieer Ouyang, Yongjun Xu, and Wei Shi. 2024. M-rag: Reinforcing large language model performance through retrieval-augmented generation with multiple partitions. ACL

    2024

  26. [34]

    Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Weidi Xie, and Yanfeng Wang. 2024. Pmc-llama: toward building open-source language models for medicine. Journal of the American Medical Informatics Association, page ocae045

    2024

  27. [35]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864

    2023 arXiv

  28. [36]

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighof. 2023. C-pack: Packaged resources to advance general chinese embedding. arXiv preprint arXiv:2309.07597

    2023 arXiv

  29. [37]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244

    2023 arXiv

  30. [38]

    Shicheng Xu, Liang Pang, Mo Yu, Fandong Meng, Huawei Shen, Xueqi Cheng, and Jie Zhou. 2024 a . Unsupervised information refinement training of large language models for retrieval-augmented generation. ACL

    2024

  31. [39]

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024 b . A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116

    2024 arXiv

  32. [40]

    Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023. Fingpt: Open-source financial large language models. FinLLM at IJCAI

    2023

  33. [41]

    Yi Yang, Mark Christopher Siy UY, and Allen Huang. 2020. http://arxiv.org/abs/2006.08097 Finbert: A pretrained language model for financial communications

    2020 arXiv

  34. [42]

    Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. 2023. Rrhf: Rank responses to align language models with human feedback without tears. arXiv preprint arXiv:2304.05302

    2023 arXiv

  35. [43]

    Cyril Zakka, Rohan Shad, Akash Chaurasia, Alex R Dalal, Jennifer L Kim, Michael Moor, Robyn Fong, Curran Phillips, Kevin Alexander, Euan Ashley, et al. 2024. Almanac—retrieval-augmented language models for clinical medicine. NEJM AI, 1(2):AIoa2300068

    2024

  36. [44]

    Yuhao Zhang, Hongji Zhu, Yongliang Wang, Nan Xu, Xiaobo Li, and Binqiang Zhao. 2022. A contrastive framework for learning sentence representations from pairwise and triple-wise perspective in angular space. In Proceedings of the 60th Annual Meeting of the Association for Compu...

    2022

  37. [45]

    Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Huadong Wang, Deming Ye, Chaojun Xiao, Xu Han, Zhiyuan Liu, Peng Li, Maosong Sun, et al. 2023. Plug-and-play knowledge injection for pre-trained language models. In Proceedings of the 61st Annual Meeting of the Association for Computat...

    2023

  38. [46]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  39. [47]

    Xiaoling Zhou, Wei Ye, Yidong Wang, Chaoya Jiang, Zhemg Lee, Rui Xie, and Shikun Zhang. 2024. Enhancing in-context learning via implicit demonstration augmentation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2024

  40. [48]

    Yicheng Zou, Hongwei Liu, Tao Gui, Junzhe Wang, Qi Zhang, Meng Tang, Haixiang Li, and Daniell Wang. 2022. Divide and conquer: Text semantic matching with disentangled keywords and intents. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3622--3632

    2022

  41. [49]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  42. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.