Pith. sign in

REVIEW 1 major objections 6 minor 38 references

Multi-task retriever fine-tuning for domain-specific and efficient RAG

T0 review · 1 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single multi-task fine-tuned retriever of 305M parameters outperforms BM25 and all evaluated open-source embedding models on domain-specific RAG retrieval, and transfers to new domains, languages, and retrieval tasks.

desk verdict Useful industrial case study with a fine-tuned 305M retriever beating BM25 on structured enterprise data, though the large instruction-tuned baselines are invoked with unspecified prompts and their near-zero scores look unfair. read the letter →

arxiv 2501.04652 v2 pith:6FFDRFOU submitted 2025-01-08 cs.CL cs.IRcs.LG

Patrice Béchard , Orlando Marquez Ayala This is my paper · ORCID
classification cs.CLcs.IRcs.LG
keywords Retrieval-AugmentedGenerationmulti-taskfine-tuninginstruction-tunedembeddingsdomain-specificretrievalworkflowBM25Recall@Kcontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that for real-world Retrieval-Augmented Generation (RAG), the right place to adapt to a domain is the retriever, not the large language model. The authors instruction fine-tune a single small multilingual encoder, mGTE-base (305M parameters), on a multi-task dataset built from existing labeled workflows and database tables, covering retrieval of workflow steps, table names, and table field names. On out-of-domain splits from ten enterprise deployments, this one fine-tuned model reaches 0.90 Recall@15 for steps, 0.90 Recall@5 for tables, and 0.60 Recall@5 for fields, beating BM25 and every open-source embedding baseline they compare against. The same model also transfers to a related but different task, workflow retrieval, and improves over the base model on five non-English languages. The practical stake is that one small retriever can serve many RAG applications at low cost and latency, instead of deploying separate retrievers or fine-tuning expensive LLMs.

What carries the argument

The load-bearing mechanism is multi-task instruction fine-tuning of a small retriever, mGTE-base, a multilingual encoder with an 8,192-token context chosen so instructions can be long and varied. The training data is generated without new labeling: positive pairs are extracted from existing labeled workflows and database tables, negatives come from random sampling and from hard negatives (e.g., a different step in the same scope), and 15 instruction templates add input diversity. A contrastive loss trains the encoder to pull matching text-object pairs together and push others apart, while exponential downsampling of very frequent steps, tables, and fields corrects the heavy data imbalance in real-world workflow data. This single recipe is what lets one encoder serve step, table, field, and subsequently workflow retrieval in the deployed ecosystem.

What would settle it

Recompute Recall@K on the ten OOD splits after independently verifying, through human annotation or exhaustive search of the enterprise databases, that no additional valid steps, tables, or fields exist for any query; if the relevant-item set for even a few queries grows, the reported gaps between the fine-tuned model and BM25 (and between it and the base model) could narrow or reverse. A cheaper check is to rerun the same multi-task fine-tuning on a public retrieval benchmark with complete relevance judgments, such as a domain-specific BEIR-style dataset, and see whether the advantage over BM25 persists out of the paper's own evaluation construction.

Watch

Extended reading notes

Core claim

On its own domain-specific retrieval benchmarks, the multi-task instruction fine-tuned mGTE-base model achieves the highest performance across all metrics, substantially outperforming both BM25 and all evaluated open-source embedding models. The fine-tuned model reaches 0.90 on both Step@15 and Table@5 and 0.60 on Field@5, whereas BM25 reaches 0.82, 0.79, and 0.26 and the base mGTE model reaches 0.72, 0.63, and 0.08. Increasing the scale of the mE5 retrievers from small to large yields no improvement, and larger instruction-tuned LLM-based embeddings such as GTE-Qwen2 perform poorly on this structured data. The paper further claims that the multi-task fine-tuning transfers to an unseen retrieval task, workflow retrieval, improving recall@5 from 0.87 (base model) to 0.94, and that fine-tuning on English data improves retrieval on German, Spanish, French, Japanese, and Hebrew relative to the base model, though still below English performance.

Load-bearing premise

Every reported Recall@K number rests on the assumption that the out-of-domain candidate pools are completely and correctly labeled, so that the labeled items are the only relevant items per query and everything else in the pool is a true negative; if any labeled set is incomplete, recall is underestimated and the comparison across models is distorted.

Editorial extensions

If this is right

  • One instruction fine-tuned retriever can replace multiple task-specific retrievers in an ecosystem of RAG applications, cutting deployment and maintenance costs.
  • Retriever fine-tuning offers a cheaper route to domain-adapted RAG than LLM fine-tuning, since the 305M-parameter encoder is far smaller than the generator.
  • Data imbalance, not just task diversity, determines multi-task retrieval quality; downsampling frequent steps improved step Recall@15 from 0.77 to 0.86 on the development set.
  • The same multi-task training transfers to related retrieval tasks such as workflow retrieval, so new RAG applications can reuse the retriever without retraining.
  • Multilingual capability is partially retained after English-only fine-tuning, but the performance gap to English suggests that multilingual domain data should be added for production use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The comparison suggests a division of labor: BM25 is competitive on exact-name retrieval (steps, tables), while the fine-tuned model's largest gain is on field names (0.60 vs 0.26 for BM25), where semantic matching over lexical matching matters most; this could be tested by ablating hard negatives.
  • A natural extension is to apply the same instruction-tuning recipe to other structured-data retrieval targets, such as database schema elements, API signatures, or configuration objects, where the same text-description-to-structured-item pattern holds.
  • The field-retrieval ceiling (0.60) is the clearest next target; adding harder negatives mined from same-table field lists, or synthesizing more field-description pairs, would likely move it more than adding model capacity.
  • The dependence of the reported recall numbers on complete relevance labels means the method's true ranking against BM25 in production should be validated with human feedback on retrieved results, not just offline pools.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 6 minor

Summary. The paper describes an approach to building a small, multi-task instruction-tuned retriever for domain-specific RAG. Starting from mGTE-base (305M parameters), the authors construct a large multi-task retrieval dataset from enterprise workflow data and database tables, covering step, table, and field retrieval, and fine-tune with a contrastive loss after applying an exponential downsampling schedule to frequent items. They evaluate on ten out-of-domain (OOD) splits from enterprise deployments, on translated multilingual versions of a development set, and on a related but unseen workflow retrieval task. The central empirical claim, stated in Section 5.2, is that the fine-tuned model substantially outperforms both BM25 and all evaluated open-source embedding models on their OOD benchmarks, reaching Step@15 0.90, Table@5 0.90, and Field@5 0.60, while also preserving and even improving multilingual retrieval and transferring to workflow retrieval.

Significance. If the empirical claims hold, this is a practically valuable case study: it shows that a single small retriever can serve multiple RAG use cases with different data types, with held-out OOD generalization and some evidence of transfer to a related task. The paper has clear strengths: evaluation on OOD splits that are not used for training, a related-task transfer evaluation, multilingual evaluation, explicit negative mining strategies, and reproducible training details in Appendix A. The main weakness is that the comparison against instruction-tuned LLM-based embedders is not described well enough to establish the headline claim, and the evaluation protocol omits candidate-pool and variance information that is needed to interpret the reported recall numbers. These issues are fixable and do not appear to invalidate the core engineering result, but they are load-bearing for the paper's strongest statements.

major comments (1)
  1. [Section 4.2, Table 4, Section 5.2] Section 5.1, Table 3: The exponential downsampling schedule is described as something the authors 'experimented with' on the development set, and the same development set is used for model selection and for the multilingual evaluation in Table 5. This is not circular for the OOD claim, but the paper should state how many configurations were tried, confirm that the final downsampling factors and hyperparameters were fixed before the OOD runs, and ideally report the chosen factors explicitly rather than only the qualitative rule.
minor comments (6)
  1. [Section 3.2] In the instruction template, 'rcope' should be 'scope'; as printed, the example contains a typo in a central illustration of the input format.
  2. [Section 1] The comparison with the embedding service is written as 'V oyage 1' with an odd space; it should be 'Voyage'.
  3. [Section 4.3] The sentence 'these metrics highly correlate with recall in our use case' should be 'correlate highly with recall' or 'are highly correlated with recall'.
  4. [Section 3.1] There is a typo in 'all Y AML lines up to definition'; it should be 'all YAML lines'.
  5. [Section 5.1] 'occuring' should be 'occurring' in the discussion of the imbalance in the step retrieval task.
  6. [Reference list] Reference [14] appears to have an extra 'Ge Yu.' fragment after the author list; the citation formatting should be corrected.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity in the derivation chain; the sole self-citation is non-load-bearing data provenance.

full rationale

The paper's central claim is an empirical comparison on held-out data. Training pairs are extracted from the Flow Generation training set and database tables (Section 3.2); the development set is the Flow Generation development set, and the OOD splits come from deployments in different domains (Section 4.1). The fine-tuned model is then evaluated on these OOD splits (Table 4), on a translated development set (Table 5), and on a workflow retrieval task built from OOD flows (Table 6). No equation defines the reported Recall@K values in terms of the fitted weights or dev-set hyperparameters; the downsampling factors are selected on the development set and then applied, and the OOD and workflow-retrieval evaluations are not used in training. The only prior self-citation, [3], is used to identify the Flow Generation training set and to frame the work as 'a natural extension'; it is not invoked as a uniqueness theorem and it does not constrain the outcome of the evaluations. Concerns about the instruction-tuned baselines, such as unspecified prompts for GTE-Qwen2 and E5-Mistral, are experimental-validity issues rather than circularity, so they do not change the circularity verdict. No circular step is exhibited.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the training data construction (172,658 pairs from internal databases), the instruction templates, the downsampling schedule, and standard contrastive training. The only tuned numeric is the downsampling schedule; all other choices are standard hyperparameters. No new physical entities are introduced.

free parameters (1)
  • Exponential downsampling schedule for frequent steps, tables, and fields = Steps occurring 50 times downsampled 4x; steps occurring 500 times downsampled 16x; similar interpolation for tables…
    Chosen after experiments on the development set; improves Step@15 from 0.77 to 0.86 at a small cost to Field@5 (0.73 to 0.71). This is a model-selection constant, not a test-set fit.
assumptions (4)
  • domain assumption Improvements in retrieved results translate into improvements in the downstream generation task.
    Stated explicitly in Section 1: 'assuming that improvements in the retrieved results translate into improvements in the downstream generation task.' The paper never measures end-to-end RAG quality.
  • domain assumption The ten OOD splits from deployments are representative of real-world customer settings and do not overlap with the training data.
    Section 4.1 states the splits include steps and tables not present in the IT domain, but no overlap test is described.
  • domain assumption Google Translate produces valid multilingual test data that preserves retrieval difficulty.
    Section 5.3 evaluates on translated dev sets; no manual validation of translations is reported.
  • standard math Contrastive loss trained with random and hard negatives produces a well-calibrated retriever.
    Uses standard contrastive learning (Hadsell et al. 2006) with batch size 32 and no in-batch negatives; this is a standard training objective.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-task retriever fine-tuning for domain-specific and efficient RAG." pith.science (2026). https://pith.science/paper/6FFDRFOU

@misc{pith2026250104652,
  author       = {Pith},
  title        = {Pith review of: Multi-task retriever fine-tuning for domain-specific and efficient RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6FFDRFOU}},
  note         = {Machine review of arXiv:2501.04652}
}
read the original abstract

Retrieval-Augmented Generation (RAG) has become ubiquitous when deploying Large Language Models (LLMs), as it can address typical limitations such as generating hallucinated or outdated information. However, when building real-world RAG applications, practical issues arise. First, the retrieved information is generally domain-specific. Since it is computationally expensive to fine-tune LLMs, it is more feasible to fine-tune the retriever to improve the quality of the data included in the LLM input. Second, as more applications are deployed in the same real-world system, one cannot afford to deploy separate retrievers. Moreover, these RAG applications normally retrieve different kinds of data. Our solution is to instruction fine-tune a small retriever encoder on a variety of domain-specific tasks to allow us to deploy one encoder that can serve many use cases, thereby achieving low-cost, scalability, and speed. We show how this encoder generalizes to out-of-domain settings as well as to an unseen retrieval task on real-world enterprise use cases.

Figures

Figures reproduced from arXiv: 2501.04652 by the authors.

Figure 1
Figure 1. Given an ecosystem of RAG applications, how do we [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Dataset example of Flow Generation. We constructed retrieval multi-task examples from them. (3) Retrieve table fields that can be used in step inputs or in code, given a table name. To add diversity to each of the tasks, we included several permu￾tations of the input, such as: • Retrieve all the steps used in the workflow given the require￾ment. In the example in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 13 canonical work pages

  1. [1]

    Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, and Wen-tau Yih. 2023. Task-aware Retrieval with Instructions. In Findings of the Association for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Ca...

    doi:10.18653/v1/ 2023

  2. [2]

    Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization

    Ryan C. Barron, Ves Grantcharov, Selma Wanna, Maksim E. Eren, Manish Bhat- tarai, Nicholas Solovyev, George Tompkins, Charles Nicholas, Kim Ø. Rasmussen, Cynthia Matuszek, and Boian S. Alexandrov. 2024. Domain-Specific Retrieval- Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization. arXiv:2410.02721 [cs.CL] https://arxiv.or...

    work page Pith review arXiv 2024

  3. [3]

    Patrice Bechard and Orlando Ayala. 2024. Reducing hallucination in structured out- puts via Retrieval-Augmented Generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track), Yi Yang, Aida Davani, Avi Sil, and Anoop Kumar (Eds.). Asso...

    doi:10.18653/v1/2024.naacl-industry.19 2024

  4. [4]

    Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961 (2024)

    arXiv 2024

  5. [5]

    Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. 2016. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06174 (2016)

    arXiv 2016

  6. [6]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (Barcelona, Spain) (KDD ’24). Association for Computing Machinery, New York, NY ...

    arXiv 2024

  7. [7]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. 2020. Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155 (2020)

    arXiv 2020

  8. [8]

    Luyu Gao, Yunyi Zhang, Jiawei Han, and Jamie Callan. 2021. Scaling deep contrastive learning batch size under memory limited setup. arXiv preprint arXiv:2101.06983 (2021)

    arXiv 2021

Show all 38 references
  1. [9]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997 [cs.CL] https://arxiv.org/abs/2312.10997

    2024 arXiv

  2. [10]

    Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2020. Graphcodebert: Pre-training code representations with data flow. arXiv preprint arXiv:2009.08366 (2020)

    2020 arXiv

  3. [11]

    Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), V ol. 2. IEEE, 1735–1742

    2006

  4. [12]

    Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS) 20, 4 (2002), 422–446

    2002

  5. [13]

    Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, and Nan Duan. 2022. Coderetriever: A large scale contrastive pre-training method for code search. In Proceedings of the 2022 Conference on Empirical Methods in Natural...

    2022

  6. [14]

    Xinze Li, Zhenghao Liu, Chenyan Xiong, Shi Yu, Yu Gu, and Zhiyuan Liu. 2023. Ge Yu. Structure-aware language model pretraining improves dense retrieval on structured data. arXiv preprint arXiv:2305.19912 (2023)

    2023 arXiv

  7. [15]

    Jean Maillard, Vladimir Karpukhin, Fabio Petroni, Wen-tau Yih, Barlas O ˘guz, Veselin Stoyanov, and Gargi Ghosh. 2021. Multi-task retrieval for knowledge- intensive tasks. arXiv preprint arXiv:2101.00117 (2021)

    2021 arXiv

  8. [16]

    Christopher D Manning. 2008. Introduction to information retrieval. Syngress Publishing,

    2008

  9. [17]

    Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2024. Generative Representational Instruc- tion Tuning. arXiv:2402.09906 [cs.CL]

    2024 arXiv

  10. [18]

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al

  11. [19]

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff ...

    2024 arXiv

  12. [20]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, a...

    2022

  13. [21]

    Gabriel Poesia, Oleksandr Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christo- pher Meek, and Sumit Gulwani. 2022. Synchromesh: Reliable code generation from pre-trained language models. arXiv preprint arXiv:2201.11227 (2022)

    2022 arXiv

  14. [22]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Em- pirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-I...

    2019 doi

  15. [23]

    S. E. Robertson and S. Walker. 1994. Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Dublin, Ireland) (S...

    1994

  16. [24]

    Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning. PMLR, 4596–4604. Multi-task retriever fine-tuning for domain-specific and efficient RAG KDD Workshop ’25, August 03–07,2025, T...

    2018

  17. [25]

    Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kaluarachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering. Transactions of the Association for Comp...

    2023 doi

  18. [26]

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A Smith, Luke Zettlemoyer, and Tao Yu. 2022. One embedder, any task: Instruction-finetuned text embeddings. arXiv preprint arXiv:2212.09741 (2022)

    2022 arXiv

  19. [27]

    Ellen M V oorhees, Dawn M Tice, et al. 1999. The TREC-8 Question Answering Track Evaluation.. In TREC, V ol. 1999. 82

    1999

  20. [28]

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533 (2022)

    2022 arXiv

  21. [29]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Improving Text Embeddings with Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre M...

    2024 doi

  22. [30]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Multilingual E5 Text Embeddings: A Technical Report. arXiv:2402.05672 [cs.CL] https://arxiv.org/abs/2402.05672

    2024 arXiv

  23. [31]

    Dai, and Quoc V Le

    Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022. Finetuned Language Models are Zero-Shot Learners. In International Conference on Learning Repre- sentations. https://openreview.net/forum?id=gEZrGCozdqR

    2022

  24. [32]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 (2024)

    2024 arXiv

  25. [33]

    Dejiao Zhang*, Wasi Ahmad*, Ming Tan, Hantian Ding, Ramesh Nallapati, Dan Roth, Xiaofei Ma, and Bing Xiang. 2024. CodeSage: Code Representation Learn- ing At Scale. In The Twelfth International Conference on Learning Representa- tions. https://openreview.net/forum?id=vfzRRjumpX

    2024

  26. [34]

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2024. Instruction Tuning for Large Language Models: A Survey. arXiv:2308.10792 [cs.CL] https: //arxiv.org/abs/2308.10792

    2024

  27. [35]

    Gonzalez

    Tianjun Zhang, Shishir G Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E. Gonzalez. 2024. RAFT: Adapting Language Model to Domain Specific RAG. In First Conference on Language Modeling . https:// openreview.net/forum?id=rzQGHXNReU

    2024

  28. [36]

    Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. 2024. mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval. InPr...

    2024

  29. [1412]

    Training hyperparameters were as follows: a batch size of 32 was employed, with a per-device batch size of 2 and 16 gradient accumulation steps

    doi:10.18653/v1/2024.emnlp-industry.103 A Training Details We fine-tuned mGTE-base for 5,000 steps, equivalent to approxi- mately 2 epochs on the dataset. Training hyperparameters were as follows: a batch size of 32 was employed, with a per-device batch size of 2 and 16 gradie...

    2024 doi

  30. [2022]

    arXiv preprint arXiv:2201.10005 (2022)

    Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005 (2022)

    2022 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.