Pith. sign in

REVIEW 4 major objections 4 minor 51 references

This paper argues that a fully automated multi-agent annotation and three-stage training pipeline can replace manual relevance labeling in social search, improving both offline relevance prediction and online user engagement on the QQ searc

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:43 UTC pith:UTZ3SC4A

load-bearing objection This is a credible production-relevance pipeline with clean internal logic, but the offline evaluation is substantially self-referential; the online A/B test is the only fully independent evidence. the 4 major comments →

arxiv 2607.26593 v1 pith:UTZ3SC4A submitted 2026-07-29 cs.IR

ASARL: Autonomous Social-Aware Relevance Learning for QQ Search

Tao Su , Jinjing Hu , Xiao Wang , Xingzhong Cao , Hui Wang This is my paper
classification cs.IR
keywords search relevancesocial searchmulti-agent data curationLLM reasoning tracespreference-guided optimizationknowledge distillationlong-tail queriesonline A/B testing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

ASARL claims that relevance in social search—matching casual, community-specific queries to group and channel titles—can be learned with almost no human labeling. Three specialized LLM agents build a training set: one reasons to a relevance label, one criticizes and corrects, and one invents titles for underrepresented long-tail queries. The curated data then drives three training stages: social-aware reasoning supervision, preference optimization against click and join signals, and distillation into a compact serving model. On QQ search the pipeline lifts offline Macro F1 from the best fine-tuned baseline's 68.35 to 75.59 for the small model and 83.66 for the larger one, with online CTR up 2.69%, join rate up 2.59%, and GSB up 11.66% in channel search. If right, it means the bottleneck for social search relevance is not annotation cost but the design of the agent loop and the social-attribute schema.

Core claim

The central claim is that social relevance can be decomposed into intent–attribute alignment against a small fixed set of social attributes, and that this decomposition can be taught to a small model via reasoning-trace supervision. The paper demonstrates that Social-Aware CoT plus label supervision reaches 75.59 Macro F1 versus 68.35 for label-only supervision, and that adding preference-guided optimization pushes the larger model to 83.66 Macro F1. The distilled student model outperforms the production encoder baseline by about 7 Macro F1 points, and an online A/B test on 12 million daily active users shows consistent gains in click-through rate, join rate, and GSB in both channel and grou

What carries the argument

The engine is the closed Reason–Critic–Gen loop. ReasonAgent produces (reasoning trace, relevance label) pairs by matching queries and titles against 10 structured social attributes; CriticAgent validates logical consistency and returns structured feedback, with re-annotation looping until convergence (about 21% of samples are corrected); GenAgent detects under-covered attribute–label regions and synthesizes new titles. This curated dataset feeds three training stages: Social Context Training (SCT) jointly optimizes reason-plus-label and label-only prompts, Preference-Guided Optimization (PGO) applies a DPO-style objective where the positive sequence is beam-searched to be consistent with a

Load-bearing premise

The ground-truth labels and the test labels are both produced by the same LLM agent loop, with the test set only 'manually reviewed' and no details of that review; if those labels carry systematic bias, the offline gains measure agreement with that bias rather than true relevance.

What would settle it

Take the ASARL-trained models and evaluate them on a test set labeled independently by multiple human annotators, with inter-annotator agreement reported, spanning the same long-tail social queries; if the Macro F1 advantage over fine-tuned baselines shrinks toward zero or reverses, the offline superiority is an artifact of the agent loop's self-consistent labels.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Social search systems can bootstrap relevance training without large human annotation efforts, since agent-generated labels plus a self-critique loop replace manual labeling.
  • Behavioral signals like clicks and group-join rates can be folded into relevance training through preference pairs, without needing new human preference labels.
  • Domain-specific reasoning supervision (social-aware CoT) is worth more than generic chain-of-thought or label-only supervision, a finding that likely transfers to other community-driven search verticals.
  • The gains are compact enough to survive distillation: the distilled student outperforms the production encoder baseline, so the approach fits latency-constrained deployment.
  • The online A/B gains indicate that better social relevance translates into measurable user engagement, not just offline metric movement.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: Because the agent loop's own labels define both training and test ground truth, the reported Macro F1 gap may substantially measure how well the small model imitates the large annotator's reasoning style; a human-labeled test set is the missing control.
  • Inference: The 10 social attributes are a design choice, not a discovered structure; other platforms would need to adapt or learn the schema, and the framework gives no method for doing so automatically.
  • Inference: The PGO stage relies on unspecified CTR/join-rate thresholds to decide which mispredictions are 'verified'; those thresholds are likely a key hidden hyperparameter that could change the magnitude of online gains.
  • Inference: A testable extension is to run the same pipeline with the critic stage removed but the same total annotation budget; the ~21% correction rate suggests that quality, not just quantity, drives the offline gains, which can be isolated.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ASARL, an automated framework for social-search relevance learning on the QQ platform. It combines a multi-agent data-curation pipeline (ReasonAgent, CriticAgent, GenAgent) with three-stage training: Social Context Training (SCT), Preference-Guided Optimization (PGO), and Social Distillation (SD). Offline experiments report Macro F1 / NDCG / Accuracy gains over BERT, RoBERTa, and Qwen3 baselines, and online A/B tests report CTR, join-rate, and GSB improvements in channel and group search. The abstract also claims 'enhanced annotation efficiency' without providing a quantitative measure.

Significance. If the results hold, the paper demonstrates a practical industrial system that automates training-data curation for a hard domain (informal social search) and shows both offline and online improvements. The online A/B test is a real-world validation with 12M DAU, which is a notable strength. However, the central offline relevance claim is weakened by the fact that the test labels are produced by the same agent system that generates the training data, and by the absence of statistical significance testing. The reported gains are plausible but not yet convincingly separated from the possibility that the student model is simply imitating the agent's annotation style.

major comments (4)
  1. [§5.1.1 / §4.1.1] The test-set ground-truth labels are generated by the ReasonAgent/CriticAgent loop using Qwen3-235B and are only described as 'manually reviewed to ensure annotation accuracy.' No reviewer count, annotation guidelines, agreement statistics, or independent human-labeled benchmark are provided. Because SCT (Eq. 9) trains the student on the same reasoning-trace-plus-label format produced by the agent system, offline gains in Tables 3–5 may partly measure fidelity to the agent's label distribution rather than true social relevance. Please provide an independently labeled human test set, or report inter-annotator agreement and a detailed description of the manual review process. Without this, the key relevance claim is not fully established.
  2. [Tables 3–6] No confidence intervals, significance tests, or repeated-run variance are reported anywhere. For example, in Table 3 the difference between ASARL SCT 0.6B (Macro F1 75.59) and ASARL SCT+PGO 0.6B (75.65) is 0.06 points, which is almost certainly within noise. The paper repeatedly states 'improvements' but does not quantify uncertainty. Please add statistical tests (e.g., bootstrap or paired tests) or report variance across seeds. This is especially important because the offline gains over the fine-tuned LLMBase baseline are fairly large, but the reader needs to know whether the incremental claims (PGO, agent ablations) are reliable.
  3. [Abstract / §6] The abstract and conclusion claim 'enhanced annotation efficiency' from the multi-agent system, but no efficiency metric is ever defined or measured. If the claim is about reduced human labeling effort, compare with the cost of manual annotation, or specify the throughput/COST of the agent pipeline. As written, this is an unsupported claim and should either be quantified or removed.
  4. [§4.2.2] PGO relies on user-interaction signals 'CTR, join rate' with 'predefined thresholds' to verify labels, but these thresholds are never specified. This makes the method non-reproducible and also introduces free parameters that could affect the results. Please state the thresholds, how they were chosen, and whether results are sensitive to them. Also, Eq. (11) selects the positive sequence by beam search under the same model being optimized; this may introduce a self-training bias that is not discussed.
minor comments (4)
  1. [§5.2.3] Section title uses 'Effective of constructed COT Tuning'; should be 'Effect of...'.
  2. [References] Reference [22] has a typo in the arXiv identifier: '2402.128751' should probably be '2402.12875' or similar. Please verify.
  3. [Eq. (9)] The notation T_i in the first summand is not defined. Specify whether T_i is the token length of the reasoning-label sequence for sample i.
  4. [Table 5] The 'Social-Aware CoT' row (73.55/76.41/75.75) is identical to the 'Reason+Critic+Gen' row in Table 4. This is presumably intentional but should be stated explicitly, since the reader may otherwise wonder if the table labels are swapped.

Circularity Check

1 steps flagged

Offline relevance evaluation is self-referential: the same agent system writes both the training labels and the test-set 'ground truth', so the headline gains partly measure imitation of the agent label distribution rather than independent social relevance.

specific steps
  1. self definitional [§4.1.1 Eq. (1); §4.2.1 Eq. (9); §5.1.1 Datasets; §5.1.2 Metrics]
    "For each pair (q_i,t_i), the agent performs intent–attribute matching and outputs a relevance label l_i ... along with a reasoning trace r_i: (r_i,l_i)←R(q_i,t_i). ... Both the training and test splits are drawn from the same overall distribution. ... The test set has been manually reviewed to ensure annotation accuracy."

    The ground-truth labels used for offline evaluation are produced by the same ReasonAgent (Qwen3-235B) and CriticAgent loop that creates the training labels. SCT (Eq. 9) optimizes the model to output exactly those (reason,label) pairs; the test set is drawn from the same distribution and the only stated human check is an unquantified 'manual review'. Thus the offline Macro F1/NDCG/ACC comparison is, by construction, a measure of agreement with the agent label-generation distribution, not with an independent human relevance judgment. The claim 'better alignment with ground-truth relevance labels' (§5.1.2) is therefore self-referential unless the manual review is shown to be independent and reliable, which the paper does not specify.

full rationale

The paper does not rely on self-citations or imported uniqueness theorems; the method is a standard multi-agent LLM annotation plus SFT/DPO/distillation pipeline. The circularity risk is concentrated in offline evaluation: the relevance labels that define 'ground truth' are generated by the same ReasonAgent/CriticAgent loop (Qwen3-235B) that produces the training data, and SCT's objective (Eq. 9) is to reproduce exactly that label-and-reasoning format. The test split is stated to come from the same overall distribution, and the only human check is an unspecified 'manually reviewed' sentence with no reviewer count, agreement statistics, or criteria. Consequently the large offline gains (e.g., Table 3: LLMBase 68.35 → ASARL SCT 0.6B 75.59) are partly a measure of how faithfully the student imitates the agent label distribution rather than of independent social relevance. The online A/B engagement metrics (CTR/JR/GSB) are real and non-circular behavioral evidence, but they speak to engagement, not to correctness of the relevance labels. The abstract's 'enhanced annotation efficiency' is also never quantified. Score 6 reflects this partial, evaluation-level circularity; it is not a claim of authorial intent.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

Central quantitative claims rest on four unstated modeling choices: the PGO verification thresholds, the 10-attribute taxonomy, the three-tier sampling mix, and the DPO temperature/distillation form. In addition, the paper assumes LLM-generated labels plus an undescribed manual review are valid relevance ground truth, that click/join signals certify labels, and that agent-loop correction converges to higher quality. No new scientific entities are postulated; the three agents are software wrappers around Qwen3-235B, and the attribute taxonomy is a categorization rather than an invented entity.

free parameters (4)
  • PGO verification thresholds (CTR, join rate) = not reported
    §4.2.2: samples are 'considered valid if the metrics satisfy predefined thresholds'; these unreported thresholds determine which mispredicted samples enter DPO training, so the PGO-stage gains (0.06–0.94 Macro F1 points) depend on an invisible choice.
  • Social attribute taxonomy (Table 2) = 10 categories: Film, Lifestyle, Industry, Education, Gaming, Location, Gender, Age, Knowledge, Otaku
    ReasonAgent's intent–attribute matching and SCT's reasoning supervision are defined over this hand-chosen list; no justification, validation, or coverage analysis is given.
  • Three-tier candidate sampling proportions = not reported
    §5.1.1 samples 10 titles/query 'from three tiers: top-ranked, token-matched, randomly selected'; the mix determines label balance and task difficulty but is unstated.
  • DPO β and distillation form (hard/soft) = not reported
    Eq. (12) contains β, and Eq. (13) allows either softmax teacher logits or one-hot labels; neither choice is reported, and both affect the final numbers.
axioms (4)
  • domain assumption Qwen3-235B agent labels are valid relevance ground truth, with CriticAgent feedback improving quality (21% correction rate).
    §4.1, Table 4. The offline evaluation treats agent-generated labels (lightly human-reviewed) as truth; the 21% correction rate (§5.2.2) is asserted without evidence that corrections improve rather than merely change labels.
  • domain assumption User engagement signals above unstated thresholds certify label validity for DPO.
    §4.2.2. Treating CTR/join rate as verification ignores position bias, clickbait, and mismatched query intent.
  • domain assumption The Reason→Critic feedback loop converges to higher-quality annotations.
    Algorithm 1; no convergence analysis or per-iteration quality measurement is given.
  • domain assumption Offline label agreement predicts online engagement.
    The paper pairs offline relevance gains with online CTR/JR gains without testing the correlational link between them.

pith-pipeline@v1.3.0-daily-deepseek · 13478 in / 20468 out tokens · 197030 ms · 2026-08-01T12:43:05.105695+00:00 · methodology

0 comments
read the original abstract

The rapid growth of online social platforms has transformed communication and information retrieval, giving rise to social search, where queries-titles are typically expressed in informal, community-specific language. While large language models provide strong general-purpose semantic understanding, their effectiveness in social search is constrained by contextual discrepancy, data scarcity, and behavior-driven dynamics. To address these challenges, we propose the Autonomous Social-Aware Relevance Learning (ASARL), a fully automated framework that integrates multi-agent data curation with staged model training. ASARL leverages a collaborative agent system: ReasonAgent generates interpretable relevance labels grounded in social attributes, CriticAgent validates and ensures logical consistency, and GenAgent augments long-tail data through synthetic query-title pairs. Building on the curated dataset, ASARL employs three-stage training: Social Context Training (SCT) to capture social language patterns, Preference-Guided Optimization (PGO) to align model predictions with behavioral signals, and Social Distillation (SD) to transfer these improvements into compact models for efficient deployment. Extensive offline and online experiments on the QQ search platform demonstrate significant improvements in both offline relevance metrics and online user engagement indicators, along with enhanced annotation efficiency. These results validate the effectiveness of combining autonomous, socially grounded data governance with preference-aligned training in practical search systems.

Figures

Figures reproduced from arXiv: 2607.26593 by Hui Wang, Jinjing Hu, Tao Su, Xiao Wang, Xingzhong Cao.

Figure 1
Figure 1. Figure 1: Comparison between (a) a conventional relevance [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of different search paradigms: (a) con [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of ASARL. Queries are annotated and validated by a multi-agent system (ReasonAgent, CriticAgent, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of the collaborative workflow among ReasonAgent, CriticAgent, and GenAgent. The ReasonAgent [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 22 linked inside Pith

  1. [1]

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. 2025. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925(2025)

    Pith/arXiv arXiv 2025

  2. [2]

    Qingyao Ai, Yongfeng Zhang, Keping Bi, Xu Chen, and W Bruce Croft. 2017. Learning a hierarchical embedding model for personalized product search. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 645–654

    2017

  3. [3]

    Sergey Brin and Lawrence Page. 1998. The anatomy of a large-scale hypertextual web search engine.Computer networks and ISDN systems30, 1-7 (1998), 107–117

    1998

  4. [4]

    Ben Chen, Xian Guo, Siyuan Wang, Zihan Liang, Yue Lv, Yufei Ma, Xinlong Xiao, Bowen Xue, Xuxin Zhang, Ying Yang, et al . 2025. OneSearch: A Preliminary Exploration of the Unified End-to-End Generative Framework for E-commerce Search.arXiv preprint arXiv:2509.03236(2025)

    arXiv 2025

  5. [5]

    J. C.-Y. Chen, S. Saha, E. Stengel-Eskin, and M. Bansal. 2024. MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models.arXiv preprint arXiv:2402.01620(2024)

    Pith/arXiv arXiv 2024

  6. [6]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 4171–4186

    2019

  7. [7]

    Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization.arXiv preprint arXiv:2402.01306(2024)

    Pith/arXiv arXiv 2024

  8. [8]

    Zheng Fang, Donghao Xie, Ming Pang, Chunyuan Yuan, Xue Jiang, Changping Peng, Zhangang Lin, and Zheng Luo. 2025. ADORE: Autonomous Domain- Oriented Relevance Engine for E-commerce. InProceedings of the 48th Inter- national ACM SIGIR Conference on Research and Development in Information Retrieval. 4259–4263

    2025

  9. [9]

    Yangyang Guo, Zhiyong Cheng, Liqiang Nie, Yinglong Wang, Jun Ma, and Mohan Kankanhalli. 2019. Attentive long short-term preference modeling for person- alized product search.ACM Transactions on Information Systems (TOIS)37, 2 (2019), 1–27

    2019

  10. [10]

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. De- berta: Decoding-enhanced bert with disentangled attention.arXiv preprint arXiv:2006.03654(2020)

    Pith/arXiv arXiv 2020

  11. [11]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al

  12. [12]

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes.arXiv preprint arXiv:2305.02301(2023)

    Pith/arXiv arXiv 2023

  13. [13]

    Jui-Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020. Embedding- based retrieval in facebook search. InProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2553–2561

    2020

  14. [14]

    M. Kang. 2025. Agent Distillation: Transfer Learning from LLM Agents to Small Models.arXiv preprint arXiv:2505.17612(2025)

    arXiv 2025

  15. [15]

    Ao Li, Yuexiang Xie, Songze Li, Fugee Tsung, Bolin Ding, and Yaliang Li. 2024. Agent-oriented planning in multi-agent systems.arXiv preprint arXiv:2410.02189 (2024)

    Pith/arXiv arXiv 2024

  16. [16]

    Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large language model society.Advances in Neural Information Processing Systems36 (2023), 51991–52008

    2023

  17. [17]

    Lan Li, Liri Fang, Bertram Ludäscher, and Vetle I Torvik. 2024. AutoDCWorkflow: LLM-based data cleaning workflow auto-generation and benchmark.arXiv preprint arXiv:2412.06724(2024)

    Pith/arXiv arXiv 2024

  18. [18]

    Xiaoyu Li, Xiao Li, Li Gao, Yiding Liu, Xiaoyang Wang, Shuaiqiang Wang, Junfeng Wang, and Dawei Yin. 2025. Proactive Guidance of Multi-Turn Conversation in Industrial Search.arXiv preprint arXiv:2505.24251(2025)

    Pith/arXiv arXiv 2025

  19. [19]

    Xiaopeng Li, Xiangyang Li, Hao Zhang, Zhaocheng Du, Pengyue Jia, Yichao Wang, Xiangyu Zhao, Huifeng Guo, and Ruiming Tang. 2024. Syneg: Llm-driven synthetic hard-negatives for dense retrieval.arXiv preprint arXiv:2412.17250 (2024)

    Pith/arXiv arXiv 2024

  20. [20]

    Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges.Vicinagearth1, 1 (2024), 9

    2024

  21. [21]

    Yuchen Li, Hengyi Cai, Rui Kong, Xinran Chen, Jiamin Chen, Jun Yang, Haojie Zhang, Jiayi Li, Jiayi Wu, Yiqun Chen, et al. 2025. Towards AI Search Paradigm. arXiv preprint arXiv:2506.17188(2025)

    arXiv 2025

  22. [22]

    Zhiyuan Li, Hong Liu, Denny Zhou, and Tengyu Ma. 2024. Chain of thought empowers transformers to solve inherently serial problems.arXiv preprint arXiv:2402.128751 (2024)

    Pith/arXiv arXiv 2024

  23. [23]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Cheng- gang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437(2024)

    Pith/arXiv arXiv 2024

  24. [24]

    Yiding Liu, Weixue Lu, Suqi Cheng, Daiting Shi, Shuaiqiang Wang, Zhicong Cheng, and Dawei Yin. 2021. Pre-trained language model for web-scale retrieval in baidu search. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 3365–3375

    2021

  25. [25]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 (2019)

    Pith/arXiv arXiv 2019

  26. [26]

    Mallampati et al

    S. Mallampati et al. 2025. Dynamic Strategy Adaptation in Multi-Agent Environ- ments with Large Language Models.arXiv preprint arXiv:2507.02002(2025)

    arXiv 2025

  27. [27]

    Motwani et al

    P. Motwani et al. 2024. MALT: Improving Reasoning with Multi-Agent LLM Training.arXiv preprint arXiv:2412.01928(2024)

    arXiv 2024

  28. [28]

    Wenjun Peng, Guiyang Li, Yue Jiang, Zilong Wang, Dan Ou, Xiaoyi Zeng, Derong Xu, Tong Xu, and Enhong Chen. 2024. Large language model based long-tail query rewriting in taobao search. InCompanion Proceedings of the ACM Web Conference 2024. 20–28

    2024

  29. [29]

    Danrui Qi, Zhengjie Miao, and Jiannan Wang. 2024. Cleanagent: Automating data standardization with llm-based agents.arXiv preprint arXiv:2403.08291(2024)

    Pith/arXiv arXiv 2024

  30. [30]

    Weicong Qin, Yi Xu, Weijie Yu, Chenglei Shen, Ming He, Jianping Fan, Xiao Zhang, and Jun Xu. 2025. MAPS: Motivation-Aware Personalized Search via LLM-Driven Consultation Alignment.arXiv preprint arXiv:2503.01711(2025)

    Pith/arXiv arXiv 2025

  31. [31]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems36 (2023), 53728–53741

    2023

  32. [32]

    Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2021. Colbertv2: Effective and efficient retrieval via lightweight late interaction.arXiv preprint arXiv:2112.01488(2021)

    Pith/arXiv arXiv 2021

  33. [33]

    Hongwei Shang, Nguyen Vo, Nitin Yadav, Tian Zhang, Ajit Puthenputhussery, Xunfan Cai, Shuyi Chen, Prijith Chandran, and Changsung Kang. 2025. Knowl- edge Distillation for Enhancing Walmart E-commerce Search Relevance Using Large Language Models. InCompanion Proceedings of the ACM on Web Conference

    2025

  34. [34]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300(2024)

    Pith/arXiv arXiv 2024

  35. [35]

    Tian Tang, Zhixing Tian, Zhenyu Zhu, Chenyang Wang, Haiqing Hu, Guoyu Tang, Lin Liu, and Sulong Xu. 2025. LREF: A Novel LLM-based Relevance Framework for E-commerce Search. InCompanion Proceedings of the ACM on Web Conference

    2025

  36. [36]

    Rohan Taori, Ishaan Mishra, Shishir Jiang, Shyam Narang, Ashish Arya, and Carrie Liu. 2023. Introducing Alpaca: A Strong and Performant Instruction- Following Language Model. Accessed: 2024-06-10

    2023

  37. [37]

    Xianyang Tian, Xiang Xu, Chao Wang, Tong Ruan, Baohua Wu, Maofei Que, Shenghua Ni, Zhuoran Zhuang, and Jingping Liu. 2025. Towards Explainable Search Results in E-commerce. InCompanion Proceedings of the ACM on Web Conference 2025. 476–484

    2025

  38. [38]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

    2022

  39. [39]

    Fang Wu, Vijay Prakash Dwivedi, and Jure Leskovec. 2025. Large Language Models are Good Relational Learners.arXiv preprint arXiv:2506.05725(2025)

    Pith/arXiv arXiv 2025

  40. [40]

    Mingrui Wu and Sheng Cao. 2024. Llm-augmented retrieval: Enhancing re- trieval models through language models and doc-level embedding.arXiv preprint arXiv:2404.05825(2024)

    Pith/arXiv arXiv 2024

  41. [41]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

    Pith/arXiv arXiv 2025

  42. [42]

    Yingxuan Yang, Huacan Chai, Shuai Shao, Yuanyi Song, Siyuan Qi, Renting Rui, and Weinan Zhang. 2025. Agentnet: Decentralized evolutionary coordination for llm-based multi-agent systems.arXiv preprint arXiv:2504.00587(2025)

    Pith/arXiv arXiv 2025

  43. [43]

    Shaowei Yao, Jiwei Tan, Xi Chen, Juhao Zhang, Xiaoyi Zeng, and Keping Yang

  44. [44]

    Changlong Yu, Xin Liu, Jefferson Maia, Tianyu Cao, Laurence (Yang) Li, Yifan Gao, Yangqiu Song, Rahul Goutam, Haiyang Zhang, Bing Yin, and Zheng Li. 2024. COSMO: A large-scale e-commerce common sense knowledge generation and serving system at Amazon. (2024). https://www.amazon.science/publications/cosmo-a-large-scale-e-commerce- common-sense-knowledge-gen...

    2024

  45. [45]

    Yuan et al

    Y. Yuan et al. 2024. EvoAgent: Evolutionary Multi-Agent Optimization for LLMs. arXiv preprint arXiv:2406.14228(2024). Conference’17, July 2017, Washington, DC, USA Tao Su, Jinjing Hu, Xiao Wang, Xingzhong Cao, and Hui Wang

    Pith/arXiv arXiv 2024

  46. [46]

    Qiang Zhang, Fengshun Xiao, Tianjiao Li, Li Lin, Hanyin Fang, Huyang Sun, Ruoyu Liu, Xiaoyan Zhu, and Jiayin Wang. 2025. Efficient Integration of ASR with Large Language Models to Enhance Video Search at Scale. InCompanion Proceedings of the ACM on Web Conference 2025. 601–610

    2025

  47. [47]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models.arXiv preprint arXiv:2210.03493 (2022)

    Pith/arXiv arXiv 2022

  48. [48]

    Gang Zhao, Ximing Zhang, Chenji Lu, Hui Zhao, Tianshu Wu, Pengjie Wang, Jian Xu, and Bo Zheng. 2025. Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning. InCompanion Proceedings of the ACM on Web Conference 2025. 631–640

    2025

  49. [49]

    Yaochen Zhu, Liang Wu, Qi Guo, Liangjie Hong, and Jundong Li. 2024. Collab- orative large language model for recommender systems. InProceedings of the ACM Web Conference 2024. 3162–3172

    2024

  50. [2022]

    InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining

    ReprBERT: distilling BERT to an efficient representation-based relevance model for e-commerce. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining. 4363–4371

  51. [2024]

    International Conference on Learning Representations, ICLR

    MetaGPT: Meta programming for a multi-agent collaborative framework. International Conference on Learning Representations, ICLR