{"id":"1f24e12e-9ac0-4bed-8f45-0c62e8973040","arxiv_id":"2509.05570","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Fine-tuning a LLaMA model with GRPO on live search-engine feedback yields query expansions that retrieve more relevant and diverse product results in e-commerce search.","lead":"This paper trains a small language model to rewrite vague shopping queries, using the live search engine's own retrieval results as the reward signal. The approach reports large gains in product coverage and relevance offline, plus higher engagement in an eBay A/B test.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The offline gains and the Eq. (1) reward are both computed from the same internal GBDT relevance model; since no human-rated or otherwise independent correlation is reported and the online lift is confidential, the claimed relevance improvements may be artifacts of reward hacking rather than…","rationale":"The central claim requires that LESER improves real retrieval relevance and coverage, not merely scores from an internal model. The reward in Eq. (1) and the offline evaluation metric in Section 5.2 both rely on the same GBDT relevance model described in Section 5.1. Because the policy is trained to maximize this reward, the offline comparison is partly circular: LESER is rewarded for exactly what the evaluation measures. The paper provides no sign that the GBDT's score deltas agree with human judgments on expanded queries, and the only independent signal, the online A/B test, is withheld as confidential in Section 5.4. This combination means the public evidence cannot distinguish genuine retrieval improvement from reward hacking. The reader's conditional verdict is appropriate: the method is plausible and the ablation/qualitative evidence is useful, but the empirical core needs an independent relevance measurement or disclosed online effect sizes before the strong claim can be accepted. I therefore agree with the reader's weakest-assumption analysis and see no reason to move the verdict.","tokens_in":12369,"tokens_out":5905,"duration_ms":56930,"concrete_test":"Run a blind human relevance evaluation on a random sample of 500 queries from the 10,000-query offline set: have trained raters judge the top-10 items retrieved for the original query and for the LESER-expanded query using the same five-point Perfect/Excellent/Good/Fair/Bad scale. Compute human-rated ΔRel and compare it with the GBDT-based ΔRel reported in Table 2. If GBDT-based ΔRel is positive while human-rated ΔRel is at or below zero, the reward proxy in Eq. (1) is misaligned and the offline relevance claim is an artifact. Reporting the online A/B lift with confidence intervals would be a complementary check.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (1) defines the reward from Rel(Y), the average GBDT relevance score of top-10 retrieved items, and Section 5.2 computes the headline offline metric AvgRel (top-100) with the same GBDT model described in Section 5.1. LESER is trained by maximizing this reward, so the offline ΔRel is at least partly a fixed-point of the objective: a policy can learn surface attribute patterns (e.g., 'Intended Use', 'Type') that inflate GBDT scores without resolving the user's latent intent. The paper reports no calibration of GBDT scores against human labels for expanded queries and no analysis of reward hacking. The only independent evidence is the online A/B test, but Section 5.4 explicitly states that 'specific lift percentages remain confidential', so effect sizes, confidence intervals, and the statistical protocol cannot be checked. Additionally, offline results are reported as the percentage of queries with any positive gain, not the magnitude of ΔRet/ΔRel, which can make small or even noisy gains look substantial. Without an independent relevance measure, neither Eq. (1) nor the offline evaluation supports the strong claim of improved real-world relevance.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LESER, a query-expansion framework that fine-tunes a context-aware LLM (LLaMA 3.2-3B) with Group Relative Policy Optimization (GRPO) using real-time search-engine feedback as the reward signal. The reward in Eq. (1) combines a normalized relevance metric and a retrieval-coverage metric computed from the search engine's top-k results. The authors report offline results on 10,000 real-world e-commerce queries, ablation studies over backbone sizes and training stages, qualitative examples, and a production A/B test. The central claim is that LESER substantially improves retrieval coverage and relevance relative to embedding-based expansion, few-shot GPT-4.1, and supervised fine-tuning, and delivers measurable user-engagement gains online. The paper positions LESER as a scalable replacement for prompt-based LLM expansion and supervised query rewriting in e-commerce search.","tokens_in":12630,"tokens_out":4016,"duration_ms":36833,"significance":"If the claims are validated, LESER is a practically meaningful contribution: it removes the need for supervised labels, incorporates live retrieval signals into LLM training, and demonstrates a production deployment path with a real A/B test. The use of search-engine feedback as a reward is a sensible formulation, and the GRPO-based relative-reward design is appropriate for the one-to-many nature of query expansion. The comparison against GPT-4.1 few-shot prompting and an embedding baseline is useful. However, the current evidence for the central claim is weakened by the fact that the offline reward and the offline evaluation metric share the same internal relevance model, and by the confidentiality of the online lift figures. The contribution is therefore defensible but not yet fully supported.","major_comments":[{"comment":"The reward in Eq. (1) is computed from Rel(Y), which uses relevance scores from the internal GBDT relevance model described in Section 5.1, and the offline evaluation metric AvgRel in Section 5.2 is computed with the same model. The training objective therefore directly maximizes the quantity used to measure success, so the offline ΔRel gains are at least partially a fixed point of the optimization and may reflect gaming of the model rather than genuine relevance improvement. The paper reports no calibration of GBDT scores against human labels on expanded queries and no analysis of reward hacking. To support the relevance claim, the authors should provide an evaluation that does not share the reward model, such as human relevance judgments on the union retrieval set or engagement signals on the retrieved items, and report the correlation between GBDT scores and human labels for query expansions.","section":"§5.1, §5.2, Eq. (1)"},{"comment":"The online A/B test is the only independent evidence for real-world efficacy, but Section 5.4 states that 'specific lift percentages remain confidential' and provides no confidence intervals, p-values, or details of the statistical protocol. A statement that results are 'statistically significant' is not verifiable by the reader. At a minimum, the paper should report significance levels, confidence intervals for relative changes, or an anonymized summary that does not reveal absolute revenue figures. Without such information, the central claim of 'measurable gains in user engagement' cannot be independently assessed.","section":"§5.4"},{"comment":"The metrics ΔRet and ΔRel are defined in Section 5.2 as magnitudes (number of new unique items and difference in average relevance), but Table 2 reports the percentage of queries exhibiting any positive gain. This reporting choice conflates tiny, perhaps noisy, gains with substantial ones: a query gaining one marginal item is counted the same as a query gaining hundreds, and a relevance increase of 0.001 is counted the same as a large improvement. The authors should report the distribution of gains (mean, median, quantiles), the fraction of queries with gains above a meaningful threshold, and a query-level significance test to demonstrate that the improvements are not driven by small or noisy effects.","section":"§5.2, Table 2"}],"minor_comments":[{"comment":"In Section 5.2, ΔRet is defined as the cardinality of a set difference, but the reported value in Table 2 is a percentage of queries; the relationship between these two notions should be stated explicitly.","section":"§5.2"},{"comment":"There are typographical errors such as 'with in the expansion field' and 'an LLM is trained to perform context-aware reasoning'; these should be corrected in a revision.","section":"§1"},{"comment":"The sentence 'the relevance score is generated from internal relevance model from retrieved top 10 items' is grammatically unclear; it should specify which model produces the score and whether the top-10 context is the same set used for reward computation.","section":"§5.1"},{"comment":"In Table 3, the first row (no WarmUp, no GRPO) should be explicitly labeled as the base pretrained LLaMA 3.2-3B-Instruct model to avoid ambiguity about the setting.","section":"Table 3"},{"comment":"Appendix A repeats the implementation details from Section 5.1 almost verbatim; the duplication should be removed or condensed to a summary.","section":"Appendix A"},{"comment":"The manuscript contains placeholder conference information ('Conference acronym ’XX, Woodstock, NY') and missing DOIs for several references; these should be completed before publication.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an industry contribution from eBay, and the confidential online lift figures are a serious reproducibility limitation. If the journal accepts such papers, it should require at least blinded aggregate statistics (e.g., confidence intervals without absolute levels) so the referees and readers can verify the online claims. The offline circularity between reward and metric should be addressed with an independent evaluation before the paper can be recommended for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, quick take on arXiv:2509.05570 (LESER). The paper is a credible industrial contribution: a practical GRPO-based method for expanding e-commerce queries using live search feedback, with a clean reward formulation and a sensible two-stage training pipeline. The novelty is not any single component—PRF, GRPO, and LLM expansion all exist—but the specific combination, including the normalized search-feedback reward and the strict JSON validation gate, is genuinely new relative to the cited literature. The ablations are useful, particularly the finding that GRPO without SFT warm-up fails to follow the required output format. The qualitative examples are illustrative and the online A/B test, even with confidential numbers, indicates real deployment.\n\nThe soft spot is real: the reward in Eq. (1) and the offline metric AvgRel in Section 5.2 both come from the same internal GBDT relevance model. Since the policy is trained to maximize exactly this reward, the offline gains could in part reflect reward hacking—surface pattern matching that inflates GBDT scores without improving actual user relevance. The GBDT is trained on human labels, so it is not arbitrary, but the paper reports no calibration of the model on expanded queries and no analysis of reward hacking. The decision to report only the percentage of queries with any positive ΔRet/ΔRel, rather than the magnitude, makes it impossible to tell whether the gains are practically meaningful. The online A/B test is independent, but the lifts are confidential and no statistical protocol details are given, so outsiders can't verify the strongest evidence.\n\nThese issues are significant, but they don't sink the paper. The method is sensible, the experiments are substantial, and the online deployment is a credible signal. The citation pattern looks fair, and the writing is clear. This is a solid paper for the e-commerce search community, and it deserves a serious referee. I'd ask the authors for absolute metric values, human-rated evaluation of expanded queries, and as much detail on the A/B test as they can share. I would not desk-reject it.","headline":"A credible industrial GRPO-based query expansion method with a real circularity problem in offline evaluation; the A/B test is the only independent evidence and its numbers are confidential.","tokens_in":13132,"tokens_out":2849,"would_cite":true,"duration_ms":24572,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LESER claims that fine-tuning a context-aware LLM with Group Relative Policy Optimization on live search-engine feedback yields query expansions that substantially improve retrieval coverage, relevance, and online engagement in e-commerce…","keywords":["query expansion","e-commerce search","reinforcement learning","GRPO","large language models","pseudo-relevance feedback","retrieval coverage","online A/B testing"],"falsifier":"Re-run the offline evaluation with human relevance judgments in place of the GBDT model, scoring the newly retrieved items $R_{\\mathrm{exp}} \\setminus R_{\\mathrm{orig}}$; if LESER's preferred expansions do not also raise human-judged relevance, or if the GBDT and human rankings diverge sharply on the expanded result set, the central claim would be refuted.","tokens_in":12203,"feed_emoji":"🛒","tokens_out":6021,"duration_ms":50317,"temperature":0.7,"pith_summary":"LESER is a training framework for query expansion in e-commerce search. The paper claims that fine-tuning a context-aware LLM with Group Relative Policy Optimization, using real-time search-engine feedback as the reward, produces expansions that retrieve more relevant and more diverse products than prompt-based LLM expansion or supervised query rewriting. The method removes the need for labeled expansion data and grounds the model in the actual catalog through a pseudo-relevance-feedback context set. Offline results on 10,000 real queries show the 3B model improves retrieval coverage for 72.84% of queries and relevance for 32.40%, while a production A/B test reports statistically significant engagement gains. If the claim holds, this is an annotation-free route to query understanding that can scale to production traffic.","feed_headline":"Live search feedback sharpens LLM query expansion","feed_subtitle":"A 3B model expands vague queries into multiple intents, raising coverage and relevance offline and user engagement online.","key_machinery":"The load-bearing machinery is the reward function of Eq. (1), $r(\\mathcal{Y}) = r_{\\mathrm{rel}}(\\mathcal{Y}) + \\lambda \\cdot r_{\\mathrm{size}}(\\mathcal{Y})$ for valid outputs and $0$ otherwise, where $r_{\\mathrm{rel}}$ normalizes the average relevance of the top-$k$ combined retrieval by the original query's average relevance and $r_{\\mathrm{size}}$ normalizes the number of unique retrieved items by the original query's count. This reward turns the search engine into an oracle: the LLM samples $N$ expansion sets, each set is executed against the engine, and GRPO assigns relative advantages based on reward differences within the group. The validity gate enforces structural format, supported attributes, and length limits, while a supervised warm-up on distilled reasoning examples is required to keep the model emitting parseable `<think>`/`<answer>` JSON.","core_discovery":"The central claim is that query expansion for e-commerce search can be learned as a retrieval-optimization task: a context-aware LLM is fine-tuned with Group Relative Policy Optimization (GRPO) using a reward computed from live search-engine results, namely normalized relevance gain plus retrieval-coverage gain over the union of original and expanded queries. LESER grounds the LLM in a pseudo-relevance-feedback context set of top products with structured attributes, and a validity gate assigns zero reward to malformed output, keeping generations platform-compatible. On 10,000 real queries, the authors report that LESER with a 3B backbone yields positive retrieval gain for 72.84% of queries and positive relevance gain for 32.40%, exceeding embedding-based expansion, few-shot prompting of a proprietary LLM, and supervised fine-tuning baselines. A production A/B test on roughly 600K cached high-frequency queries showed statistically significant improvements in click-through rate, reduced query abandonment, and growth in gross merchandise value, with specific lift percentages kept confidential.","pith_inferences":["Editorial inference: the method's quality ceiling is set by the GBDT relevance model that provides the reward; if that model can be gamed by expansions that score well without reflecting true user relevance, the offline gains and the learned policy's objective would diverge from user satisfaction.","Editorial inference: the reward does not explicitly penalize overlap among the generated expansions, so the model could settle into near-duplicate queries that inflate the retrieval-size term without adding true diversity; a diversity-aware penalty would be a testable extension.","Editorial inference: the same search-engine-in-the-loop objective should transfer to other retrieval-side tasks such as attribute disambiguation, listing title rewriting, or query-to-filter prediction, wherever a catalog and a relevance model already exist.","Editorial inference: because the relevance model receives incremental updates over time, the reward distribution shifts, so LESER would likely need periodic re-training to stay aligned with the current relevance model."],"forward_implications":["Platforms with an existing relevance model can train expansion LLMs without human annotation, since the reward comes directly from live retrieval signals.","Because the reward is computed over the union of original plus expanded queries, the learned policy should favor expansions that add genuinely new relevant items rather than re-ranking the same result set.","The validity gate and warm-up stage suggest the approach transfers to other structured generation tasks where outputs must respect strict formats, such as facet extraction or filter inference.","With a 3B backbone fitting production latency budgets, similar RL-from-search-feedback training can be applied to higher-frequency or long-tail queries beyond the cached set.","The relative formulation, normalizing by the original query's own retrieval quality, makes the reward comparable across queries of very different specificity."],"supporting_citations":[{"why":"Supplies the GRPO objective used to optimize the LLM with group-relative advantages.","marker":"[27]"},{"why":"Defines the GBDT relevance model with human-judged five-point labels that scores inputs for the reward and the offline metric.","marker":"[26]"},{"why":"Classic relevance-feedback source that motivates the pseudo-relevance-feedback context-grounded expansion design.","marker":"[24]"},{"why":"Recent vector pseudo-relevance feedback that supports using retrieved context as grounding for LLM expansion.","marker":"[9]"},{"why":"Prompting-based LLM query expansion, the prior-art approach that LESER compares against and aims to improve.","marker":"[5]"},{"why":"Industrial supervised query rewriting with rejection sampling, which inspires the SFT baseline and the warm-up data construction.","marker":"[22]"},{"why":"Embedding-based semantic equivalence method used as a baseline for query expansion from search logs.","marker":"[11]"},{"why":"Contextualized embeddings for query expansion, the neural non-LLM baseline contrasted with LESER.","marker":"[15]"}],"fun_headline_variants":["LLM learns query expansion from live search feedback","Search-engine reward trains LLM to expand e-commerce queries","LESER: LLM query expansion via search feedback reward","Reinforcement from search results improves LLM query expansion","Live search signals guide LLM to expand vague queries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the internal GBDT relevance model gives a faithful proxy for what users actually find relevant; if expansions can inflate that model's scores without improving real relevance, the offline gains and the RL objective are both misaligned with user satisfaction.","fun_headline_variants_meta":{"raw":{"variants":["LLM learns query expansion from live search feedback","Search-engine reward trains LLM to expand e-commerce queries","LESER: LLM query expansion via search feedback reward","Reinforcement from search results improves LLM query expansion","Live search signals guide LLM to expand vague queries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000276,"raw_usage":{"total_tokens":1663,"prompt_tokens":976,"completion_tokens":687,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":609}},"tokens_in":592,"tokens_out":687,"duration_ms":6025,"temperature":1.0,"reasoning_tokens":609,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:22:18.014227+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the offline evaluation with human relevance judgments in place of the GBDT model, scoring the newly retrieved items $R_{\\mathrm{exp}} \\setminus R_{\\mathrm{orig}}$; if LESER's preferred expansions do not also raise human-judged relevance, or if the GBDT and human rankings diverge sharply on the expanded result set, the central claim would be refuted.","supporting_citations":[{"cited_title":"Improving Ad matching via Cluster-Adaptive Keyword Expansion and Relevance tuning","cited_arxiv_id":"2505.18897","evidence_quote":"Defines the GBDT relevance model with human-judged five-point labels that scores inputs for the reward and the offline metric."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Classic relevance-feedback source that motivates the pseudo-relevance-feedback context-grounded expansion design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contextualized embeddings for query expansion, the neural non-LLM baseline contrasted with LESER."}],"review_version":2}