Pith. sign in

REVIEW 5 major objections 3 minor 1 cited by

A verification-aware agent harness that recycles failure diagnostics as search signals can evolve production recommender architectures with a 60% effective pass rate and positive online business impact, the paper claims.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 09:59 UTC pith:6KVV3ZHX

load-bearing objection A credible industrial agent-harness paper with a genuinely new assembly, but the headline EPR margin over human experts depends on an under-specified budget and replay protocol. the 5 major comments →

arxiv 2606.27243 v3 pith:6KVV3ZHX submitted 2026-06-25 cs.IR cs.SE

NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems

Shaohua Liu , Liang Fang , Yilong Sun , Shudong Huang , Qingsong Luo , Shaoxin Liu , Xiaoyang Chen , Dongqiang Liu
show 11 more authors
Chuangang Ma Zhenzhen Chai Henghuan Wang Shijie Quan Changyuan Cui Zhangbin Zhu Peng Chen Wei Xu Lei Xiao Haijie Gu Jie Jiang
This is my paper
classification cs.IR cs.SE
keywords recommender systemsarchitecture evolutionLLM agentsmodel verificationsilent failuresarchitecture gradientindustrial advertisingproduction A/B test
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Industrial recommender models improve through architecture changes—topology, feature configuration, interaction modules—but each change must respect interface, resource, and serving constraints, and code that compiles can still be architecturally invalid. The paper presents NOVA, an agent harness that turns architecture evolution into a multi-round search over concrete implementations, where a verification cascade rejects semantically invalid candidates before expensive training and records their failure patterns as 'forbidden directions.' These diagnostics, together with previous modifications and offline AUC changes, are aggregated into an 'architecture gradient' that selects the next modification. Under a fixed offline-evaluation budget, NOVA reports effective pass rates of 54.5% (L2 ScaleUp) and 60.0% (L3 Literature-to-Production), versus 49.3% and 31.1% for replayed human-expert logs, and a production A/B test of the chosen L3 candidate shows GMV gains of +1.25%, +1.70%, and +2.02% on three pCVR objectives along with pCVR bias reductions of 58.8%, 66.7%, and 37.3%. The paper's central claim is that verification-aware feedback, not merely runnable-code generation, is what makes agent-driven architecture evolution work in production.

Core claim

On the paper's own terms, the core discovery is that silent failures—candidates that run without errors but violate recommender-specific architecture semantics or fail to improve offline metrics—are the main barrier to automating architecture evolution, and that reusing their diagnostics as forbidden directions turns verification from a post-hoc filter into a component of the search signal. The architecture gradient encodes this: it aggregates the previous modification, verification diagnostics, metric changes, and trajectory memory into weak components, candidate directions, and forbidden patterns. Empirically, NOVA achieves local pass rates of 99.0% (L2) and 86.7% (L3) and effective pass r

What carries the argument

The central machinery is the architecture gradient, g_t = Grad(e_{t-1}, V_t, ΔJ_t, H_t), an SGD-inspired update signal that converts the previous modification, verification diagnostics, offline metric change (ΔAUC), and trajectory memory into three kinds of guidance: weak components, modification directions, and forbidden directions. It is supported by a verification cascade—a structure-semantic gate that checks masking direction, feature routing, logit fusion, shapes, and dtypes before training; a local executability gate; offline AUC evaluation; and online GMV/Bias validation. Rejected candidates write their failure patterns into trajectory memory as forbidden directions, so verification b

Load-bearing premise

The load-bearing premise is that the evaluation protocol is neutral: the 'same offline budget' is genuinely equal across methods, and the human-expert baseline, replayed from historical logs, faithfully represents what senior engineers would achieve if they could iterate live with the same verification feedback.

What would settle it

Re-run the L3 task with a live human-expert loop (no replay) under the same budget, verification cascade, and trajectory-memory feedback; if human EPR reaches or exceeds 60%, the headline superiority claim fails. Separately, sweep the AUC-positive threshold around 0.001; if the EPR ranking across methods flips or the online GMV gains cease to be significant, the reported comparisons are measurement artifacts.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, recommender R&D can shift from expert-intensive manual iteration to budgeted, auditable agent search, with human time reserved for high-risk decisions.
  • Forbidden directions accumulate across tasks, so later architecture searches inherit the failure patterns learned from earlier ones rather than repeating them.
  • The offline-to-online consistency (offline AUC-best candidate improves GMV while reducing bias) supports using AUC as the inner-loop selection metric in similar systems.
  • Level-aware routing with AutoRun/Copilot modes offers a deployable risk-control template: low-risk changes run autonomously, uncovered or high-risk changes require human confirmation.
  • The reported 13× reduction in human-attended time per literature-to-production cycle implies a step-change in experimental throughput at fixed headcount.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The architecture-gradient pattern—verification diagnostics as a search signal, not just a gate—may transfer to other constrained code-evolution domains (compiler passes, hardware design, ETL pipelines) where 'runnable' is a weak correctness criterion.
  • A decisive test not run by the paper: give human experts the same verification cascade and forbidden-direction memory live, under the same budget; if their EPR closes the gap to NOVA's 60%, the harness protocol rather than the gradient signal carries the advantage.
  • Because the semantic gate's 'validity' is defined by self-authored skill specifications, an audit comparing the gate's rejections with independent senior-engineer judgments would reveal how much of the EPR gain is gate conservatism versus genuine search improvement.
  • The ΔAUC>0.001 positivity threshold defines the headline EPR; reporting EPR at several thresholds would show whether the ranking across methods is robust or an artifact of a single cutoff.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 3 minor

Summary. The paper introduces NOVA, a verification-aware multi-agent harness for automated architecture evolution in an industrial advertising recommender system. NOVA iteratively proposes candidate modifications to a production model, filters them through a multi-stage verification cascade (structure-semantic, local executability, offline AUC, online GMV/Bias), and uses an 'architecture gradient' — an aggregate of prior modifications, verification diagnostics, metric deltas, and trajectory memory — to steer subsequent search. The authors report offline experiments on L2 ScaleUp and L3 Literature-to-Production tasks, claiming NOVA achieves effective pass rates (EPR) of 54.5% and 60.0%, versus 49.3% and 31.1% for a replayable Human Expert Loop, and outperform generic coding agents and Optuna-TPE. A production A/B test on 5% of traffic shows the offline-selected L3 candidate improves GMV by +1.25%/+1.70%/+2.02% on three pCVR objectives with pCVR bias reductions of 58.8%/66.7%/37.3%.

Significance. If the results hold, this is a significant industrial demonstration: LLM-agent-driven architecture evolution with pre-training semantic verification appears to outperform replayable human expert loops and generic coding agents under a fixed budget, and the online A/B gains ground the central claim in live business metrics. The paper's strengths include real training runs rather than fitted surrogate curves, a production A/B test with reported significant GMV gains, and unusually concrete reproducibility artifacts in the appendix (prompt skeleton, trajectory snippet, skill summaries) that let the reader inspect the harness mechanisms. The main technical risk is whether the offline comparison protocol is measurement-neutral across methods; this is where the manuscript currently needs the most work.

major comments (5)
  1. [Abstract vs. §5.4, Table 5] The abstract reports EPR of 53.3% (ScaleUp) and 51.7% (Literature-to-Production), while Table 5 and the body abstract report 54.5% and 60.0%. Since EPR is the paper's headline metric, this internal contradiction is load-bearing and must be resolved. Please state which numbers are final and ensure the abstract matches the experimental tables.
  2. [§5.2, Budget and protocol] The claim 'All methods are evaluated under the same offline budget' cannot be verified because budget B is never quantified. §3 defines B as 'rounds, LLM tokens, GPU hours, and wall-clock time,' but §5.2 only reports N_task, N_iter, and K. Without per-method accounting in at least one common unit (e.g., GPU-hours, number of offline training runs, or token counts), the headline L3 EPR gap (60.0% vs. 31.1%) could be an artifact of differential resource consumption rather than harness design. Please provide concrete resource budgets and show each method operates within them.
  3. [§5.3, Human Expert Loop replay] The Human Expert Loop is defined as historical logs 'replayed under our unified evaluation protocol,' but the replay protocol is not described. If human debugging and edit attempts are counted as separate generated candidates N_g while NOVA's N_g is capped at K per iteration, human LPR and EPR are mechanically depressed. The human L3 LPR of 40.0% with a low SFR of 22.2% is consistent with an inflated N_g. Please specify the replay mapping and report per-task N_g, N_p, and N_+ counts for every method, or the human comparison is not interpretable.
  4. [§5.2, Table 5; §5.5, Table 6] All LPR/SFR/EPR values are aggregate rates over N_task=10 tasks, with no confidence intervals, exact counts, or per-task variability. Candidates within a task are not independent samples, so two-decimal precision (e.g., 54.5%) is spurious without raw counts or bootstrapped intervals. Add the underlying counts (N_g, N_p, N_+) or per-task distributions to support the claimed differences, especially for the ablation comparisons in Table 6.
  5. [§4.4, §5.2, Table 5] Semantic validity is determined by skill specifications that are authored and maintained as part of NOVA itself; these same decisions define LPR/SFR. Comparisons against ReActAgent-only and OpenHands therefore partly measure the gate's self-consistent notion of 'architecture validity' rather than an externally anchored property. To make the silent-failure claim credible, provide at least one external adjudication (e.g., senior-engineer review of a random sample of semantically rejected and passed candidates) showing the gate's validity labels agree with human expert judgment.
minor comments (3)
  1. [§3, Table 2] The 'architecture gradient' is explicitly analogical ('does not imply mathematical gradients'). Clarify which parts of Eqs. (5)–(7) and Algorithm 1 are actually implemented and which are intended as conceptual framing; as written, the formalization risks being read as stronger than the implementation.
  2. [§5.7, Table 7] The paper states GMV gains are 'significant under the platform's standard significance test' but gives no p-values, confidence intervals, or test details. Please report the test statistic or a confidence interval for each objective.
  3. [Appendix A.2, Table 9] The abstract says 'over 13× in human-attended time' and the appendix reports 13.5×. Use consistent rounding and clarify whether the 2.2× wall-clock reduction is relevant to the core claims or merely context.

Circularity Check

0 steps flagged

No significant circularity: the paper's headline EPR, ablation, and online-GMV claims are measured from real training/A-B outcomes, not derived from fitted inputs.

full rationale

The paper's core claims are empirical: effective pass rate is measured from actual generated/trained candidates (EPR = N+/Ng = LPR·(1−SFR), Eqs. 10–12), and the online GMV/bias improvements come from a live 5%-traffic A/B test (Table 7). These are measurement definitions and experimental results, not predictions derived from fitted parameters. The architecture-gradient loop (Eqs. 5–7, Algorithm 1) is an iterative feedback procedure whose components are tested by ablations (Table 6); no step claims a mathematical derivation from a prior theorem. The semantic gate's skill specifications define what counts as an architecture-semantic violation, but the downstream evaluation of silent failures uses offline AUC and online GMV, so the gate is not used as its own success metric. The paper contains no load-bearing self-citation or imported uniqueness theorem. The unquantified 'same offline budget' (B is never enumerated) and the replay-based Human Expert Loop baseline raise legitimate measurement-fairness and reproducibility questions, but those are not circularity: they do not make any claimed result equal to its own input by construction. Accordingly, no circular steps are identified.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 2 invented entities

The paper's load-bearing postulates are measurement choices, not mathematical axioms: a ΔAUC>0.001 threshold converts AUC to pass/fail; an unquantified 'same budget' and author-designed human-log replay define the comparisons; in-house skill specifications define 'architecture validity'; a single fixed LLM is assumed neutral. No invented physical entities — the 'architecture gradient' is an acknowledged analogy. The central empirical claims rest on these assumptions plus unverifiable proprietary A/B infrastructure.

free parameters (7)
  • AUC-positive threshold (ΔAUC > 0.001) = 0.001 ΔAUC
    Defined in §5.2; converts continuous ΔAUC into the binary AUC-positive label that drives N+, SFR, and EPR. The choice of threshold directly changes every headline rate; no sensitivity analysis is given.
  • Candidate count K per round = not specified
    Algorithm 1 and §5.2 say 'up to K candidates' but K is never given. EPR = N+/Ng depends on how many generated candidates are counted.
  • Iteration/task budget (N_iter=10, N_task=10) = 10 tasks × 10 iterations
    §5.2 protocol. Every EPR/LPR/SFR number rests on 10 independent tasks with no variance reporting; per-candidate counts are not independent samples.
  • Offline evaluation budget B = not quantified
    §5.2 states all methods run 'under the same offline budget' but B (GPU hours, wall-clock) is never quantified, so the claimed budget-equality across methods is unverifiable.
  • Early-stop AUC improvement threshold = not specified
    Table 4 defines Converged(·) as 'a predefined offline AUC improvement threshold' without giving the value; it controls how many rounds each task runs.
  • Base LLM (Claude Sonnet 4.6) = Claude Sonnet 4.6
    §5.2 fixes a single base model for all LLM methods. All results are conditional on this model's code-generation and review ability; no cross-model test is reported.
  • Online business weights w_i = not specified
    Eq. (4) defines J_online as a weighted sum of GMV/Bias, but the weights are never given, so the online objective is not fully specified.
axioms (6)
  • domain assumption AUC is a sufficient offline proxy for online business quality (Eq. 3 uses AUC as the sole offline objective; bias is only checked online)
    §3 Eq. (3) and §5.7. The entire offline selection pipeline (and every EPR number) treats ΔAUC > 0.001 as 'effective'. If AUC gains do not track GMV, the offline pass rates lose meaning; the paper validates the link for only one selected candidate.
  • domain assumption The skill specifications behind the semantic gate correctly define architecture validity (mask direction, logit fusion, feature-to-token mapping)
    §4.4: the semantic gate 'checks these constraints according to skill specifications' that are seeded and maintained by the same team reporting the results; coverage is self-assessed ('avoids overconfident rejection'). No external validation of the rules.
  • domain assumption The replayed Human Expert Loop logs faithfully represent expert performance under the unified protocol
    §5.3: human performance is 'historical iteration logs ... replayed under our unified evaluation protocol'. The replay encoding (what counts as a candidate, how debug iterations map to Ng) is author-designed and can bias the headline 60.0% vs 31.1% EPR gap.
  • domain assumption The single base LLM (Claude Sonnet 4.6) contributes no method-specific advantage; differences reflect harness design
    §5.2 states all LLM methods share the base model. This is a control, not a proof: the harness includes prompt bundles, skills, and gates, so results remain conditional on this one model.
  • domain assumption Every candidate from every method was actually trained from scratch on the identical month-long, billion-scale dataset
    §5.2: 'Every candidate model produced by every method is trained from scratch on the same dataset.' This is a large computational claim (methods × 10 tasks × ≤10 rounds × K candidates) with no compute budget disclosed; if candidates were filtered asymmetrically before training, EPR comparisons shift.
  • domain assumption The production A/B significance test is valid and the 5% traffic split is unbiased
    §5.7: 'All reported GMV gains are significant under the platform's standard significance test' — with no test statistic, p-value, confidence interval, or experiment duration. Bias reductions are reported without any uncertainty.
invented entities (2)
  • Architecture gradient g_t no independent evidence
    purpose: LLM-constructed search-direction signal aggregating previous modification, verification diagnostics, metric feedback, and trajectory memory (Eq. 5)
    The paper explicitly disclaims a mathematical gradient ('the comparison is only an analogy', §3). The only evidence it works is the in-paper ablation (EPR 60.0% → 37.5% without it). No out-of-paper falsifiable handle.
  • Forbidden directions / trajectory memory H no independent evidence
    purpose: Memory of failed, semantically invalid, or repeatedly negative modifications that blocks future proposals (Algorithm 1, §4.4)
    An engineering construct whose benefit is supported only by the in-paper ablation and a single, apparently crafted trajectory snippet (Appendix A.3.2) with no statistics attached.

pith-pipeline@v1.3.0-alltime-deepseek · 16087 in / 24457 out tokens · 230656 ms · 2026-08-02T09:59:33.477607+00:00 · methodology

0 comments
read the original abstract

Industrial advertising recommender systems are continually improved through architecture modifications, yet production iteration remains expert-intensive because coordinated changes to model topology, feature configuration, and interaction modules must satisfy strict interface, resource, and serving constraints. AutoML is limited to predefined search spaces, while generic coding agents verify runnability rather than recommender-specific semantic validity. Executable candidates may therefore violate architectural contracts, while the lack of structured reuse of semantic diagnostics and evaluation outcomes can lead to repeated invalid or ineffective modifications. We present NOVA, a verification-aware agent harness that organizes production architecture modification as multi-round search over concrete implementations within a fixed evaluation budget. At each round, NOVA generates multiple candidates under production constraints, rejects semantic violations, and ranks the valid survivors for local testing and offline evaluation. Across rounds, trajectory memory synthesizes semantic diagnostics, local-test outcomes, and offline metric changes into modification directions and forbidden patterns that guide subsequent search. Under the same maximum offline-evaluation budget for automated methods, NOVA achieves the highest effective pass rate, reaching 53.3% on ScaleUp and 51.7% on Literature-to-Production tasks. In a production A/B test covering 5% of traffic in an advertising system serving over one billion users, the selected Literature-to-Production candidate yields GMV gains of +1.25%, +1.70%, and +2.02% across three major pCVR objectives, with corresponding relative reductions in absolute pCVR bias of 58.8%, 66.7%, and 37.3%, respectively.

Figures

Figures reproduced from arXiv: 2606.27243 by Changyuan Cui, Chuangang Ma, Dongqiang Liu, Haijie Gu, Henghuan Wang, Jie Jiang, Lei Xiao, Liang Fang, Peng Chen, Qingsong Luo, Shaohua Liu, Shaoxin Liu, Shijie Quan, Shudong Huang, Wei Xu, Xiaoyang Chen, Yilong Sun, Zhangbin Zhu, Zhenzhen Chai.

Figure 1
Figure 1. Figure 1: Overview of the NOVA level-aware architecture-gradient workflow. The Main Agent fixes the task level and execution [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Silent-failure-aware verification cascade. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Future evolution roadmap of NOVA across full [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Review history (3 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems

    cs.IR 2026-07 conditional novelty 5.0

    A bandit router allocates trials to human-defined edit directions while an LLM writes code within that direction, producing more stable recommender improvements and a positive online A/B result.

Reference graph

Works this paper leans on

40 extracted references · 5 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Frame- work. InProceedings of the 25th ACM SIGKDD International Conference on Knowl- edge Discovery and Data Mining. 2623–2631

    2019

  2. [2]

    Anthropic. 2026. Claude Sonnet 4.6. https://www.anthropic.com/claude/sonnet. Accessed: 2026-06-08

    2026

  3. [3]

    James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for Hyper-Parameter Optimization. InAdvances in Neural Information Processing Systems (NeurIPS), Vol. 24

    2011

  4. [4]

    Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, and Hemal Shah

  5. [5]

    Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and Robust Automated Machine Learning. InAdvances in Neural Information Processing Systems

    2015

  6. [6]

    Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017. DeepFM: A Factorization-Machine Based Neural Network for CTR Prediction. InProceedings of the 26th International Joint Conference on Artificial Intelligence. 1725–1731

    2017

  7. [7]

    Xu Huang, Hao Zhang, Zhifang Fan, Yunwen Huang, Zhuoxing Wei, Zheng Chai, Jinan Ni, Yuchao Zheng, and Qiwei Chen. 2026. MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders.arXiv preprint arXiv:2602.14110 (2026)

    Pith/arXiv arXiv 2026

  8. [8]

    Yunwen Huang, Shiyong Hong, Xijun Xiao, Jinqiu Jin, Xuanyuan Luo, Zhe Wang, Zheng Chai, Shikang Wu, Yuchao Zheng, and Jingjian Lin. 2026. HyFormer: Revisiting the Roles of Sequence Modeling and Feature Interaction in CTR 9 Prediction.arXiv preprint arXiv:2601.12681(2026)

    arXiv 2026

  9. [9]

    Yuchen Jiang, Jie Zhu, Xintian Han, Hui Lu, Kunmin Bai, Mingyu Yang, Shikang Wu, Ruihao Zhang, Wenlin Zhao, Shipeng Bai, et al. 2026. TokenMixer-Large: Scaling Up Large Ranking Models in Industrial Recommenders.arXiv preprint arXiv:2602.06563(2026)

    arXiv 2026

  10. [10]

    Yuchin Juan, Yong Zhuang, Wei-Sheng Chin, and Chih-Jen Lin. 2016. Field- aware Factorization Machines for CTR Prediction. InProceedings of the 10th ACM Conference on Recommender Systems. 43–50

    2016

  11. [11]

    Ashwin Kumar, Erwin Gao, Matan Levi, Sheela Yadawad, Sherman Wong, Sneha Iyer, and Vinodh Kumar Sunkara. 2026. Ranking Engineer Agent (REA): The Autonomous AI Agent Accelerating Meta’s Ads Ranking Innovation. Meta Engineering Blog. https://engineering.fb.com/2026/03/17/developer- tools/ranking-engineer-agent-rea-autonomous-ai-system-accelerating-meta- ads...

    2026

  12. [12]

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2019. DARTS: Differentiable Architecture Search. InInternational Conference on Learning Representations

    2019

  13. [13]

    H. Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, Sharat Chikkerur, Dan Liu, Martin Wattenberg, Arnar Mar Hrafnkelsson, Tom Boulos, and Jeremy Kubica. 2013. Ad Click Prediction: A View from the Trenches. In Proceedings of the 19th ACM SIGKDD International Confe...

    2013

  14. [14]

    Le, and Jeff Dean

    Hieu Pham, Melody Guan, Barret Zoph, Quoc V. Le, and Jeff Dean. 2018. Efficient Neural Architecture Search via Parameter Sharing. InProceedings of the 35th International Conference on Machine Learning. 4095–4104

    2018

  15. [15]

    Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2020. Search-based Interest Model for Lifelong User Behavior Sequence Modeling in Click-Through Rate Prediction. InProceedings of the 29th ACM International Conference on Information and Knowledge Management. 2685–2692

    2020

  16. [16]

    Reid Pryzant, Dan Iter, Jerry Li, Yin Lee, Chenguang Zhu, and Michael Zeng

  17. [17]

    Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le. 2019. Regularized Evolution for Image Classifier Architecture Search. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 4780–4789

    2019

  18. [18]

    Steffen Rendle. 2010. Factorization Machines. InProceedings of the 2010 IEEE International Conference on Data Mining. 995–1000

    2010

  19. [19]

    Kimi Team, Guangyu Chen, Yu Zhang, Jianlin Su, Weixin Xu, Siyuan Pan, Yaoyu Wang, Yucheng Wang, Guanduo Chen, Bohong Yin, et al . 2026. Attention residuals.arXiv preprint arXiv:2603.15031(2026)

    Pith/arXiv arXiv 2026

  20. [20]

    Hoos, and Kevin Leyton-Brown

    Chris Thornton, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. Auto-WEKA: Combined Selection and Hyperparameter Optimization of Clas- sification Algorithms. InProceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 847–855

    2013

  21. [21]

    Haochen Wang, Yi Wu, Daryl Chang, Li Wei, and Lukasz Heldt. 2026. Self- evolving recommendation system: End-to-end autonomous model optimization with LLM agents.arXiv preprint arXiv:2602.10226(2026)

    Pith/arXiv arXiv 2026

  22. [22]

    Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & Cross Network for Ad Click Predictions. InProceedings of the ADKDD’17. 1–7

    2017

  23. [23]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Gra- ham Neubig. 2025. OpenHands: An Open Platform for...

    2025

  24. [24]

    Xidong Wu, Yue Zhuan, Ruoqiao Wei, Hangxin Chen, Di Bai, Jintao Liu, Xinyi Wang, Xue Wang, Luoshu Wang, and Xinwu Cheng. 2026. AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimiza- tion.arXiv preprint arXiv:2604.26969(2026)

    Pith/arXiv arXiv 2026

  25. [25]

    Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-Computer Inter- faces Enable Automated Software Engineering. InAdvances in Neural Information Processing Systems, Vol. 37

    2024

  26. [26]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. InThe Eleventh International Conference on Learning Representations (ICLR)

    2023

  27. [27]

    Differentiation

    Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. 2024. TextGrad: Automatic “Differentiation” via Text. https://doi.org/10.48550/arXiv.2406.07496 arXiv:2406.07496 [cs.CL]

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.07496 2024

  28. [28]

    Zhaoqi Zhang, Haolei Pei, Jun Guo, Tianyu Wang, Yufei Feng, Hui Sun, Shaowei Liu, and Aixin Sun. 2026. Onetrans: Unified feature interaction and sequence modeling with one transformer in industrial recommender. InProceedings of the ACM Web Conference 2026. 8162–8170

    2026

  29. [29]

    Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep Interest Evolution Network for Click-Through Rate Prediction. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5941–5948

    2019

  30. [30]

    Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep Interest Network for Click- Through Rate Prediction. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1059–1068

    2018

  31. [31]

    Size” is the prompt + skill bundle size; “Input

    Jie Zhu, Zhifang Fan, Xiaoxie Zhu, Yuchen Jiang, Hangyu Wang, Xintian Han, Haoran Ding, Xinmin Wang, Wenlin Zhao, Zhen Gong, et al. 2025. Rankmixer: Scaling up ranking models in industrial recommenders. InProceedings of the 34th ACM International Conference on Information and Knowledge Management. 6309–6316. 10 A Appendix: Harness Footprint, Efficiency, a...

    2025

  32. [34]

    Parse the full paper before any local implementation; extract architecture, equations, tensor shapes, and dependencies

  33. [35]

    Separate paper-stated facts from inferences and engineering assumptions; log unresolved ambiguities explicitly

  34. [36]

    Generate faithful code, tests, runnable examples, and audit artifacts under paper_repro/. ... # REPRESENTATIVE GUARDRAILS - Never code directly from vague intuition. - Every major implementation choice MUST be tagged as {paper-stated|inferred-from-paper|engineering-assumption}. ... # OUTPUTS spec.md, equation_map.md, ambiguity_log.md, src/model.py, tests/...

  35. [37]

    Retrieve the correct context/topo/ and context/scene/ files BEFORE proposing any modification

  36. [38]

    # REPRESENTATIVE GUARDRAILS - Read topology and scene grounding files first

    Select one optimization direction using priority matrices plus failure history from prior rounds... # REPRESENTATIVE GUARDRAILS - Read topology and scene grounding files first. - Reject changes that violate latency budget, exported-graph schema, or production deployment constraints. ... # OUTPUTS A ranked design.md containing records of the form (explanat...

  37. [39]

    Build unified diffs from each candidate to the baseline rather than reviewing raw code in isolation

  38. [40]

    # REPRESENTATIVE GUARDRAILS - Every finding MUST cite line ranges

    Launch heterogeneous LLM reviewers in parallel and reconcile their findings by location and severity... # REPRESENTATIVE GUARDRAILS - Every finding MUST cite line ranges. - Unresolved block-level findings MUST be fixed or explicitly waived before training. ... # OUTPUTS - Per-reviewer reports - Consolidated summary.md - gate_decision∈{pass, revise, reject...

  39. [2016]

    InProceedings of the 1st Workshop on Deep Learning for Recommender Systems

    Wide & Deep Learning for Recommender Systems. InProceedings of the 1st Workshop on Deep Learning for Recommender Systems. 7–10

  40. [2023]

    Gradient Descent

    Automatic Prompt Optimization with “Gradient Descent” and Beam Search. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 7957–7968. https://doi. org/10.18653/v1/2023.emnlp-main.494

    doi:10.18653/v1/2023.emnlp-main.494 2023