{"id":"38c9913f-e458-4da5-afd8-e7f72d10e26a","arxiv_id":"2607.07820","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A verifiable offline Wikipedia tool environment plus iterative scaffold-to-ReAct self-distillation lets a 9B agent reach competitive BrowseComp/GAIA/HotpotQA scores without stronger-model distillation.","lead":"The paper builds a deterministic offline Wikipedia search environment and a self-distillation loop so a 9B web agent improves from its own verified tool-use trajectories. It shows competitive open-source deep-search scores without trajectories from stronger proprietary models.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The competitive claim rests on a small live-tool GRPO stage whose contribution is not isolated from the offline self-evolution loop.","rationale":"The reader’s weakest assumption correctly identifies transfer from Wikipedia BM25/SQLite to live web tools and the authors’ own GRPO mitigation. I agree that is the soft spot, but the sharper load-bearing failure mode is attribution: the published competitive scores are post-GRPO, while the paper’s conceptual contribution is offline self-distillation without stronger teachers. If the pre-GRPO model already matches Table 1, the central claim holds and the GRPO stage is only a small domain adapter; if it does not, the narrative over-credits the verifiable environment. That single missing checkpoint comparison would settle the issue without requiring full re-training. Reproducibility promises and Wikipedia coverage limits remain secondary. Verdict stays CONDITIONAL for the same release-and-transfer reasons the reader gave; the concrete test would either firm acceptance of the self-evolution story or force a more modest claim about hybrid offline SFT + light live RL.","tokens_in":25129,"tokens_out":604,"duration_ms":7123,"concrete_test":"Evaluate the checkpoint immediately after the 11 offline evolving-SFT rounds (before GRPO) on BrowseComp, GAIA, and HotpotQA under the same SerpAPI/Jina tool stack used for the final model and for open-source baselines. If BrowseComp falls by more than ~5–8 absolute points (or below the strongest open-source peers that also use live tools), the claim that offline verifiable self-evolution alone drives competitive performance is not supported by the reported numbers.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim is that verifiable offline self-distillation alone yields competitive open-source deep-search agents without stronger-model distillation. Table 1 reports DeepSearch-World-9B at 31.2 BrowseComp / 61.5 GAIA / 93.4 HotpotQA after 11 evolving SFT rounds in DeepSearch-World plus a final GRPO stage on 1,600 real-tool (SerpAPI/Jina) instances (Sec. 4.1). The paper never reports the same model after only the offline loop, so the headline numbers may partly reflect conventional live-tool RL rather than the claimed offline verifiable self-evolution. The reader correctly flags the offline-to-real gap, but the load-bearing issue is more specific: without an ablation that freezes the model after evolving SFT and evaluates it under the same live tools used for baselines, one cannot attribute competitiveness to DeepSearch-World/DeepSearch-Evolve rather than the small GRPO patch. Ablations in Tables 2–3 and Fig. 5 are confined to SearchQA / DeepSearch-Val (offline), so they do not close this gap.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper introduces DeepSearch-World, a deterministic offline Wikipedia environment with BM25 search and SQLite visit tools plus 420K multi-hop QA tasks from entity-level random walks, and DeepSearch-Evolve, an iterative self-distillation loop that generates scaffold (Plan–Act–End) trajectories, filters them by answer correctness and quality, converts them to ReAct format via state internalization and reflection rewriting, and fine-tunes the student. Starting from Qwen3.5-9B-Instruct, DeepSearch-World-9B is trained for 11 evolving-SFT rounds and a final GRPO stage on 1,600 live-tool instances, reporting 31.2% BrowseComp, 61.5% GAIA, and 93.4% HotpotQA without distillation from stronger proprietary models. Ablations on SearchQA/DeepSearch-Val isolate rejection sampling, quality filtering, and scaffold-to-ReAct conversion; behavior analyses show longer horizons and higher advanced-capability scores.","tokens_in":25475,"tokens_out":848,"duration_ms":17812,"significance":"If the attribution holds, the work is a solid systems contribution to self-evolving tool-use agents: a reproducible, entity-verifiable offline environment that supplies process-level signals (progress verification, grounded reflection, failure recovery) and an evolving-SFT recipe that avoids sparse-reward RL instability and proprietary trajectory distillation. Planned release of the environment, 420K pool, validation set, model, and code is a genuine community asset. Large gains over the same backbone and competitive open-source numbers would support the thesis that verifiable environments can scale long-horizon web-agent self-improvement. The significance is tempered until live-tool competitiveness is cleanly attributed to the offline loop rather than the final GRPO patch.","major_comments":[{"comment":"Sec. 4.1 and Table 1: the headline claim that offline verifiable self-evolution alone yields competitive open-source deep-search agents is not isolated. DeepSearch-World-9B is evaluated after 11 evolving-SFT rounds plus GRPO on 1,600 SerpAPI/Jina instances, yet no checkpoint after offline-only training is reported under the same live tools used for baselines. Ablations (Tables 2–3, Fig. 5) stay on SearchQA/DeepSearch-Val. Without an offline-only live evaluation (and ideally a GRPO-only control from the backbone), competitiveness cannot be attributed to DeepSearch-World/DeepSearch-Evolve rather than conventional live-tool RL. This is load-bearing for the abstract’s central claim.","section":null},{"comment":"Sec. 3.1.3 and 3.2: teacher rollouts receive privileged environment-grounded reflection that can escalate to canonical entity names or fuzzy descriptions of unresolved walk entities. Student training rewrites these signals, but teacher trajectory quality—and thus the self-evolution loop—depends on process labels unavailable in open-web deployment. The paper should quantify how much performance depends on this privileged process supervision (e.g., teacher with answer-only filtering vs. entity-progress reflection) and clarify that “self-evolution” here is self-distillation under construction-time entity ground truth, not unsupervised discovery of intermediate progress.","section":null},{"comment":"Table 1 / Sec. 4.1–4.2: baseline comparison fairness is under-specified for the competitiveness claim. Open-source agents span 7B–9B scales, heterogeneous tool backends, and often stronger-model distillation; proprietary systems use different tool stacks. The paper notes some of this but does not report a controlled same-tools, same-budget evaluation of DeepSearch-World-9B vs. the strongest open baselines (e.g., MiroThinker, Marco-DR, WebExplorer). Without that, “competitive with open-source agents” remains only partially supported even if the GRPO confound is fixed.","section":null}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: they built a deterministic Wikipedia tool env with entity-level progress checks and a scaffold-to-ReAct self-distillation loop, then got a 9B model competitive with recent open deep-search agents without stronger-model trajectory distillation. That combination is the real product, not a new theory of self-evolution.\n\nWhat is new is the packaging. Entity-walk multi-hop tasks, BM25/SQLite offline search-visit tools, grounded reflection on unresolved entities, and converting Plan–Act–End scaffolds into ordinary ReAct traces for evolving SFT. The ablations on rejection sampling, quality filtering, state internalization, and reflection rewriting are clean enough. Behavior metrics (longer horizons, more visits, higher advanced-capability score) match the story. Promised release of env, 420K pool, val set, model, and code is the right posture if they actually ship it.\n\nThe soft spot that matters is attribution, not fraud. Table 1 numbers come after 11 offline rounds plus GRPO on 1,600 SerpAPI/Jina instances. They never report the post-SFT, pre-GRPO model on BrowseComp/GAIA under the same live tools. Offline ablations do not close that. So the abstract’s “verifiable environments enable scalable self-evolution” claim is a bit stronger than the live-web evidence. Wikipedia coverage and the offline-to-real gap are already noted by the authors; treat those as known limits, not surprises.\n\nMath is ordinary SFT/KL framing; citations are appropriate; circularity is low because answer correctness and entity hits are external. This is for people building tool agents who care about reproducible training loops. I would send it to peer review. Engage if you work on agent post-training or offline simulators; skim the GRPO section carefully before citing the headline scores as pure self-evolution.","headline":"Solid systems paper on offline-verifiable self-distillation for search agents; main live-web numbers still mix in a small GRPO patch that is not isolated.","tokens_in":26085,"tokens_out":488,"would_cite":true,"duration_ms":11332,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A verifiable offline search world lets web agents improve from their own tool traces without stronger teachers.","keywords":["self-distillation","web agents","tool use","multi-hop QA","verifiable environment","ReAct","evolving SFT","deep search"],"falsifier":"Retrain the same self-evolving loop with answer-correct trajectories only, without entity-level verification and without scaffold-to-ReAct conversion, and check whether BrowseComp and GAIA scores fall sharply; or evaluate the offline-only agent on live search tools without the small real-tool adaptation stage and measure the drop.","tokens_in":26052,"feed_emoji":"🔍","tokens_out":865,"duration_ms":17422,"temperature":0.7,"pith_summary":"Agents that search and read the web struggle to improve from their own experience: fixed imitation of good trajectories saturates, and sparse final rewards give little guidance over long multi-step interactions. This paper builds DeepSearch-World, a fully deterministic offline Wikipedia environment with reproducible search and page-visit tools, entity-level progress checks, and grounded reflection, plus 420,000 multi-hop questions from entity random walks. On top of it, DeepSearch-Evolve runs an iterative self-distillation loop: the current agent generates scaffolded trajectories that track plans, memory, failures, and recovery; answer-correct traces are quality-filtered, converted into standard ReAct supervision, mixed across rounds, and used for supervised fine-tuning. A 9B model trained this way, without distillation from more capable models, reaches competitive open-source scores on deep-search benchmarks. The claim is that process-verifiable environments supply the dense, reliable supervision long-horizon tool agents need to self-evolve at scale.","feed_headline":"Web agents self-improve without stronger teachers","feed_subtitle":"A 9B model trained only on its own verified Wikipedia tool traces hits competitive deep-search scores.","key_machinery":"DeepSearch-World and DeepSearch-Evolve: an offline BM25/SQLite Wikipedia tool pair with order-free entity progress verification and staged grounded reflection, feeding an evolving supervised loop that rewrites scaffolded teacher traces (plan, memory, failure recovery) into deployable ReAct student data.","core_discovery":"Deterministic, entity-verifiable offline search-and-read tools plus scaffold-to-ReAct self-distillation are enough for a 9B agent to climb to competitive deep-search performance from its own filtered rollouts, without trajectories from stronger models.","pith_inferences":["The same entity-walk construction and progress checks could be ported to other structured corpora beyond encyclopedias.","Once process labels are reliable, soft on-policy distillation may become more stable than it is in noisy live tool settings.","Competitive deep-search numbers from pure self-experience imply proprietary trajectory distillation is less necessary for search agents than is often assumed.","Failure-recovery behaviors induced by staged reflection may be the main transferable skill when the agent leaves Wikipedia."],"forward_implications":["Open-source search agents can close much of the tool-use gap by iterating in verifiable sandboxes rather than copying proprietary traces.","Entity-level process verification can substitute for sparse outcome rewards when training long-horizon search agents.","Scaffold planning and reflection can be stripped before training and still leave memory tracking and self-correction inside a standard ReAct policy.","Larger multi-hop task pools sustain gains across more self-evolving rounds by covering more failure and recovery patterns.","Schema-aligned offline tools can later be swapped for live search and visit APIs with limited extra adaptation."],"fun_headline_variants":["9B web agent self-evolves on verified tool traces alone","Self-distillation trains competitive deep-search agents without teachers","Verifiable offline tools let 9B agents climb long-horizon search scores","Agents reach open-source deep-search levels from filtered self-rollouts","Deterministic search env enables scaffold-to-ReAct self-improvement"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Skills learned under entity checkmarks inside a closed Wikipedia simulator will transfer to messy live web tools and non-encyclopedia domains.","fun_headline_variants_meta":{"raw":{"variants":["9B web agent self-evolves on verified tool traces alone","Self-distillation trains competitive deep-search agents without teachers","Verifiable offline tools let 9B agents climb long-horizon search scores","Agents reach open-source deep-search levels from filtered self-rollouts","Deterministic search env enables scaffold-to-ReAct self-improvement"]},"model":"grok-4.5","effort":"low","cost_usd":0.003364,"raw_usage":{"total_tokens":1128,"prompt_tokens":755,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":33640000,"prompt_tokens_details":{"text_tokens":755,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":295,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":755,"tokens_out":78,"duration_ms":3012,"temperature":1.0,"reasoning_tokens":295,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T15:40:13.644716+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Retrain the same self-evolving loop with answer-correct trajectories only, without entity-level verification and without scaffold-to-ReAct conversion, and check whether BrowseComp and GAIA scores fall sharply; or evaluate the offline-only agent on live search tools without the small real-tool adaptation stage and measure the drop.","supporting_citations":[],"review_version":2}