{"id":"87e6aae7-7263-4d37-ab62-e5b03d378667","arxiv_id":"2608.05102","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Answer-backtracked clue recovery plus clue-anchored step scoring converts sparse pass/fail outcomes into dense per-step rewards that improve SFT and GRPO for search agents.","lead":"This paper trains a 4-billion-parameter search agent whose training signal marks every step as useful, neutral, or harmful, using clues recovered by backtracking from the correct answer. The resulting agent beats other 4-billion-parameter agents on browse-and-answer benchmarks and approaches the scores of several 30-billion-parameter systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Recovered clue set is unvalidated; if clue recovery is noisy, all step rewards misalign, so the core ABC claim needs a perturbation/human-check test.","rationale":"The central claim is that answer-backtracked step-level credit assignment improves long-horizon search-agent training. That claim would be true only if the step rewards rt in Eq. 4 are actually informative about step quality. The entire source of that informativeness is the recovered clue set C from Section 3.2. C is produced by a strong LLM prompted with the ground-truth answer; the paper does not report any agreement measure against external ground truth. The scorer in Appendix C.2 also sees the correct answer and awards credit for naming clues, making answer leakage a plausible alternative explanation for the positive ablation results. The Table 3 comparison to standard SFT/GRPO controls for trajectory-level supervision, but not for the quality of the clue set, because the same flawed C is used in both treatment arms. A human-verified clue set and a retraining run would isolate the mechanism. This is more load-bearing than the context-management confound or missing error bars because it targets the internal validity of the proposed credit-assignment signal itself. If the test shows the model's clue set is accurate and the retrained model retains gains, the central claim stands; otherwise the paper should be revised to a more modest claim conditional on clue quality. I agree with the reader's weakest-assumption identification and do not propose changing the CONDITIONAL verdict; the condition should explicitly be a validation of the recovered clue set. The released code and model weights are a genuine strength and would make the proposed retraining check feasible, but they do not by themselves establish that the clues are complete or correct.","tokens_in":11851,"tokens_out":5851,"duration_ms":69620,"concrete_test":"Run a controlled validation study on a random 100-question subset of the 8.5k training set: have independent human annotators, blind to the model's output, list the essential intermediate clues from query+answer, and compute precision/recall of the model's recovered C against the human-consensus clue set. Then rerun ABC-SFT and ABC-GRPO on this subset (or on the full set, if compute permits) replacing C with the human-consensus clues. If the reported gains over standard SFT/GRPO disappear or shrink materially, the unvalidated clue-recovery assumption is confirmed as load-bearing; if the gains persist with human-verified clues and human agreement is high, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's central mechanism is the recovered clue set C (Eq. 3). Section 3.2 builds C with a DeepSeek-V4-Flash ReAct loop prompted with the query and the verified answer, and it asserts that \"clues that survive this verification serve as reliable, answer-backtracked reference points.\" No precision/recall or human validation of C is reported. Section 3.3 then scores every step against C using another DeepSeek-V4-Flash prompt (Appendix C.2) that also includes the Correct Answer and awards +0.8 for \"Finds or verifies a correct clue\" and +1.0 for submitting the correct final answer (Table 1, Eq. 4). This creates a concrete failure mode: because the clue set is generated backward from the answer, any step that merely names answer-related entities can receive credit without genuinely discovering or verifying evidence; and if clue recovery omits or invents clues, the same scoring rubric will systematically reward or penalize the wrong behaviors. All downstream gains claimed for ABC-SFT (Eq. 5) and ABC-GRPO (Eqs. 6-7) are therefore contingent on C being complete, correct, and stable. The paper provides ablations of ABC vs standard SFT/GRPO (Table 3), but that comparison cannot separate genuine step-level credit from credit that is really just answer leakage through the scorer's access to the answer and the recovered clues. Since the same proprietary LLM family is used for clue recovery, step scoring, and final answer evaluation, an independent check of the clue set is the load-bearing gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Answer-Backtracked Credit Assignment (ABC) for training long-horizon search agents. Given a query and a verified answer, an LLM ReAct loop recovers a set of intermediate clues C; a second LLM scores every step in a trajectory against C (Eq. 4), producing dense step rewards that reweight SFT losses (Eq. 5) and replace trajectory-level rewards in GRPO (Eqs. 6-7). The authors train ABSeeker, a Qwen3.5-4B agent, on 8.5k OpenSeeker trajectories and report that it outperforms standard SFT/GRPO in ablations and is competitive with 30B-scale search agents on BrowseComp, BrowseComp-ZH, xbench, and GAIA-text.","tokens_in":12232,"tokens_out":4361,"duration_ms":46428,"significance":"ABC is a well-motivated and timely approach to process supervision for long-horizon search, and the idea of rewarding useful steps in failed trajectories while penalizing erroneous steps in successful ones is conceptually clean. The strengths include the explicit separation of clue recovery from step scoring, the open-sourced code and model, and ablation results that, taken at face value, support the directional claim that ABC-SFT and ABC-GRPO improve over uniform SFT and GRPO on most benchmarks. However, the central mechanism rests on an unvalidated clue set and an answer-aware scorer; the significance of the paper is therefore conditional on the additional validation and statistical rigor that the authors should provide.","major_comments":[{"comment":"The recovered clue set C is asserted to provide 'reliable, answer-backtracked reference points,' but no precision, recall, or human-validated quality measurement is reported for the clue recovery step. Because every step reward in Eq. (4) is computed against C, and because those rewards enter both the ABC-SFT weights in Eq. (5) and the ABC-GRPO rewards in Eqs. (6)-(7), any systematic omission or invention in C directly misaligns all downstream credit. Please add a human audit of a sample of recovered clue sets and a perturbation test (e.g., dropping, adding, or shuffling clues and measuring the change in step rewards and final accuracy) to demonstrate that the ABC signal is robust to realistic recovery errors.","section":"Section 3.2, Eq. (3), and Section 3.3, Eq. (4)"},{"comment":"The scoring prompt provides the scorer with the Correct Answer and the full Core Clues list, and criterion 5 awards +1.0 for submitting the correct final answer. This creates a concrete leakage path: a step that merely names entities from the answer or the provided clues can receive credit without genuinely discovering or verifying evidence. Because the same proprietary model family is used for clue recovery, step scoring, and final-answer evaluation, the dense signal may partly reflect the judge's preferences rather than intrinsic step quality. Please add a controlled experiment in which the scorer is blinded to the correct answer or to the clue labels, and compare the resulting reward assignments and downstream performance, to quantify the amount of answer leakage.","section":"Section 3.3 and Appendix C.2"},{"comment":"The core ablation results are reported as single numbers without error bars, confidence intervals, or significance tests. Some differences are small or in the wrong direction (e.g., xbench-2505: 72.0 for ABC-SFT vs 73.0 for standard SFT), while others are modest (BrowseComp: 30.8 vs 28.5). Because the paper claims that ABC 'consistently outperforms' standard SFT/GRPO, please report multiple seeds or per-evaluation variance and a significance test. Without this, the central claim is not fully supported by the evidence as presented.","section":"Table 3 and Section 4.2"},{"comment":"The headline comparison is confounded by context management. ABSeeker's 55.3% and 52.9% results on BrowseComp and BrowseComp-ZH are obtained with 256K context and the discard-all strategy, while several baselines are marked with an asterisk for no context management; it is unclear which baselines used the same context budget and which did not. The paper should explicitly tabulate the context-management setting for every baseline and compare all methods under identical inference budgets before claiming that ABSeeker outperforms same-scale agents or matches 30B agents.","section":"Table 2 and Section 4.2"}],"minor_comments":[{"comment":"The loss weight w(r_t) is defined in the main text as a sigmoid with parameters α and β, but Appendix A gives the concrete form w(r_t) = 2σ(2(r_t − 1)). Please reconcile these definitions and state the exact values of α and β used in the experiments.","section":"Section 3.4.1 and Appendix A"},{"comment":"Equation (2) defines the trajectory reward as exact string equality a = a*, while evaluation uses an LLM judge (Appendix C.3). Please clarify whether training rewards use exact matching, LLM-based matching, or both, since this affects the interpretation of the reward signal.","section":"Eq. (2) and Appendix C.3"},{"comment":"The frameworks 'Slime' and 'veRL' are mentioned without references or URLs; please add citations or links so that the training setup is reproducible.","section":"Appendix A and Section 4.1"},{"comment":"The caption of Figure 4 does not explain the x-axis labels (e.g., '0–10%' and '90–100%') clearly enough; please state explicitly that each bar represents one-tenth of the steps within a trajectory, ordered temporally.","section":"Section 4.2, Figure 4"},{"comment":"The DeepSeek-V4-Flash model is used for clue recovery, step scoring, and final-answer evaluation, but no version or snapshot date is given. Because proprietary models change over time, please specify the exact version and date used for all reported results.","section":"Section 4.1 and Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper is close to publishable, but the missing validation of the recovered clue set is a central, not cosmetic, gap: all dense rewards are built on C, and the answer-aware scoring prompt makes leakage a realistic risk. I would encourage the editor to require a human audit or perturbation analysis of the clue recovery, as well as error bars for the main ablations, before acceptance. The comparison with 30B agents is less important than the internal validity of the ABC-SFT/ABC-GRPO gains."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline: ABSeeker proposes a genuinely new way to assign step-level credit in long-horizon search-agent training—answer-backtracked clue recovery plus an LLM rubric scorer—and the controlled ablations support the core claim. This is not a restatement of IGPO, CSO, SAPO, or MindDR; the credit signal here is dense and step-local, and the idea of using the verified answer to reconstruct an evidence chain is simple and transferable. The paper also ships code and a model, which makes the work reproducible.\n\nWhat it does well: the reward distribution analysis (Figure 4) is a nice sanity check that failed trajectories contain useful steps and successful ones contain duds, which motivates the method. The ablations in Table 3 are the right comparison—ABC-SFT vs standard SFT, ABC-GRPO vs standard GRPO on the same data—and they show consistent gains across four of five benchmarks for SFT and all five for GRPO. That is credible evidence that dense step-level supervision helps, independent of the absolute numbers.\n\nThe soft spots are real but not fatal. The recovered clue set is the load-bearing component, and the paper gives no precision/recall numbers or human-verified check of the clue-recovery step. The stress-test note is right that if clue recovery is noisy, step rewards misalign; and because the scorer also sees the correct answer, there's a risk of answer leakage rewarding steps that merely mention answer-related entities. This is a legitimate concern, but it doesn't sink the paper: the ablations against standard SFT/GRPO still show gains, and those baselines don't use the clue signal at all. The correct fix is an auxiliary validation study—sample a few dozen questions, have a human check clue completeness and scoring accuracy—and the paper should do that before publication.\n\nThe comparison to 30B agents is overstated. The 55.3% BrowseComp score uses context management, while the 30B baselines are reported without it (the asterisks matter). Without context management, ABSeeker's 37.3% is below most 30B agents. The claim \"matches 30B\" is true only under a non-equivalent evaluation condition. The paper should either compare like-for-like or soften the claim.\n\nMinor: no error bars or significance tests on the three evaluation runs; the RL training uses only 1,000 questions; and the same DeepSeek-V4-Flash model is used for clue recovery, step scoring, and final-answer evaluation, which is a shared-bias risk that a human-verified subset would mitigate.\n\nOverall, this is a serious piece of work with a novel method, credible ablations, and honest limitations in the conclusion. It deserves peer review. I'd recommend accepting it with a request for clue-validation evidence and matched context-management comparisons. It's the kind of paper I'd bring to a reading group working on agent training.","headline":"ABC is a credible new credit-assignment trick for search-agent training, with solid ablations, but the unvalidated clue set and unmatched baselines keep it from being a home run.","tokens_in":12734,"tokens_out":2357,"would_cite":true,"duration_ms":24657,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Scoring each search step against clues backtracked from the final answer lets a 4-billion-parameter agent match much larger search systems on browsing benchmarks.","keywords":["search agents","credit assignment","long-horizon search","step-level rewards","answer backtracking","reinforcement learning","supervised fine-tuning"],"falsifier":"Train ABSeeker with the same step-scoring pipeline but replace the recovered clue sets with an equal number of random entities and facts drawn from the query text; if this pseudo-clue variant matches the reported benchmark scores, the answer-backtracking component is not the source of the gains.","tokens_in":11664,"feed_emoji":"🔍","tokens_out":12401,"duration_ms":105406,"temperature":0.7,"pith_summary":"The paper argues that training long-horizon search agents is held back by trajectory-level supervision, which treats every step in a successful run as good and every step in a failed run as bad. It proposes Answer-Backtracked Credit Assignment (ABC): starting from the known final answer, an LLM actively searches the web backward to recover a chain of intermediate clues, then a scorer rates each step of each training trajectory against those clues, producing a dense per-step reward on a fixed rubric. These step rewards reweight the loss in supervised fine-tuning (ABC-SFT) and replace trajectory-level rewards in GRPO (ABC-GRPO), so a failed trajectory can still reinforce the steps that found real evidence, and a successful trajectory can still penalize steps that discarded it. Training a 4B-parameter agent on 8.5 thousand examples with ABC yields 37.3% on BrowseComp and 39.1% on BrowseComp-ZH (55.3% and 52.9% with context management), the best among same-scale agents and competitive with roughly 30B-parameter systems. The framework turns a single correctness bit per trajectory into fine-grained, answer-anchored supervision for every action.","feed_headline":"4B agent hits 55% on BrowseComp via answer-backtracked step credit","feed_subtitle":"Rewarding useful steps in failed runs and penalizing bad steps in good runs lets a 4B model match bigger search agents.","key_machinery":"The central mechanism is the pair of answer-backtracked anchors and clue-anchored step scoring, which together replace a single outcome bit with a dense per-step scalar. Answer-Backtracked Clue Recovery is an active search loop that starts from the ground-truth answer and travels backward through web evidence to assemble a set of verifiable clues, so each clue is a named entity, fact, attribute, or relation that connects the query constraints to the answer. Clue-Anchored Step Scoring then rates each step of each trajectory against this fixed clue set using an additive rubric on a base score of 1.0, clipped to the interval [0, 2.0]. These step scores are consumed by ABC-SFT, which maps each score through a sigmoid to a per-turn loss weight, and by ABC-GRPO, which uses the scores as step-level rewards with a discounted advantage and the standard clipped GRPO objective. The fixed, answer-backtracked clue set is what keeps the credit signal stable across policy updates and across both successful and failed trajectories.","core_discovery":"The paper's central claim is that answer-backtracked step-level credit assignment makes long-horizon search agents train better than uniform step treatment, and the evidence is a concrete training pipeline and a set of benchmark results. Given a query and a verified answer, Answer-Backtracked Clue Recovery reconstructs the intermediate evidence chain—entities, facts, attributes, and relations—that a valid search should establish, using the same tool-call protocol as the forward agent to verify each clue against real web content. Clue-Anchored Step Scoring then evaluates every step in every collected trajectory against this fixed clue set, starting from a base score of 1.0 and adding 0.8 for discovering or verifying a correct clue, 0.4 for correctly ruling out a wrong candidate, subtracting 0.8 for incorrectly dismissing a correct clue, and adding or subtracting 1.0 for the correct or wrong final answer, with the total clipped to [0, 2.0]. These scores become per-turn loss weights in ABC-SFT and step-level rewards in ABC-GRPO, and the trained 4B agent ABSeeker outperforms all 4B baselines on five benchmarks while matching several ~30B agents. The paper concludes that the model learns to search more efficiently and more deliberately as a result of this fine-grained credit.","pith_inferences":["An untested risk is that the step scorer receives the correct answer as part of its input, so scores may partly reward steps that merely mention the answer or its clues rather than genuinely verify evidence; a cleaner variant would score steps without revealing the answer.","The recovered clue set is assumed to be complete and correct, but the paper reports no quantitative check of recovery accuracy; a perturbation study that corrupts or shuffles clues would reveal how much of the gain actually comes from the backtracked content versus the mere presence of dense supervision.","The method's dependence on a single verifiable answer suggests it will not transfer directly to open-ended research tasks with multiple defensible answers, where no unique evidence chain exists to anchor the scoring.","A practical extension the paper does not explore is using the recovered clue chains at inference time as a search guide or verification checklist, since they encode the evidence structure the agent is expected to discover."],"forward_implications":["Failed trajectories stop being wasted training data: steps that discover or verify correct clues receive positive credit even when the final answer is wrong, so the agent can learn from near-misses.","Successful trajectories stop being treated as uniformly correct: erroneous or redundant steps are down-weighted or penalized, which should reduce reward hacking and sloppy intermediate reasoning.","The clue set provides a fixed, externally anchored evaluation criterion that does not shift with the policy's own beliefs, avoiding the instability of model-based credit signals.","Because the only requirement is a unique, verifiable final answer, the ABC pipeline transfers to any benchmark or task with that property, and the paper shows gains generalize from BrowseComp-style training to xbench and GAIA.","The longer trajectories observed with ABC-GRPO suggest that step-level credit encourages more extensive exploration and verification rather than premature answer submission."],"supporting_citations":[{"why":"Supplies BrowseComp, the benchmark whose unique verifiable answers make answer backtracking possible and serve as the main evaluation.","marker":"(Wei et al., 2025)"},{"why":"Supplies the GRPO objective that ABC-GRPO modifies by replacing trajectory-level advantage with step-level advantage.","marker":"(Shao et al., 2024)"},{"why":"Establishes the ReAct interleaved reasoning/tool-call format that defines the steps being scored.","marker":"(Yao et al., 2022)"},{"why":"Provides the OpenSeeker training trajectories, both correct and incorrect, on which clue scoring and ABC-SFT are applied.","marker":"(Du et al., 2026)"},{"why":"Supplies Qwen3.5-4B, the base model that ABSeeker starts from.","marker":"(Yang et al., 2025)"},{"why":"Provides BrowseComp-ZH, the Chinese-language benchmark used to test cross-lingual transfer.","marker":"(Zhou et al., 2025)"},{"why":"Provides GAIA-text, one of the generalization benchmarks showing the trained agent transfers beyond BrowseComp-style questions.","marker":"(Mialon et al., 2024)"},{"why":"Provides the xbench suites, the other generalization benchmarks on which ABSeeker is evaluated.","marker":"(Chen et al., 2025a)"}],"fun_headline_variants":["Answer-backtracked step credit lifts 4B agent to 55% BrowseComp","Dense step rewards from answer tracing power long-horizon search","Backtrack from answer to reward useful steps, boost search agents","Step-level credit from answer clues lets 4B model match 30B agents"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the recovered clue set is a complete, correct, and fixed record of the evidence a good search must find; if clue recovery misses, invents, or misorders clues, every step reward derived from it is misaligned with true step quality.","fun_headline_variants_meta":{"raw":{"variants":["Answer-backtracked step credit lifts 4B agent to 55% BrowseComp","Dense step rewards from answer tracing power long-horizon search","Backtrack from answer to reward useful steps, boost search agents","Step-level credit from answer clues lets 4B model match 30B agents"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000242,"raw_usage":{"total_tokens":1634,"prompt_tokens":1163,"completion_tokens":471,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":779,"completion_tokens_details":{"reasoning_tokens":390}},"tokens_in":779,"tokens_out":471,"duration_ms":5746,"temperature":1.0,"reasoning_tokens":390,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:12:39.312548+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train ABSeeker with the same step-scoring pipeline but replace the recovered clue sets with an equal number of random entities and facts drawn from the query text; if this pseudo-clue variant matches the reported benchmark scores, the answer-backtracking component is not the source of the gains.","supporting_citations":[{"cited_title":"React: Synergizing reasoning and acting in language models","cited_arxiv_id":null,"evidence_quote":"Establishes the ReAct interleaved reasoning/tool-call format that defines the steps being scored."}],"review_version":1}