{"id":"2312458d-384c-4d0c-a8c2-f553d6e18223","arxiv_id":"2607.13854","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Distilling successful visual tool-use trajectories into a self-updating hierarchical skill library during RL training outperforms both RL-only and memory-only baselines on eight multimodal benchmarks.","lead":"SPyCE is a training framework for multimodal agents that read images: it saves successful tool-use patterns (zooming, rotating, cropping) as reusable skills and keeps updating them while the policy trains. On eight benchmarks it beats both reward-only reinforcement learning and test-time memory retrieval baselines, using fewer tool calls.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No error bars or seeds reported; several headline gains are ≤1.5 points, so 'consistently outperforms' and the co-evolution attribution are not yet statistically supported.","rationale":"The paper's empirical claims rest entirely on point estimates. Several headline margins are below 1.5 points, which is the same order of magnitude as the co-evolution ablation gap (3.1 points on TIR-Bench). Without error bars, multiple seeds, or a stated evaluation protocol that averages over stochastic training runs, 'consistently outperforms' is not falsifiable from the printed tables. This is the most load-bearing concern because it affects every benchmark and every attribution. I do not fully share the reader's weakest-assumption framing: the Frozen Skills ablation, if implemented as described, already removes the online evolution loop while keeping the reward fixed, so the co-evolution mechanism can in principle be isolated. The unresolved issue is not the absence of that control but the absence of any indication of variance. The reader's conditional verdict remains appropriate; the concern strengthens the need for the stated revision rather than moving the verdict.","tokens_in":13597,"tokens_out":9758,"duration_ms":111319,"concrete_test":"Run each configuration in Tables 1–3 with at least 5 independent RL seeds (or a per-sample bootstrap over the evaluation set) and report mean ± 95% CI. Focus on TIR-Bench and ChartQAPro, where margins are smallest; test whether SPyCE beats every baseline on every benchmark and whether Full > FrozenSkills with non-overlapping intervals. If several deltas reverse sign or CIs include zero, the central claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Tables 1–2 report single numbers with no seeds, error bars, or significance tests, and §4.1 does not state how many runs were averaged. Many of the claimed wins over the strongest baseline are small: HRBench-4K +0.8, ChartQAPro +0.9, WeMath +1.2, HRBench-8K +1.4, V* +2.1 (8B backbone). On TIR-Bench, the w/o-penalty ablation is only +1.7 over GRPO, while the co-evolution delta (Full vs Frozen Skills) is 3.1 on a single benchmark. If run-to-run variability is at the 1–3 point level, the 'consistently outperforms' claim and the 'co-evolution is critical' claim could both be noise. The reader's missing-control objection is weaker than it appears: Frozen Skills is a static library under the same reward with evolution removed, so the causal comparison exists in principle. The decisive gap is that no distribution over runs is provided, so we cannot tell whether any of these deltas is real.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SPyCE, a framework for multimodal tool-use agents that couples GRPO-style reinforcement learning with a hierarchical skill library. During training, successful rollout trajectories are distilled by a large teacher MLLM (Qwen3-VL-235B) into execution skills (condition–action–effect triples) and workflow skills (bottleneck description plus workflow sketch). The policy conditions on retrieved skills during rollouts, and the libraries are periodically updated from high-quality rollouts, creating a claimed closed skill-policy co-evolution loop. Experiments on eight benchmarks (TIR-Bench, V*, HRBench-4K/8K, ChartQAPro, MathVerse, MathVision, WeMath) report that SPyCE outperforms prompt-based, RL-based (GRPO, RLOO), and memory-based (MemP, Dynamic Cheatsheet, Agent-KB) baselines. Ablations on TIR-Bench attribute gains to the hierarchical skill design, online skill evolution, and the tool-call penalty.","tokens_in":13838,"tokens_out":3182,"duration_ms":33170,"significance":"If the reported results are robust, the paper makes a worthwhile contribution: it addresses an important limitation of current RL-trained multimodal agents—trajectories are reduced to scalar rewards rather than reused as structured knowledge—and the proposed closed-loop skill-policy co-evolution is a timely idea. The evaluation covers several public benchmarks and compares with both RL and memory-augmented baselines, and the ablations attempt to isolate the contributions of the two skill levels and online evolution. The convergence and tool-use analysis is also informative. However, the central claims currently rest on single-run numbers with no variance estimates, and several comparisons have margins small enough to be within plausible run-to-run noise. The reward used for the RL baselines is not specified, which is a direct confound for the headline comparisons. These issues are fixable but currently block acceptance.","major_comments":[{"comment":"No standard deviations, seeds, or significance tests are reported. Several claimed gains over the strongest baseline are small: HRBench-4K +0.8, ChartQAPro +0.9, WeMath +1.2, HRBench-8K +1.4, V* +2.1 (8B backbone). Without a distribution over runs, the statement 'consistently outperforms' is not statistically supported. Please report the number of runs, mean ± std, and a significance test where appropriate.","section":"§4.2, Tables 1–2"},{"comment":"The reward function used for the GRPO and RLOO baselines is not specified. SPyCE uses R(τ)=R_acc(τ)−λ_call·1[Succ]·N_tool, with λ_call=0.05. If the baselines were trained with a different reward (e.g., accuracy only, or a different penalty), the comparison is confounded: the gains attributed to skills could partly reflect reward-shaping differences. The exact reward used for every method must be stated in the main text or a table.","section":"§4.1.2 and Eq. (3)"},{"comment":"The ablations are single numbers with no variance. The key attribution claims—that 'both the hierarchical skill design and the co-evolution mechanism are critical'—depend on deltas of 3.1–4.0 points on one benchmark. For example, Frozen Skills (28.9) vs Full Model (32.0) is a 3.1-point gap; with no error bars, this could be noise. Please provide repeated runs/seeds for at least the main ablations, and clarify exactly what 'Frozen Skills' means: is the library built from the same initial rollouts and then kept fixed, with all other components identical? Only then does the comparison isolate online evolution.","section":"§4.3, Table 3"},{"comment":"The method depends on a large number of hyperparameters—λ_call, merge thresholds θK_sim and θH_sim, retrieval thresholds τ_sim/θ_sim, top-K and top-M, evolution window N, retention fractions ρ_wf and ρ_ex, and library capacities. No sensitivity analysis is reported. While this alone is not fatal, it matters for the claim that the framework is broadly useful, especially because some of these thresholds directly control the trade-off between adding vs merging skills. A sensitivity study for the most critical thresholds (e.g., λ_call and the merge thresholds) would substantially strengthen the paper.","section":"§3.2–§3.3"}],"minor_comments":[{"comment":"Typo: 'Consitent' should be 'Consistent'.","section":"§3.2.2"},{"comment":"Typo: 'valitaion set' should be 'validation set'.","section":"§4.4"},{"comment":"The tool name 'Imags.crop' appears to be a typo for 'Images.crop'.","section":"Figure 3"},{"comment":"The label '~12% Faster Converge' is unclear: does it mean 12% fewer steps to reach a given success rate, or a 12% higher success rate at a fixed step? Please clarify.","section":"§4.4, Figure 5"},{"comment":"RL data filtering from 34,795 to ~12K samples is described at a high level. Please provide the per-step counts or a short description of each filtering stage so the reader can assess potential data-selection bias.","section":"§4.1.1"},{"comment":"Several implementation details are deferred to the supplementary material, which is not included in the submission. For a self-contained review, please include the supplementary or at least specify baseline reward definitions and retrieval details in an appendix.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's central idea is interesting and the empirical comparisons are broad, but the missing statistical rigor is a real barrier. I would ask for seeds/variance, exact baseline rewards, and clarification of the Frozen Skills ablation. If the results hold under repeated runs, this could be a solid contribution; as written, the evidence is not yet sufficient to support the 'consistently outperforms' and 'co-evolution is critical' claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. SPyCE is a training recipe worth taking seriously, but the evidence as printed is not yet strong enough to call the headline claim proven. The genuinely new piece is the closed loop: distill successful multimodal tool-use trajectories into a two-level skill library (execution skills as condition–action–effect triples; workflow skills as bottleneck plus sketch), condition the policy on retrieved skills during GRPO, and periodically refresh the library from high-value rollouts. That loop is not in DeepEyes, VTool-R1, CodeVision, or the memory baselines, and it is a plausible way to make skill reuse part of policy learning rather than test-time retrieval.\n\nThe paper does several things well. The method section is coherent; the merge-or-add consolidation and success-rate pruning are sensible. The ablations are internally consistent: removing workflow skills, execution skills, or online evolution each costs 3–4 points on TIR-Bench, and removing the tool-call penalty costs 2.6 while raising tool calls. The case study illustrates how retrieved skills change tool ordering. The reported tables do support the stated pattern of wins over the listed baselines.\n\nThe soft spots are real but not fatal. First, no seeds, error bars, or significance tests anywhere. Several headline margins are 0.8–1.8 points, so 'consistently outperforms' is an assessment of point estimates only. Second, the reward is not aligned across methods: SPyCE adds a tool-call penalty, and the GRPO/RLOO baseline reward is unspecified. On TIR-Bench the penalty alone accounts for about 60% of the gap over GRPO, so part of the 'skill' gain may just be reward shaping. Third, artifacts are missing—code, filtered data, skill libraries, and the supplementary that contains thresholds and capacities—so the method is under-specified as printed. Fourth, the 'first co-evolution loop' claim overreaches; prior skill-discovery and memory-evolution work is discussed but not engaged at that level of detail.\n\nOne nuance: the reader's missing-control objection is weaker than it looks. The Frozen Skills variant is a static teacher-built library under the same reward with evolution removed, so a causal comparison for co-evolution exists in principle. The decisive gap is that we have no distribution over runs, so we cannot tell whether the 3.1-point evolution delta is signal or noise.\n\nWho is this for? People working on multimodal tool-use RL will want to read it as a candidate recipe and as a cautionary example of evaluation reporting. It deserves a serious referee: the mechanism is novel, the framework is coherent, and the questions are answerable. I would accept it for review but ask for variance estimates, aligned reward across baselines, and artifact release before the claims are taken as established.","headline":"A coherent and plausibly novel training recipe for multimodal tool-use agents, but the reported evidence is point estimates only and the reward is not aligned across baselines—worth a serious referee, not yet a proven claim.","tokens_in":14447,"tokens_out":2887,"would_cite":true,"duration_ms":30457,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SPyCE claims that distilling trajectories into an evolving hierarchical skill library makes multimodal agents outperform both reward-only reinforcement learning and static-memory retrieval.","keywords":["multimodal agents","skill-policy co-evolution","hierarchical skill library","reinforcement learning","tool use","visual reasoning","trajectory distillation","GRPO"],"falsifier":"Run the same training recipe with a skill library built once by the same teacher model and then frozen, keeping the identical reward, retrieval procedure, and rollout budget. If the frozen-library version matches or exceeds the online co-evolution version on the agentic benchmark, the claim that co-evolution is critical is refuted.","tokens_in":13335,"feed_emoji":"🧩","tokens_out":6305,"duration_ms":59799,"temperature":0.7,"pith_summary":"SPyCE tries to show that a multimodal agent's own successful trajectories should be turned into reusable skills during training, rather than thrown away as scalar rewards or stored as static memories. The paper builds a two-level skill library — short execution skills and longer workflow skills — and continuously updates it from the policy's best rollouts while the policy trains on retrieved skills. The claim is that this closed loop beats both RL-only and memory-only alternatives, with gains of roughly one to four accuracy points across eight benchmarks, fewer tool calls, and faster early convergence. A reader should care because it reframes experience itself as a training signal: the agent gets better, and the better agent's traces make the next round of skills better.","feed_headline":"Co-evolving skills beat RL baselines on eight benchmarks","feed_subtitle":"Distilling an agent's own successful traces into reusable skills lifts accuracy and cuts tool calls.","key_machinery":"The load-bearing object is the two-level skill library plus its update loop. Execution skills are triples (c, u, e) — a trigger condition, a tool action, and an expected effect — retrieved by embedding similarity on the condition. Workflow skills are pairs (b, w) — a coarse bottleneck summary and a workflow sketch — retrieved by similarity on the bottleneck. Retrieval is hierarchical: the policy first writes a bottleneck query, pulls workflow skills, then decomposes the task into local bottleneck descriptors and pulls one execution skill per descriptor. A merge-or-add consolidation rule merges near-duplicate candidates and prunes the least-used entries, keeping both libraries bounded. The po","core_discovery":"The paper's central claim is that a hierarchical skill library updated in lockstep with the policy produces measurably better multimodal tool-use agents. Concretely, on the reported benchmarks the co-evolved agent beats plain group-relative policy optimization by 0.8 to 4.3 accuracy points, and on an agentic reasoning benchmark it reaches 32.0 average accuracy with 4.83 average tool calls, compared with 27.7 and 5.13 for GRPO. The mechanism is a closed loop: execution skills (trigger condition, tool action, expected visual effect) are matched to local bottlenecks; workflow skills (bottleneck summary, workflow sketch) guide multi-step planning; the policy conditions on both while optimizing t","pith_inferences":["A clean next experiment would compare the online co-evolved library against a static library distilled by the same teacher under the identical reward; that isolates the update loop from the teacher's priors.","Because skills are stored as text, one could transfer a library trained with one backbone to a smaller or larger backbone at inference, potentially turning training-time co-evolution into test-time skill reuse.","The observed early convergence suggests the skill library acts partly as a curriculum, shaping exploration toward bottlenecks that recur across tasks; testing whether skill diversity correlates with exploration would connect this to exploration-exploitation theory."],"forward_implications":["If the central claim holds, trajectory distillation into skills should be adopted alongside policy-gradient training for multimodal tool-use agents, not treated as an alternative to it.","The reported tool-call counts imply that skill conditioning makes agents more selective about tools, not merely more eager to use them.","The ablation outcomes imply that neither skill level alone is sufficient: workflow-level guidance and execution-level precision contribute separately, and freezing the library costs about three points.","The tool-call penalty results imply that a penalty applied only to successful episodes can improve both accuracy and efficiency, whereas a blanket penalty suppresses useful exploration."],"fun_headline_variants":["Skills and policy co-evolve, beating RL on 8 benchmarks","Co-evolving skill library lifts multimodal agent accuracy","SPyCE: train skills with policy, not just rewards","Better agents by distilling traces into reusable skills","Closed-loop skill-policy updates cut tool calls, improve scores"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the online co-evolution update, rather than the teacher's distilled skill priors or the changed reward, is what produces the final-points gain; the paper does not run a frozen-teacher-library control under the identical reward.","fun_headline_variants_meta":{"raw":{"variants":["Skills and policy co-evolve, beating RL on 8 benchmarks","Co-evolving skill library lifts multimodal agent accuracy","SPyCE: train skills with policy, not just rewards","Better agents by distilling traces into reusable skills","Closed-loop skill-policy updates cut tool calls, improve scores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000746,"raw_usage":{"total_tokens":3184,"prompt_tokens":792,"completion_tokens":2392,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":2311}},"tokens_in":536,"tokens_out":2392,"duration_ms":18227,"temperature":1.0,"reasoning_tokens":2311,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T03:31:10.741770+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same training recipe with a skill library built once by the same teacher model and then frozen, keeping the identical reward, retrieval procedure, and rollout budget. If the frozen-library version matches or exceeds the online co-evolution version on the agentic benchmark, the claim that co-evolution is critical is refuted.","supporting_citations":[],"review_version":1}