{"id":"2dbb9fe0-f7f1-4334-812a-f617bc9ea880","arxiv_id":"2505.10522","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A three-stage grasping-picking-stacking curriculum with a redesigned reward reduces training time by about 40 percent and improves block-stacking success by roughly 10 percent over direct learning in the CausalWorld simulator.","lead":"This paper describes a way to train a simulated robot arm to stack blocks by first practicing easier sub-tasks and then reusing that experience. The authors report faster training and better success on one stacking benchmark, but the comparison baseline and missing error bars require caution.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 40% training-time and 10% success-rate claims are not yet testable because \"convergence\" is undefined, no variance is reported for the 5-seed averages, and the winning curriculum was selected post hoc.","rationale":"The reader's weakest assumption about the similarity metric M is valid and worth flagging: the binary reward-component cosine similarity in Eq. (7) is an unvalidated proxy for transferability, and the framework-level guidance in Section 6 rests on it. However, the headline quantitative claim about a 40% training-time reduction does not actually depend on M being correct: the best 3-stage curriculum was obtained by sweeping transition times and learning rates, not by deriving 60/900/540 from the similarity values. The load-bearing condition for the central empirical claim is instead a well-defined, statistically meaningful comparison between the selected curriculum and a direct-learning baseline. That condition is currently unmet because \"convergence\" is not defined, no error bars or per-seed data are reported, and the best configuration appears to have been chosen after inspecting many experimental variants. This is an evidence-quality problem rather than an internal inconsistency: nothing in the paper suggests the proposed method could not work, but the reported magnitude of the improvement is not yet established. I therefore keep the reader's CONDITIONAL verdict unchanged; a revision that adds a precise convergence criterion, per-seed variability, and a pre-specified comparison protocol would address the concern. I also note that the paper's own Section 5.3 states the best candidate is the 60-episode grasping variant, even though Section 5.2 argued 300 episodes was the optimal transition for grasping-to-picking; reconciling this would strengthen the post-hoc selection concern.","tokens_in":12410,"tokens_out":7823,"duration_ms":73755,"concrete_test":"Obtain the per-seed learning curves for direct stacking and for the 60-grasping/900-picking/3-stage curriculum with the same stacking-stage learning rate. Define convergence as the first episode at which a rolling-window (e.g., 50-episode) mean of fractional success exceeds a pre-specified threshold and remains above it for the rest of training. Recompute the episode reduction and final-success difference for thresholds 0.5, 0.7, and 0.9, and report the per-seed range. If the 40%/10% advantage does not hold across thresholds and exceeds the seed spread, the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result in Section 5.3 is that the best 3-stage curriculum (60 grasping + 900 picking + 540 stacking = 1500 episodes) beats direct stacking (2500 episodes) by 40% and raises final fractional success by 10%. For this comparison to be meaningful, the authors must define what \"completes training\" or \"convergence time\" means, and must show that the difference is not within seed noise. Neither condition is met. The text states all results are averaged over 5 random seeds but never reports standard deviations, confidence intervals, or individual seeds; \"approximately 540 episodes\" is used as the stacking-stage convergence time, while direct learning \"requires 2500 episodes\" and \"still fails to achieve 100% fractional success.\" No success-rate threshold is specified. The winning curriculum is chosen after inspecting many variants (60 vs 300 grasping episodes, 900 vs 1200 vs 1800 picking episodes, three learning-rate schedules), so the comparison is vulnerable to selection bias. Additionally, the direct-stacking baseline is compared without specifying which learning rate (1e-4, 5e-5, or 1e-5) is used. Without a pre-specified convergence criterion and variance estimates, the 40%/10% gap cannot be distinguished from threshold or schedule artifacts. This concern is distinct from the similarity-metric issue: even if the binary reward-component cosine similarity in Eq. (7) were valid, the headline empirical claim would still lack a defensible quantitative basis.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Knowledge Capture, Adaptation, and Composition (KCAC) framework for cross-task curriculum learning in robotic manipulation, evaluated on a two-block stacking task in CausalWorld. The authors redesign the benchmark reward to remove conditional sequencing, introduce two hand-designed subtasks (grasping and picking), and represent tasks as binary reward-component vectors with cosine similarity. They systematically vary transition timing and learning rate in two-stage curricula and then construct a three-stage curriculum. The central claim is that the best three-stage curriculum (60 episodes grasping, 900 episodes picking, then stacking) completes stacking in about 1,500 episodes versus 2,500 episodes for direct learning with the redesigned reward, a 40% reduction in training time with a 10% higher final fractional success.","tokens_in":12833,"tokens_out":4538,"duration_ms":40790,"significance":"If substantiated, the results would provide a useful demonstration that a compound reward function without hard sequencing, combined with a staged curriculum, can accelerate SAC training on a contact-rich manipulation task. The paper also makes a design-oriented contribution by cataloguing how transition timing and learning rate interact with task similarity. However, the evidence as presented is not sufficient: the headline 40%/10% gains are based on five-seed averages with no variance, an undefined convergence criterion, a post hoc selected curriculum, and an unspecified baseline learning rate. The similarity measure in Eq. (7) is asserted rather than validated. The reward-redesign observation itself is interesting but is confounded by an initialization change. The work is therefore promising but currently falls short of a solid quantitative claim.","major_comments":[{"comment":"The headline claim of a 40% training-time reduction rests on the assertion that the best 3-stage curriculum 'completes training in 1500 episodes' while direct stacking 'requires 2500 episodes.' No convergence threshold is defined, and all curves are averages over five seeds with no standard deviation or confidence band. Because the winning curriculum was selected after inspecting many two- and three-stage configurations with three learning rates, the comparison is vulnerable to selection bias. Please report per-seed curves or variance, define the success-rate threshold used to determine convergence, and state whether the baseline learning rate was the most favorable one.","section":"5.3; Fig. 15"},{"comment":"The redesigned reward is evaluated after initializing block_1 at its goal position, which deactivates the first condition of the original CausalWorld reward. This changes the task itself, so the large improvement over the baseline in Figures 4-5 may reflect a simpler task rather than better knowledge capture. Please compare against the unmodified benchmark task, or clearly present the modified task as a separate benchmark and temper the comparison.","section":"3.1; Eq. (3); Figs. 4-5"},{"comment":"For the direct-stacking baseline in Figure 15, the learning rate is not stated. Section 5.2 shows that learning rate strongly affects curriculum performance; without knowing which setting was used for direct learning, the 40% improvement cannot be attributed to the curriculum. Please report the baseline learning rate and, ideally, the best direct-learning result across the three settings.","section":"5.3"},{"comment":"The guidance function M is based on binary cosine similarity between reward-component vectors. The manuscript asserts that relative component contributions are comparable when tasks are implemented appropriately, but no evidence is given. Since Eq. (7) drives the transition-timing and learning-rate recommendations in Section 5.2 (e.g., low similarity -> early transition and high learning rate), this assumption is load-bearing. Please validate it with transfer experiments or present M as a descriptive heuristic rather than a predictive principle.","section":"4.2; Eq. (7)"},{"comment":"The KCAC procedure is presented as a framework, but functions G and M are only specified after the results are known, and no algorithm for constructing them is given. As presented, the framework does not offer a predictive rule for new tasks; Section 6 itself states that future work is needed for mathematical formulation of M. The paper should clarify whether KCAC is a conceptual framework or an implementable method.","section":"3.2; Table 1"}],"minor_comments":[{"comment":"The title typo 'ADAPTAION' should be corrected to 'ADAPTATION'.","section":"Title"},{"comment":"The caption word 'Comparision' should be 'Comparison'.","section":"Fig. 15 caption"},{"comment":"The phrase 'In this session' should be 'In this section'.","section":"4.3"},{"comment":"References [29] and [31] duplicate the same Cooper 1999 citation, and references [30] and [33] duplicate the same Amini et al. citation; please consolidate them.","section":"References"},{"comment":"Equation (1) contains garbled symbols (e.g., '$&’!()*(+!<,’&-’' and '𝟏') that render incorrectly; please ensure all mathematical expressions are typeset cleanly.","section":"Eq. (1)"},{"comment":"The curve labels such as 'stack2' and 'grasp2_60_curri_lr_1e-5' are not defined in the captions; please explain all legend entries explicitly.","section":"Figures 6-14"}],"recommendation":"major_revision","confidential_remarks":"The paper reads as a compact conference-style manuscript. The empirical claims are potentially sound but lack the statistical and methodological detail expected of a journal article. The missing variance, undefined convergence, and unspecified baseline are fixable if the authors still have per-seed data. I recommend major revision rather than rejection, with the expectation that the authors supply the missing quantitative support and either validate or reframe the similarity-based guidance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know: this paper is a case study on CausalWorld two-block stacking where the authors redesign the reward function and run a 3-stage grasp-pick-stack curriculum. The specific empirical result—about 1500 episodes vs 2500 for direct learning—is new in that exact form, but it is not a new method; it is a sensible combination of reward shaping, curriculum learning, and transfer. The paper does a few things well. The diagnosis that the original CausalWorld reward forces a strict learning order and that this hurts is plausible, and the redesigned reward (removing conditional gates) is clearly described. The binary reward-component similarity vector is a simple, transparent way to compare tasks, and the qualitative guidance (low similarity -> early transition/high LR, high similarity -> late transition/low LR) is consistent with the reported trends.\n\nThe soft spots are real. First, the headline 40%/10% claim is not testable as reported. 'Convergence time' is never defined, no standard deviation or confidence interval is given for the five-seed averages, and the winning curriculum (60 grasp + 900 pick + 540 stack) was selected after scanning many variants, so selection bias is a live threat. The direct-learning baseline is compared without specifying which learning rate was used, and it is compared against modified-reward direct learning, not the original benchmark reward. Second, the M and G functions are defined retroactively from the same experiments that are used to demonstrate them; the similarity measure is based on binary presence of reward terms and ignores reward magnitudes, dynamics, and state-space overlap, so the guidance it produces is a heuristic, not an established principle. Third, no code or data are provided, only 'upon reasonable request,' which limits independent checking.\n\nThese are not fatal to the core idea. The trends across the figures (described in the text) are consistent with the claimed effect, and the reward redesign seems genuinely useful for this benchmark. But the paper overstates what is established: it presents heuristics as a 'framework' and a single post-hoc best run as a quantitative result.\n\nWho should read it: researchers working on CausalWorld or on curriculum design for robotic manipulation will find the reward breakdown and the similarity-based timing suggestions worth a look, mainly as inspiration rather than as a citable result. I would not cite the 40%/10% numbers in my own work.\n\nRecommendation: send it to peer review, but with a clear request for error bars, a defined success threshold, a pre-specified or at least fully disclosed curriculum search, and a baseline comparison against the original reward. With those fixes it could be a solid case study; without them, the headline claim should be downgraded.","headline":"A plausible reward-redesign plus curriculum case study on CausalWorld stacking, but the headline 40%/10% numbers are not yet supported because there is no defined convergence criterion, no variance reporting, and the winning curriculum was picked post hoc.","tokens_in":13312,"tokens_out":2845,"would_cite":false,"duration_ms":27551,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On a simulated two-block stacking task, a three-stage curriculum with a redesigned reward reaches success in about 1,500 episodes, a 40 percent reduction from direct learning with a roughly 10 percent higher final success rate.","keywords":["reinforcement learning","curriculum learning","knowledge transfer","robotic manipulation","reward function design","two-block stacking","Soft Actor-Critic"],"falsifier":"Run the low-similarity grasping-to-stacking curriculum at learning rate 1e-5: the paper's similarity rule predicts slower convergence than at 1e-4, because low similarity is said to need a high learning rate; a faster convergence at the low rate would falsify the M function. Also, construct two tasks with identical binary reward vectors but very different state and contact dynamics; if transfer between them is poor, the binary vector misses what actually transfers.","tokens_in":12203,"feed_emoji":"🤖","tokens_out":7554,"duration_ms":70507,"temperature":0.7,"pith_summary":"This paper argues that the reason agents struggle on a simulated two-block stacking task is not the learning algorithm but the reward function and the absence of a structured knowledge-transfer plan. It proposes KCAC, a knowledge-capture, adaptation, and composition framework that replaces the benchmark's conditional, staged reward with an unconditional sum of dense and sparse terms, and that pre-trains the agent on a grasping task and then a picking task before the full stacking task, choosing when to switch and what learning rate to use from a cosine-similarity measure between reward-component vectors. The central claim is that this combination finishes training in about 1,500 episodes versus 2,500 for direct learning, a 40 percent reduction, while raising the final fractional success by about 10 percent. If correct, reward engineering plus cross-task curricula can substitute for more algorithm-level intervention on complex manipulation tasks.","feed_headline":"Three-stage robot training cuts stacking time by 40 percent","feed_subtitle":"A relaxed reward plus grasping-picking-stacking pretraining beats direct learning on a two-block assembly task.","key_machinery":"The load-bearing object is a binary reward-component vector representing each task: a task's compound reward is written as a sum of eight named terms, and a task is coded by which terms are present (1) or absent (0). Cosine similarity between two such vectors, $\\operatorname{Sim}(V_i,V_j)=V_i\\cdot V_j/(\\|V_i\\|\\,\\|V_j\\|)$, is used to choose when to transfer weights and what learning rate to use: low similarity (grasping to stacking, 0.4) recommends early transition and high learning rate, while high similarity (picking to stacking, 0.8) recommends late transition and low learning rate. The second half of the mechanism is the redesigned stacking reward of Equation (3), which removes the indicator-function gates of the benchmark reward so that dense and sparse terms can all be optimized concurrently. These two pieces, unconditional reward plus similarity-guided curriculum, carry the argument.","core_discovery":"The paper's central claim is that the KCAC curriculum, with 60 episodes of grasping pretraining, 900 episodes of picking pretraining, and a learning rate of 1e-5 for the stacking stage, learns the two-block stacking task in about 1,500 episodes total, compared with about 2,500 episodes for direct learning on the redesigned reward, a 40 percent reduction in training time with about a 10 percent higher final fractional success. The argument has two parts. First, the benchmark reward, which gates each reward component on conditions that force a fixed ordering of sub-skills, is replaced by an unconditional compound reward that lets the agent accumulate dense and sparse rewards simultaneously; in the paper's experiments this alone activates learning of the top block that the baseline never achieves. Second, knowledge from simpler tasks is transferred through weight initialization, with the transition point and learning rate set according to task similarity: low-similarity grasping-to-stacking benefits from early transition and high learning rate, while high-similarity picking-to-stacking benefits from long pretraining and low learning rate. A three-stage curriculum combines these, cutting the picking pretraining needed from 1,800 to 900 episodes while matching or exceeding the two-stage result.","pith_inferences":["An immediate test of the similarity measure would be to recompute it with reward weights instead of binary flags; if the recommended curriculum order changes, the binary simplification is not neutral.","Since the reported metric is fractional overlap, a stricter binary success threshold might erase part of the 10 percent gain; the 40 percent time saving should be re-checked under a thresholded success definition.","The paper's M and G functions are conceptual, not fitted; with more sub-task pairs and similarity scores, they could become explicit regression models that predict transition timing and learning rate, which the paper itself lists as future work."],"forward_implications":["Reward functions that gate learning on a fixed order of sub-skills can be the main obstacle to solving a manipulation task; removing the gates can activate learning of previously ignored components.","Pairwise task similarity, measured from reward structure alone, can be used to set two curriculum knobs, transition timing and learning rate, without exhaustive sweeps.","A three-stage curriculum can replace a long high-similarity pretraining stage with a chain of shorter low- and medium-similarity stages, reducing total episodes while keeping final success.","The same reward-decomposition-and-transfer recipe applies to other tasks whose reward is a sum of identifiable movement and goal components."],"supporting_citations":[{"why":"Defines the two-block stacking task, its original conditional reward function, and the baseline model whose failure motivates the redesign.","marker":"[1]"},{"why":"Supplies the off-policy maximum-entropy actor-critic algorithm used for all training runs.","marker":"[2]"},{"why":"The paper's earlier study of the same benchmark, cited as evidence that cross-task curricula with early transitions outperform in-task curricula.","marker":"[3]"},{"why":"Previous transfer study cited as evidence that similarity between source and target tasks controls transfer efficiency, the basis for the M function.","marker":"[6]"},{"why":"Earlier result showing that RL on the original reward fails to exceed 50 percent fractional success, used to argue existing approaches do not capture the needed knowledge.","marker":"[28]"}],"fun_headline_variants":["Robot stacking learns 40% faster with staged curriculum","Relaxed rewards plus pretraining speed up robot stacking by 40%","Curriculum learning trims robot stacking time by 40%, lifts success","Objective relaxation and skill pretraining cut stacking training by 40%","Three-stage robot curriculum cuts stacking time by 40%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything hinges on the assumption that the presence or absence of shared reward terms, measured by cosine similarity over binary vectors, tells you how much skill transfers between two tasks; if reward magnitudes, state-space overlap, or contact dynamics matter more, the recommended transition timing and learning rates have no reliable basis.","fun_headline_variants_meta":{"raw":{"variants":["Robot stacking learns 40% faster with staged curriculum","Relaxed rewards plus pretraining speed up robot stacking by 40%","Curriculum learning trims robot stacking time by 40%, lifts success","Objective relaxation and skill pretraining cut stacking training by 40%","Three-stage robot curriculum cuts stacking time by 40%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001097,"raw_usage":{"total_tokens":4628,"prompt_tokens":1042,"completion_tokens":3586,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":3498}},"tokens_in":658,"tokens_out":3586,"duration_ms":25102,"temperature":1.0,"reasoning_tokens":3498,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:07:30.251130+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the low-similarity grasping-to-stacking curriculum at learning rate 1e-5: the paper's similarity rule predicts slower convergence than at 1e-4, because low similarity is said to need a high learning rate; a faster convergence at the low rate would falsify the M function. Also, construct two tasks with identical binary reward vectors but very different state and contact dynamics; if transfer between them is poor, the binary vector misses what actually transfers.","supporting_citations":[{"cited_title":"of Aerospace & Mechanical Engineering University of Southern California Los Angeles, USA xinruiw@usc.edu Yan Jin* Dept","cited_arxiv_id":null,"evidence_quote":"Defines the two-block stacking task, its original conditional reward function, and the baseline model whose failure motivates the redesign."},{"cited_title":"In the domain of robotic manipulation, machine learning has significantly advanced its capabilities in handling objects and executing complex tasks","cited_arxiv_id":null,"evidence_quote":"Supplies the off-policy maximum-entropy actor-critic algorithm used for all training runs."},{"cited_title":"Reinforcement learning is one of the solutions, allowing agents to understand and optimize the process through continuous interaction with their environment [14-16]","cited_arxiv_id":null,"evidence_quote":"The paper's earlier study of the same benchmark, cited as evidence that cross-task curricula with early transitions outperform in-task curricula."},{"cited_title":"The default learning rate of 1e-4, as provided by the baseline, is used for all pre-training phases in the curriculum","cited_arxiv_id":null,"evidence_quote":"Previous transfer study cited as evidence that similarity between source and target tasks controls transfer efficiency, the basis for the M function."}],"review_version":1}