{"id":"3d1bbc00-a090-4bcf-9755-f70c499a01bb","arxiv_id":"2508.08982","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A robot training framework, SDAX, uses unsupervised skill discovery as an exploration signal with a learned balancing weight, letting a Unitree A1 learn leap, climb, crawl, and wall-jump behaviors in simulation and on hardware.","lead":"Researchers trained a quadruped robot to learn agile moves such as leaping over gaps and jumping off walls by combining a basic goal reward with an automatic exploration reward and letting the algorithm tune the balance. The result is a step toward robot training that needs less hand-designed reward shaping and curriculum staging.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SDAX's exploration is guided by manually chosen state sub-dimensions, so the claimed reduction in human engineering is not established for novel task dimensions.","rationale":"The reader's weakest assumption is the same as my most load-bearing concern: SDAX's exploration is not autonomous over all state dimensions, but is focused by a human-chosen projection of state into the diversity objective. The paper's own Section 6 explicitly admits this, and the wall-jump experiment shows that task-only reward fails without the orientation sub-dimension being added. This concern does not invalidate the demonstrated successes on the four reported tasks, so I would not move the verdict to REJECT. However, it justifies the CONDITIONAL verdict already given by the reader: the central claim about reducing human engineering is only supported for tasks where the experimenter already knows the relevant behavioral dimension. The proposed concrete test would directly settle whether the method's success depends on this manual choice.","tokens_in":11164,"tokens_out":6278,"duration_ms":76387,"concrete_test":"Re-run the leap experiment with the diversity reward computed on a deliberately wrong sub-dimension (e.g., base height instead of forward velocity), keeping all hyperparameters and the lambda schedule identical. If the obstacle-pass rate stays near the reported ~90%, the method is robust to sub-dimension choice; if it collapses toward the task-only baseline, success depends on the human having selected the correct exploration dimension. A complementary check is to train on a new obstacle whose solution requires lateral base velocity or yaw exploration without any manual sub-dimension selection, and record whether SDAX solves it.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To deliver the paper's headline reduction in human engineering, SDAX must discover the behavioral dimension that unlocks task progress. The diversity reward r_div = (phi(s') - phi(s))^T z only rewards diversity along the state features fed into phi, and the paper reports that for each task a human must choose those features: base height for climb/crawl, forward velocity for leap, and base orientation for wall-jump. Section 6 says 'effective training requires manual specification of sub-dimensions of the state space to guide exploration.' If the correct dimension is omitted, lambda has no mechanism to increase exploration along it because r_div contains no signal there; the method then reduces to task-only training, which the paper itself shows fails on the wall-jump (Figure 9b). The four successes therefore demonstrate that SDAX amplifies human-provided task insight, not that it autonomously finds the right exploration space. This does not contradict the in-task results, but it does undercut the general claim that SDAX 'autonomously exploring a diverse set of strategies' eliminates the need for task-specific engineering.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SDAX, a framework for legged locomotion that combines a task reward with an unsupervised skill-discovery diversity reward (METRA or DIAYN) and learns a scalar balancing parameter λ through a bi-level surrogate gradient (Eq. 3). The policy is skill-conditioned and trained with PPO. Experiments on a Unitree A1 in Isaac Gym show that SDAX with METRA solves leaping, climbing, crawling, and a wall-jump, outperforms fixed-λ variants on the leap task, and transfers to real hardware for leap, climb, and crawl. The paper also reports a 'positive collapse' phenomenon in which a growing fraction of sampled skill vectors solve the task.","tokens_in":11389,"tokens_out":5227,"duration_ms":58048,"significance":"If the results hold, SDAX provides a simple exploration recipe that can replace curricula or demonstrations for several agile locomotion skills, and the multi-seed training curves and real-robot deployment are genuine strengths. The method's reliance on manually chosen state sub-dimensions for the diversity reward and a human-specified guideline for the wall-jump, however, materially limits the claimed reduction in human engineering effort.","major_comments":[{"comment":"The derivation of the balancing-parameter gradient is not valid as written. In Eq. (6), A^{task+div} is treated as a constant when differentiating the product A^{task+div}∇θ log πθ(a|s,z) with respect to λ, but both A^{task} and A^{div} depend on λ through the policy, and the derivative of ∇θ J^{task} with respect to λ is omitted. The derivation also drops expectation signs and treats single-sample advantages as exact gradient estimates. Since the λ update is contribution (2) and is used in Algorithm 1 (line 17), please supply a correct derivation or explicitly state that Eq. (3) is a heuristic surrogate, and justify it empirically, for example by comparing against alternative λ-update rules on the same tasks.","section":"Appendix A, Eq. (3) and Eq. (7)"},{"comment":"The central claim of autonomous exploration is not established for tasks whose relevant behavioral dimension is not known in advance. The diversity reward r_div = (φ(s')−φ(s))^T z only rewards change along the state features provided to φ, and the paper reports that for each task a human chose the sub-dimensions: base height for climb/crawl, forward velocity for leap, and base orientation for wall-jump. Section 6 concedes that 'effective training requires manual specification of sub-dimensions of the state space to guide exploration.' If the correct dimension is omitted, λ has no mechanism to increase exploration along it, and SDAX reduces to task-only training, which the paper itself shows fails on the wall-jump (Fig. 9b). Please either add a mechanism for discovering or expanding the explored dimensions, or temper the claim that SDAX 'autonomously exploring a diverse set of strategies' reduces human engineering for novel tasks.","section":"Section 6 and Section 4.1"},{"comment":"The wall-jump experiment uses a human-specified guideline reward as r_task, consisting of a sequence of user-defined waypoints. This is a form of task-specific reward engineering, which the Introduction lists among the techniques SDAX aims to reduce. The wall-jump result should be framed as SDAX combined with a hand-designed task reward, and the paper should report how sensitive the result is to the guideline shape, such as the number of waypoints or their spacing.","section":"Section 4.4 and Appendix D.1"},{"comment":"The adaptive-λ comparison against fixed λ values is conducted only on the leaping task. Since the paper claims that the learned λ dynamically balances exploration and exploitation and that 'our adaptive method outperforms all fixed-value experiments,' the claim is stronger than the evidence. Please run the fixed-λ comparison on at least one additional task (e.g., climb or crawl) or present the λ trajectories for all tasks to support the general mechanism.","section":"Section 4.2"}],"minor_comments":[{"comment":"There is a typo: 'straightfoward' should be 'straightforward.'","section":"Section 3.2"},{"comment":"The observation table lists 'Sampled skill' with dimension 1, but the text says z is sampled from N(0,I) and is used in the diversity reward as a vector. Please clarify the skill dimension used in experiments and whether z is scalar or vector.","section":"Table 3"},{"comment":"The Div-only baseline is given the robot's base x position as an additional input to the skill discovery algorithm, while the other methods are not described as receiving this input. Please state why this extra input is needed for Div-only and how it affects the comparison.","section":"Section 4.1"},{"comment":"The real-robot experiments are reported qualitatively via figures and video. To support the transfer claim, please report the number of trials and the success count for each hardware experiment.","section":"Section 4.5"},{"comment":"The derivation defines θ′ as the parameter after a single gradient step, but Algorithm 1 uses PPO with multiple epochs. Please clarify the exact update used when computing the surrogate gradient in practice.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The empirical core is interesting and the hardware transfer is a valuable contribution, but the Appendix A derivation is formally incorrect and the framing overstates the autonomy of the method given the manual sub-dimension selection and the wall-jump guideline. I believe these points can be fixed in revision without changing the experimental design, so major revision rather than rejection seems appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on 2508.08982. The paper shows that combining a METRA-style diversity reward with a learned balancing weight (lambda) lets a quadruped learn leap, climb, crawl, and wall-jump without demos or explicit curricula. The wall-jump is the real show: it requires running, a front-flip onto a wall, a kick-off, and a landing—a genuinely hard parkour skill. They report five-seed training curves, fixed-lambda comparisons, and real-robot videos, so the central empirical claim is credible.\n\nWhat's actually new: the lambda adaptation via the bi-level update (borrowed from LIRPG but applied to a fixed diversity reward) and the observation of 'positive collapse'—skills converge so that random skill sampling works at test time. That last bit is practically useful and not stated clearly elsewhere. The paper is also honest about its main limitation: you have to specify the sub-dimensions of state that phi feeds on (height for crawl/climb, velocity for leap, orientation for wall-jump). That choice does most of the exploration-design work, so the 'autonomously exploring' language in the abstract oversells it. The authors admit this in Section 6, so it's not a hidden flaw, but it does cap the significance: SDAX is a good way to amplify human insight about which state dimensions matter, not a way to discover those dimensions.\n\nWhere it's softest: the derivation of Eq. (3) is sloppy. Appendix A writes expectations as if they were point estimates and treats a sample advantage as an exact gradient in the chain rule. The result is an approximation masquerading as a proof. That's fixable with a clearer statement of the estimator and its bias, but as written it will draw fire. There's also no code release and the hardware results are qualitative—fine for a conference paper, but the claims aren't independently checkable yet.\n\nThe stress-test note lands: if you omit the right state dimension, lambda can't push exploration along it, and the method reduces to task-only training, which they show fails on the wall-jump. So the four successes are demonstrations of task-specific feature selection plus a generic diversity objective.\n\nBottom line: this is a solid incremental contribution, not a field-reorganizing one. It deserves a serious peer review; I'd expect a conditional accept after the derivation is cleaned up and the framing is toned down. I'd probably cite the lambda-learning trick and the collapse analysis. Take a look if you work on exploration for legged locomotion.","headline":"The combination works and the wall-jump is real, but the 'autonomous exploration' framing oversells it—you still have to hand-pick which state dimensions matter.","tokens_in":11892,"tokens_out":2470,"would_cite":true,"duration_ms":26458,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simple task reward plus unsupervised skill discovery can train a quadruped to leap, climb, crawl, and execute a wall-jump.","keywords":["Unsupervised Skill Discovery","Exploration","Legged Locomotion","Reinforcement Learning","Quadruped Robot","Parkour","Sim-to-Real Transfer","Bi-level Optimization"],"falsifier":"Give SDAX a new obstacle whose solution depends on a state dimension absent from the manual list—say, a narrow diagonal gap that requires lateral roll when only base height and forward velocity are specified—and train several seeds with the same budget. If no skill vector consistently solves the task, the claim that SDAX reduces human engineering effort for arbitrary obstacles is refuted.","tokens_in":10971,"feed_emoji":"🤖","tokens_out":9441,"duration_ms":89837,"temperature":0.7,"pith_summary":"The paper demonstrates that a quadruped can learn four agile obstacle-crossing behaviors—leaping, climbing, crawling, and a wall-jump—using only a simple task reward plus an unsupervised skill-discovery reward, with no demonstrations, curricula, or careful reward shaping. The key claim is that diversity-based exploration, rather than state-level novelty, is what lets the policy find the required heights, velocities, and orientations. A learned balancing parameter automatically schedules exploration: high diversity pressure early, more task focus later. If correct, this removes a significant human-engineering bottleneck in agile locomotion and points to skill discovery as a generic exploration primitive for robot learning.","feed_headline":"No demos or curricula: quadruped learns leap, climb, crawl, wall-jump","feed_subtitle":"A learnable exploration weight balances task and diversity rewards so one quadruped masters four agile skills and transfers to real…","key_machinery":"The load-bearing mechanism is the combination of a skill-conditioned policy $\\pi_\\theta(a|s,z)$ with a learnable balancing parameter $\\lambda$ that weights a diversity reward against a task reward. The diversity reward is taken from METRA, $r^{\\mathrm{div}} = (\\phi(s_{t+1})-\\phi(s_t))^\\top z$, with $\\phi$ a state representation constrained to be 1-Lipschitz, so maximizing it encourages different latent skills $z$ to visit different states. The balancing parameter is updated by a bi-level gradient rule, Equation (3), which increases $\\lambda$ when the task-advantage gradient and diversity-advantage gradient agree and decreases it when they conflict. What makes the scheme practical is 'positive collapse': because all skills share one policy network, once task rewards select a successful behavior, an increasing fraction of randomly sampled skills adopts it, so no skill-selection mechanism is needed at deployment.","core_discovery":"The paper's central claim is that the SDAX framework—a skill-conditioned policy trained on $r^{\\mathrm{task}} + \\lambda r^{\\mathrm{div}}$ with $\\lambda$ adjusted by a bi-level gradient rule—can learn a quadruped's agile obstacle-crossing skills, including leaping a 48 cm gap, climbing a 25 cm platform, crawling under a 29 cm obstacle, and performing a wall-jump, without demonstration data or a manually designed curriculum. The task reward is deliberately simple (forward velocity tracking plus mild penalty terms), and the diversity reward comes from unsupervised skill discovery, either METRA or DIAYN. The adaptive $\\lambda$ is what makes this work: it starts high to encourage broad exploration and then decreases as task reward becomes the dominant signal, outperforming all fixed values of $\\lambda$ tested on the leap task. After a short fine-tuning phase with observation noise and domain randomization, the learned policies transfer to a real Unitree A1 robot.","pith_inferences":["Editorial inference: The manual sub-dimension requirement means SDAX is best framed as replacing reward or curriculum engineering with a cheaper but still human-provided 'exploration dimension' choice; an obvious extension is to learn these dimensions automatically from the task reward itself.","Editorial inference: Because the benefit of METRA over DIAYN is attributed to non-saturating diversity, SDAX should benefit from any future skill-discovery method with a non-saturating metric, making the framework modular.","Editorial inference: The positive-collapse effect suggests SDAX could be adapted to multi-task or open-ended settings where a single shared policy is fine-tuned on successive obstacles, with the diversity reward re-seeding exploration each time."],"forward_implications":["On the three obstacle tasks, SDAX with METRA learns to clear all obstacles while the task-only baseline does not, showing diversity reward is necessary for these skills.","Adaptively training $\\lambda$ beats every fixed value tested (0.01, 0.1, 1, 10) on leaping, with better sample efficiency and final performance.","As training progresses, the fraction of random skill vectors that solve each task rises (for leaping, from roughly 43% to 97%), so deployment needs no skill-selection module.","The same framework, with the robot's roll, pitch, and yaw added to the diversity objective, learns a wall-jump that a task-only policy cannot achieve.","After fine-tuning with observation noise and domain randomization, the learned policies transfer to real hardware for leaping, climbing, and crawling."],"supporting_citations":[{"why":"Supplies the diversity reward formula and the 1-Lipschitz representation constraint that produce $r^{\\mathrm{div}}$, the core exploration signal.","marker":"[21]"},{"why":"Provides the alternative skill-discovery reward, DIAYN, used to show SDAX is agnostic to the choice of discovery method.","marker":"[18]"},{"why":"Supplies the bi-level gradient template that SDAX adapts to train the balancing parameter $\\lambda$.","marker":"[31]"},{"why":"Defines the leap, climb, and crawl tasks and the hand-designed-reward baseline that SDAX is compared against.","marker":"[9]"},{"why":"Provides the massively parallel PPO training codebase and infrastructure on which SDAX is built.","marker":"[34]"},{"why":"The policy-gradient algorithm used to optimize the skill-conditioned policy.","marker":"[35]"},{"why":"The novelty-based exploration baseline, RND, that SDAX outperforms in the comparisons.","marker":"[36]"},{"why":"The domain-randomization technique used to transfer the learned policy from simulation to real hardware.","marker":"[39]"}],"fun_headline_variants":["Self-tuned exploration: quadruped masters leap, climb, crawl, wall-jump","No demos or curricula: adaptive skill discovery yields agile quadruped","Agile quadruped learns four obstacles with unsupervised skill discovery","Adaptive exploration weight lets quadruped learn wall-jump from scratch","Jump off walls, climb, crawl: one robot, no hand-crafted curriculum"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"SDAX's success rests on a human first specifying which sub-dimensions of the robot state the diversity reward should explore; if a task needs exploration along a dimension the designer did not list, the framework has no mechanism to discover that dimension.","fun_headline_variants_meta":{"raw":{"variants":["Self-tuned exploration: quadruped masters leap, climb, crawl, wall-jump","No demos or curricula: adaptive skill discovery yields agile quadruped","Agile quadruped learns four obstacles with unsupervised skill discovery","Adaptive exploration weight lets quadruped learn wall-jump from scratch","Jump off walls, climb, crawl: one robot, no hand-crafted curriculum"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1360,"prompt_tokens":901,"completion_tokens":459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":363}},"tokens_in":517,"tokens_out":459,"duration_ms":5286,"temperature":1.0,"reasoning_tokens":363,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:31:46.544373+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give SDAX a new obstacle whose solution depends on a state dimension absent from the manual list—say, a narrow diagonal gap that requires lateral roll when only base height and forward velocity are specified—and train several seeds with the same budget. If no skill vector consistently solves the task, the claim that SDAX reduces human engineering effort for arbitrary obstacles is refuted.","supporting_citations":[{"cited_title":"Zheng, J","cited_arxiv_id":null,"evidence_quote":"Supplies the bi-level gradient template that SDAX adapts to train the balancing parameter $\\lambda$."}],"review_version":2}