{"id":"773326d9-7f65-4889-b5d8-8d4526f21b2b","arxiv_id":"2507.20382","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper trains a quadrupedal robot to walk bipedally using a risk-adaptive distributional reinforcement learning method, and demonstrates front-leg manipulation in simulation and on a Unitree Go2.","lead":"Quadruped robots can walk on their hind legs with a reinforcement-learned policy that treats risk as adjustable, freeing the front legs to push carts, probe obstacles, and carry payloads. If the approach works as reported, it offers a cheaper way to make legged robots manipulate objects without bolting on an arm.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported advantage of risk-adaptive DPPO could stem from the deterministic time-decay in Eq. (8); no ablation removes the coefficient-of-variation coupling to test whether the uncertainty signal itself matters.","rationale":"The reader's identified weakest assumption is precisely the unsupported reliance on the CV-based adaptation law in Eq. (8) and the hand-chosen alpha endpoints, with no ablation separating the CV dependence from a time-only decay. My independent reading of the manuscript reaches the same conclusion: this is the most load-bearing gap. The paper's simulation results show plausible improvements over fixed-risk baselines, but the causal mechanism for that improvement is not established. A purely time-decaying risk schedule (from conservative to optimistic) could plausibly produce the same training behavior, especially since CVt is monotonically related to training progress in typical RL settings. The external-force experiment in Table V also has very large standard errors (e.g., 0.601 ± 0.418 for the adaptive method), which further weakens the robustness claim, but that is secondary to the core novelty. The real-world deployment is qualitative, as the reader notes, and provides supporting but not decisive evidence. The absence of code or a full configuration file also hampers independent verification. These issues are addressable in a revision and do not require rejection outright, so the appropriate verdict remains CONDITIONAL, matching the reader's verdict. The proposed concrete test directly targets the load-bearing assumption: it would empirically isolate whether the CV term contributes beyond annealing, and whether the chosen endpoints are robust. If the time-only ablation performs comparably, the paper's main claim would need to be substantially reframed; if the adaptive version clearly outperforms, the concern would be resolved.","tokens_in":13436,"tokens_out":6806,"duration_ms":73215,"concrete_test":"Run the identical DPPO training procedure with alpha_t = (alpha0 - alphaT) e^{-t/T} + alphaT, i.e., set CVt = 1 for all t, keeping alpha0 = 0 and alphaT = -0.2 and all other hyperparameters fixed. Compare the velocity-tracking reward and total reward curves against the DPPO-adaptive results in Figure 3. As a second control, replace CVt in Eq. (8) with a random permutation of the recorded CVt trajectory from the original adaptive run. If either variant matches the adaptive performance, the coefficient-of-variation signal is not responsible for the reported advantage. Additionally, rerun the adaptive method with alphaT = -0.1 and -0.3 to test sensitivity of the result to the hand-chosen endpoint.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty is the risk-adaptation law in Eq. (8): alpha_t = (alpha0 - alphaT) e^{-t/T} CVt + alphaT. The coefficient of variation CVt is positive and generally decreases during training as returns grow and the critic improves, so the adaptive schedule is strongly correlated with a purely time-dependent annealing from risk-neutral to risk-seeking. The paper compares against fixed-risk baselines only (alpha = 0, 0.2, -0.2), not against a time-only schedule alpha_t = (alpha0 - alphaT) e^{-t/T} + alphaT or a version with CVt replaced by a scrambled signal. Consequently, the headline claim in Section V-A that 'our method consistently outperforms the baselines' is not attributable to the uncertainty-adaptive mechanism. The endpoints alpha0 = 0 and alphaT = -0.2 are hand-chosen and no sensitivity analysis is reported, so even the annealing component could be tuned to favor the method. Because the risk-adaptive mechanism is the paper's primary contribution, the lack of an ablation isolating the CV term is a load-bearing gap: if the CV coupling is irrelevant, the method reduces to a risk annealing schedule and the claimed novelty is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes risk-adaptive distributional PPO (DPPO) for quadrupedal robots walking bipedally, with the goal of freeing the front legs for loco-manipulation. The risk parameter alpha in Wang's distortion is updated online according to Eq. (8) as a function of the batch-averaged coefficient of variation (CV) of the critic's predicted return distribution and an exponential time-decay term. The method is evaluated in Isaac Gym against fixed-risk DPPO (neutral, averse, seeking) and standard PPO, with reported improvements in training reward, velocity-tracking error, success rate, and robustness to external forces. The policy is also deployed on a Unitree Go2 to demonstrate cart pushing, obstacle probing, and payload carrying.","tokens_in":13629,"tokens_out":4392,"duration_ms":48030,"significance":"If the risk-adaptive mechanism is genuinely responsible for the reported gains, the paper makes a useful contribution to legged-locomotion RL by replacing manual risk-level selection with an automatically adjusted risk preference, and the real-world bipedal loco-manipulation demos are interesting. The paper includes four baselines, a reward-function ablation, and out-of-distribution velocity evaluation, which are positive elements. However, the central novelty claim hinges on the CV signal in Eq. (8), and the paper currently lacks the control experiment needed to separate that signal from a pure time-decay schedule. The external-force result in Table V is also statistically weak. These issues are fixable with additional experiments, so the paper is not beyond repair.","major_comments":[{"comment":"The paper does not ablate the coefficient-of-variation term CV_t against a pure time-dependent schedule. In Eq. (8), alpha_t = (alpha0-alphaT)e^{-t/T} CV_t + alphaT, and since CV_t generally decreases during training as the critic improves, alpha_t is strongly correlated with time. The comparison against fixed-risk baselines (alpha = 0, 0.2, -0.2) therefore cannot establish that the uncertainty signal, rather than the exponential annealing, causes the reported advantage. A time-only baseline alpha_t = (alpha0-alphaT)e^{-t/T} + alphaT, or a version with CV_t replaced by a constant or a scrambled signal, is necessary to support the claim that risk adaptiveness is the source of improvement. This is load-bearing because the adaptive risk law is the paper's primary contribution.","section":"Section V-A and Eq. (8)"},{"comment":"The external-force success-rate numbers do not support the stated conclusion. DPPOadaptive achieves 0.601 ± 0.418 and DPPOaverse achieves 0.327 ± 0.268; these intervals overlap heavily, so the claim that 'our method achieves the highest success rate, nearly doubling the second-best' is not statistically supported. The paper should report the number of seeds, per-seed values, and a proper confidence interval or statistical test. As written, this table undermines the robustness claim rather than supporting it.","section":"Table V"},{"comment":"The endpoints alpha0 = 0 and alphaT = -0.2 are hand-chosen, and no sensitivity analysis is provided. Because the fixed-risk seeking baseline is set to alpha = -0.2, matching the final adaptive value, the reported advantage could be an artifact of a well-tuned annealing schedule rather than of adaptation to uncertainty. A sweep over (alpha0, alphaT), or at least a comparison with several time-only schedules, is needed to rule out tuning effects.","section":"Section V-A (hyperparameter selection)"},{"comment":"The real-world evaluation is entirely qualitative. No task success rates, cart masses, payload weights, disturbance magnitudes, or quantitative tracking errors are reported for cart pushing, obstacle probing, or payload carrying. Since the paper lists 'robust real-world applications' as a main contribution, the current video-based evidence is anecdotal. Quantitative measurements from a small number of repeated trials would make the real-world claim commensurate with the simulation claims.","section":"Section V-D"}],"minor_comments":[{"comment":"The text says 'Generalizable Advantage Estimation'; the standard term is 'Generalized Advantage Estimation', and the ratio expression contains a typo ('at|ott').","section":"Equation (2)"},{"comment":"The caption says 'Coefficient of Variance'; the correct term is 'Coefficient of Variation'.","section":"Figure 7 caption"},{"comment":"Eq. (8) uses the batch-averaged CV (denoted CV_t), but Figure 7 plots a per-timestep CV during evaluation. The relationship between these two quantities and the way alpha is applied at deployment should be clarified.","section":"Section IV-C and Figure 7"},{"comment":"The statement that 'all methods share the same hyperparameters if applicable' is vague because the adaptive method has additional hyperparameters (alpha0, alphaT, and time constant T); the paper should state which hyperparameters are shared and which are method-specific.","section":"Table II and Section V-A"},{"comment":"Several numbers contain stray spaces (e.g., '0 .337', '64 .11%'), and the 'Success Rate Drop' column should explicitly define the reference success rate used to compute the drop.","section":"Tables IV and V"},{"comment":"Reference [12] appears to have an incomplete title; please verify the bibliographic entry.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is plausible but currently lacks the key control experiment isolating the CV-driven adaptation from a time-only decay, and the external-force table is statistically weak. If the authors can supply the missing ablation and report proper uncertainty in Table V, the paper could become acceptable for publication; otherwise the central novelty claim remains unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nWhat you should know: this is a solid empirical paper with a real gap in its central claim. The new piece is training a quadruped to walk on hind legs with a single risk-adaptive distributional PPO, where alpha is adjusted by the coefficient of variation of the critic's return distribution, and then using the freed front legs for cart pushing, probing, and carrying on a Unitree Go2. That demonstration is genuinely useful, and the paper is honest about what it does and does not show.\n\nWhat it does well: the components are prior art, but the pairing is new. It cites Schneider, Liu, Shi, and the bipedal locomotion works appropriately. The reward ablation (support polygon and upright balance) is a nice touch; it shows those terms matter. Training curves are plausible, three seeds, and the real-world video presumably carries weight in this community. The external-force table, though noisy, at least attempts to measure robustness.\n\nSoft spots, in order of severity.\n\nFirst, the adaptation law in Eq. (8) multiplies the exponential time decay by CV_t. Since CV_t generally falls as training progresses, the schedule is strongly confounded with a pure time-annealing from risk-neutral to risk-seeking. The paper compares only against fixed-risk baselines. There is no ablation with the CV term replaced by a constant or a scrambled signal, so the headline “consistently outperforms baselines” cannot be attributed to the uncertainty-adaptive mechanism. That matters because this mechanism is the main contribution. If it reduces to a hand-tuned annealing schedule, the novelty is thin. I think the stress-test is right on this, and it is the load-bearing gap.\n\nSecond, the external-force success rate for the proposed method is 0.601 with a 0.418 standard error. That is a huge error bar; the “nearly doubling” the second-best claim is not statistically solid. Third, real-world results are qualitative only. No measured success rates, speeds, or forces. Common for hardware demos, but worth flagging. Fourth, alpha0=0 and alphaT=-0.2 are hand-chosen with no sensitivity analysis. Fifth, no code or config release, though the training framework is built on [54].\n\nThe circularity concern is minor; using the same critic for the adaptation signal and the advantage is a control loop, not evidence that assumes the conclusion, and the performance comparison is independent.\n\nVerdict: this deserves peer review. The gap is addressable — add a time-only annealed baseline and a shuffled-CV ablation, tighten the external-force statistics, and the paper would be convincing. For a reading group, it would spark good discussion about what counts as an uncertainty-adaptive mechanism. I would cite it as a reference for risk-aware legged locomotion, but with a “see also” caveat until the ablation is done.\n\nRecommendation: send it to review, but reviewers should push hard on the ablation.","headline":"Useful real-world bipedal loco-manipulation demo and a plausible training study, but the central uncertainty-adaptive claim is undercut by a missing ablation that would separate the CV signal from a time-decaying risk schedule.","tokens_in":14231,"tokens_out":2378,"would_cite":true,"duration_ms":24976,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A quadruped robot with a risk-adaptive reinforcement-learning policy walks on its hind legs and uses the freed front legs to push carts, probe obstacles, and carry payloads, in simulation and on a real Unitree Go2.","keywords":["bipedal locomotion","quadrupedal robot","loco-manipulation","distributional reinforcement learning","risk-adaptive policy","coefficient of variation","sim-to-real transfer","Unitree Go2"],"falsifier":"Train the same distributional PPO with Eq. (8) replaced by a time-only schedule from $\\alpha_0=0$ to $\\alpha_T=-0.2$, keeping all other hyperparameters and seeds identical; if this ablation matches or exceeds the reported velocity-tracking reward and success rate, the coefficient-of-variation term is not the cause of the improvement. Varying $\\alpha_T$ across $-0.1$, $-0.2$, and $-0.4$ and checking whether the margin over fixed-risk baselines persists would also show whether the result depends on one hand-picked endpoint.","tokens_in":13187,"feed_emoji":"🦿","tokens_out":9994,"duration_ms":90275,"temperature":0.7,"pith_summary":"A quadruped robot can walk on its hind legs and use its freed front legs for real manipulation if the reinforcement-learning agent adjusts its own risk preference during training instead of committing to a fixed risk level. The proposed method, risk-adaptive distributional PPO, estimates the distribution of future returns with quantile regression and sets the risk parameter from the coefficient of variation of that distribution, becoming conservative when uncertainty is high and optimistic when it is low. In simulation the adaptive policy beats fixed-risk baselines in velocity tracking and total reward, and the same single policy transfers to a real Unitree Go2 robot for cart pushing, obstacle probing, and payload carrying. The paper argues that this uncertainty-driven risk adaptation is what makes the inherently unstable bipedal gait stable enough for loco-manipulation.","feed_headline":"Risk-adaptive RL gets a quadruped walking on two legs","feed_subtitle":"A single policy also pushes carts, probes obstacles, and carries payloads on real hardware.","key_machinery":"The load-bearing mechanism is the risk-adaptive distortion of the critic's return distribution. The critic predicts $N$ quantiles $\\theta_i(x)$ of the return distribution, and the value used for advantage estimation is computed through Wang's distortion function $g_\\alpha(\\tau)=\\Phi(\\Phi^{-1}(\\tau)+\\alpha)$ instead of the plain mean. The distortion parameter is updated at each training step by $\\alpha_t=(\\alpha_0-\\alpha_T)e^{-t/T}\\mathrm{CV}_t+\\alpha_T$, where $\\mathrm{CV}_t$ is the batch-averaged coefficient of variation of the predicted return distribution. This couples risk preference to the critic's uncertainty: high uncertainty biases the value estimate toward worst-case outcomes, while low uncertainty allows optimism, which the authors argue encourages exploration without sacrificing stability.","core_discovery":"The central claim is that adapting the risk level during training to the critic's estimated return uncertainty, rather than presetting it, produces a bipedal gait policy that is both performant and transferable. The agent learns a distribution over returns with quantile regression, applies a distortion risk measure $\\rho_{g_\\alpha}$ to obtain a risk-adjusted value, and sets the distortion parameter by a law that makes the policy more conservative when the batch-averaged coefficient of variation of the predicted return distribution is high and more optimistic as training progresses. The authors report that this risk-adaptive distributional PPO consistently outperforms fixed-risk distributional PPO variants and standard PPO in velocity tracking and total reward, and that the trained policy transfers to real hardware for cart pushing, contact-aware obstacle probing, and payload carrying without task-specific retraining.","pith_inferences":["A direct ablation study that removes the coefficient-of-variation term from Eq. (8), leaving only the time-based schedule, would clarify whether the uncertainty adaptation itself rather than the schedule drives the reported gains.","The same uncertainty-adaptive risk mechanism could transfer to other unstable loco-manipulation settings, such as carrying a payload while climbing stairs or standing on one support leg, where the critic's return uncertainty may track destabilization.","A natural extension is to adapt the endpoint values $\\alpha_0$ and $\\alpha_T$ themselves, or to use a per-state coefficient of variation instead of a batch average, which could remove the need to hand-pick these constants for each new task.","Because the real-world demonstrations are qualitative, a quantitative benchmark of cart-pushing force, obstacle-probing success rate, and payload-carrying stability would allow direct comparison with arm-equipped or single-leg-manipulation baselines."],"forward_implications":["Because risk preference is internalized during training, deployment needs no manual risk-level tuning at test time.","A fixed risk-seeking policy fails catastrophically, while the adaptive policy keeps exploration alive and avoids those failures.","The policy retains higher success rates and lower tracking error than baselines under external forces, and generalizes to out-of-distribution velocity commands.","Three real-world loco-manipulation tasks—cart pushing, contact-aware obstacle probing, and payload carrying—are achieved by a single bipedal locomotion policy without task-specific retraining."],"supporting_citations":[{"why":"Provides the risk-aware distributional PPO baseline with fixed risk levels that the proposed adaptive method is compared against.","marker":"[45]"},{"why":"Supplies Wang's distortion function that turns the risk parameter into conservative or optimistic value estimates.","marker":"[56]"},{"why":"Gives the quantile-regression critic that approximates the return distribution with $N$ quantiles.","marker":"[50]"},{"why":"Provides the clipped PPO objective used to update the actor policy.","marker":"[27]"},{"why":"Supplies the massively parallel simulation and reward framework that the authors adapt for bipedal locomotion.","marker":"[54]"},{"why":"Provides the GPU physics simulator used for training with thousands of parallel agents.","marker":"[57]"},{"why":"Supplies generalized advantage estimation, used to compute the advantage for the actor update.","marker":"[55]"}],"fun_headline_variants":["Quadruped walks on hind legs, front legs free for manipulation","Risk-adaptive RL turns a quadruped into a biped with free limbs","Bipedal quadruped: adaptive risk policy enables cart pushing and more","Front legs freed: quadruped learns bipedal walk for real-world tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the average spread of the critic's predicted return, divided by its mean, is a trustworthy signal for how cautious or optimistic the policy should be, and that the hand-chosen endpoints $\\alpha_0=0$ and $\\alpha_T=-0.2$ work across tasks; if the signal is noisy or the constants are task-specific, the reported advantage could be an artifact of tuning rather than of uncertainty adaptation.","fun_headline_variants_meta":{"raw":{"variants":["Quadruped walks on hind legs, front legs free for manipulation","Risk-adaptive RL turns a quadruped into a biped with free limbs","Bipedal quadruped: adaptive risk policy enables cart pushing and more","Front legs freed: quadruped learns bipedal walk for real-world tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1407,"prompt_tokens":875,"completion_tokens":532,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":452}},"tokens_in":491,"tokens_out":532,"duration_ms":5485,"temperature":1.0,"reasoning_tokens":452,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:44:26.257310+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same distributional PPO with Eq. (8) replaced by a time-only schedule from $\\alpha_0=0$ to $\\alpha_T=-0.2$, keeping all other hyperparameters and seeds identical; if this ablation matches or exceeds the reported velocity-tracking reward and success rate, the coefficient-of-variation term is not the cause of the improvement. Varying $\\alpha_T$ across $-0.1$, $-0.2$, and $-0.4$ and checking whether the margin over fixed-risk baselines persists would also show whether the result depends on one hand-picked endpoint.","supporting_citations":[{"cited_title":"A class of distortion operators for pricing financial and insurance risks,","cited_arxiv_id":null,"evidence_quote":"Supplies Wang's distortion function that turns the risk parameter into conservative or optimistic value estimates."},{"cited_title":"Distributional reinforcement learning with quantile regression,","cited_arxiv_id":null,"evidence_quote":"Gives the quantile-regression critic that approximates the return distribution with $N$ quantiles."}],"review_version":2}