{"id":"3018f40a-e029-4f24-bf41-2aa9d937fcfc","arxiv_id":"2508.08240","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A single framework plans long-horizon tasks with GPT-4.1 and Qwen2.5-VL while one reinforcement-learned policy controls all 18 robot joints, evaluated on a new benchmark with limited qualitative sim-to-real demos.","lead":"ODYSSEY combines a vision-language-model task planner with a learned whole-body controller so a quadruped robot with an arm can follow long, multi-step instructions in indoor and outdoor scenes. The paper also introduces a simulation benchmark and reports qualitative real-robot transfers, but it provides no quantitative real-world success rates or released code.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-world validation covers only two isolated skill demos, so the sim-to-real generalization claim for long-horizon tasks is not established.","rationale":"The reader's weakest assumption focuses on whether the simulation benchmark is a faithful proxy for open-world conditions, with the two qualitative real-robot demos as supporting evidence. My concern is narrower and more direct: even granting full fidelity of the simulation benchmark, the real-world experiments in Section 4.3 do not test the long-horizon tasks that define the paper's headline contribution. They exercise only 'navigate to pick' and 'pick and place', which are not among the eight tasks in Table 7 and involve only atomic or two-step skills. Therefore the central 'sim-to-real transfer' claim for long-horizon deployment is under-supported by the evidence actually presented. This is not a fatal flaw, because the simulation results and the component-level transfer are still meaningful, and the gap is addressable by either adding a quantitative long-horizon real-world evaluation or by softening the claim. The reader's conditional verdict already captures the need for stronger real-world evidence, so my analysis sharpens the specific condition without changing the verdict. I agree with the reader that the sim-to-real claim is the load-bearing one and that the current support is insufficient, but I locate the problem primarily in the mismatch between the claimed long-horizon scope and the narrow real-world task set, rather than in simulation fidelity alone.","tokens_in":17237,"tokens_out":4068,"duration_ms":44403,"concrete_test":"Deploy the physical ODYSSEY robot on at least one of the eight benchmark long-horizon tasks, such as CABINET STORAGE or CARTDELIVERY, for a minimum of 10 episodes, and report per-action and overall success rates alongside the corresponding simulation numbers. If such an experiment is infeasible, restrict the abstract and conclusion claims to 'component-level sim-to-real transfer' and present the two existing demos as pilot results.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim from the abstract, 'Through successful sim-to-real transfer, we demonstrate the system's generalization and robustness in real-world deployments,' is not supported by the real-world evidence in Section 4.3. That section reports only two tasks, 'navigate to pick' and 'pick and place', tested qualitatively with five objects. These are not among the eight benchmark long-horizon tasks listed in Table 7 (e.g., CARTDELIVERY, CABINET STORAGE, OUTDOOR DELIVERY), and they involve at most a single manipulation skill preceded by navigation, not the 2-3 subgoal compositions that define the benchmark's long-horizon setting. No quantitative real-world success rates, trial counts, or error bars are reported, and the paper itself concedes persistent failures on small objects. Consequently, the real-world data can support at most component-level transfer of the low-level controller and VLM grounding for two elementary behaviors; it does not establish that the integrated planner-controller stack generalizes to long-horizon open-world tasks. The simulation benchmark is a separate contribution, but it cannot by itself validate the sim-to-real robustness of the full system when the real-world deployment never exercises the benchmark's long-horizon task structure.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ODYSSEY, a framework for legged mobile manipulation that combines a hierarchical vision-language planner (GPT-4.1 for long-horizon decomposition over an instance-level semantic graph, Qwen2.5-VL for geometry-constrained end-effector pose generation) with a two-stage reinforcement-learned whole-body controller for a quadruped with a mounted arm. The authors also introduce a simulation benchmark with eight long-horizon indoor/outdoor tasks and 246/58 variations, alongside four short-horizon tasks migrated from ARNOLD. Experiments report benchmark success rates, a comparison against PerAct on ARNOLD tasks and against RoboDuet on control tracking metrics, and qualitative real-world deployments of two tasks with five objects. The central claim is that the system achieves successful sim-to-real transfer and demonstrates generalization and robustness in real-world deployments.","tokens_in":17389,"tokens_out":4514,"duration_ms":45621,"significance":"If the claims hold, ODYSSEY would be a valuable integrated system for legged loco-manipulation, and its benchmark would be a useful resource for the community. The paper ships a substantial benchmark with many task variations, reports detailed per-action success rates, and provides an extensive failure analysis. The use of external baselines (PerAct, RoboDuet) and an external benchmark (ARNOLD) is a strength, and the two-stage curriculum with terrain-invariant end-effector sampling is a reasonable design. However, the load-bearing claim of sim-to-real generalization is currently not supported by the evidence in the manuscript, and there are unresolved inconsistencies in the reported control metrics that need addressing before the central claims can be accepted.","major_comments":[{"comment":"The claim 'Through successful sim-to-real transfer, we demonstrate the system's generalization and robustness in real-world deployments' is not supported by the evidence. Section 4.3 reports only two tasks, 'navigate to pick' and 'pick and place', tested qualitatively with five objects. These are not among the eight long-horizon benchmark tasks listed in Table 7, and they involve at most one manipulation skill preceded by navigation. No quantitative success rates, trial counts, or error bars are reported, and the text concedes that the robot 'occasionally failed at grasping small objects.' Consequently, the real-world experiments can support component-level transfer of the low-level controller and VLM grounding for two elementary behaviors, but they cannot validate the integrated planner-controller stack for long-horizon tasks. Please either add quantitative real-world evaluations on the benchmark's long-horizon tasks or substantially weaken the abstract and contribution statements.","section":"Section 4.3 and Abstract"},{"comment":"The reported end-effector tracking errors (Dpos ≈ 10.57–11.48 and Dori ≈ 46.93–47.53) are extraordinarily large for a manipulation controller, especially if the units are centimeters and degrees. A 47° orientation error would make the 'precise end-effector control' claim and the high pick/place success rates in Table 2 difficult to reconcile. The text only states that performance is 'comparable' to RoboDuet, but it does not explain how a controller with these average errors can achieve the fine manipulation reported elsewhere. Please clarify the units, the protocol (whether errors are averaged over random commanded targets, including potentially unreachable ones), and how the pick success rates are compatible with these error magnitudes.","section":"Section 4.2, Table 3"},{"comment":"The comparison to PerAct is presented as 'substantial overall improvements,' but in the Seen/Test condition ODYSSEY is substantially worse on PICKUP OBJECT (60.45 vs. 94.03 in Table 1 and Table 8). The advantage is driven entirely by novel splits, which is expected given that ODYSSEY uses a strong pretrained VLM (Qwen2.5-VL-72B) while PerAct is trained on ARNOLD demonstrations. Moreover, the ARNOLD tasks were modified (spatial layout and robot platform changed) to fit the quadruped platform. Please report aggregate mean and variance over splits, describe whether PerAct was retrained on the modified tasks or evaluated under domain shift, and discuss the fairness and interpretability of this baseline comparison.","section":"Section 4.1, Tables 1 and 8"},{"comment":"The benchmark's end-to-end evaluation is unclear about the information available to the planner. The long-horizon results in Table 2 are obtained by running the full pipeline, but it is not specified whether the instance-level semantic graph is built online from raw sensor data (as described in Appendix A.1) or from ground-truth annotations in the simulator. If the scene graph contains oracle information, the reported task success rates overestimate the performance of the perception-planning loop. Please explicitly state whether the graph is constructed online during evaluation and, if not, report an ablation with the online perception pipeline.","section":"Section 3.3 and Section 4.1"}],"minor_comments":[{"comment":"The phrases 'first comprehensive benchmark' and 'first whole-body control policy' are strong claims that are not qualified against existing work such as WildLMA (Qiu et al. 2024b) or SLIM; consider softening these claims or adding explicit comparisons.","section":"Abstract and Section 1"},{"comment":"Equation (3) defines st = (qt, ṑ), but the text says st ∈ R^36; the notation should be consistent, e.g., st = (qt, ṑt).","section":"Section 3.2, Eq. (3)"},{"comment":"The phrase 'as illusion in Fig. 4' should be 'as illustrated in Fig. 4'.","section":"Section 4.3"},{"comment":"The abbreviations P.OBJECT, R.OBJECT, O.CABINET, and C.CABINET are not expanded in the caption; please add the full task names for readability.","section":"Table 1"},{"comment":"The randomization 'method' column uses 'add', 'scale', and 'interval', but 'interval' is undefined; please explain this term or replace it with a clearer description.","section":"Appendix B.2, Table 6"},{"comment":"The main text's Table 1 reports a unified 'Novel' column, but Appendix D.1 explains that this averages over four different splits. This aggregation should be described in the main text to avoid misleading readers into thinking a single novel-split protocol was used.","section":"Appendix D.1"}],"recommendation":"major_revision","confidential_remarks":"The paper presents a useful benchmark and an interesting integrated system, but the real-world validation is far too thin to support the headline sim-to-real claim, and the reported end-effector tracking errors raise questions about internal consistency between the low-level controller evaluation and the high-level manipulation success rates. The PerAct comparison may also draw criticism because the tasks were modified and the baseline was not retrained. I would suggest the editor ask for either substantial strengthening of the real-world evaluation (quantitative results on long-horizon tasks) or a clear reframing of the paper's claims as component-level transfer plus a simulation benchmark."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline: this is a competent systems integration paper that assembles known pieces into a working legged-manipulator stack and adds a benchmark. The real-world evidence, however, is too thin to support the abstract's claim of \"successful sim-to-real transfer\" for long-horizon tasks.\n\nWhat's actually new: the terrain-invariant world-frame end-effector sampling, the geometry-constrained VLM pose generation, and the ODYSSEY benchmark with eight long-horizon tasks and 304 variations. The simulation results on converted ARNOLD tasks show consistent gains over PerAct, especially on novel splits, and the whole-body policy tracks base velocities far better than RoboDuet under dynamic conditions. The failure analysis is honest and useful; the paper admits specific weaknesses in VLM grounding and control accuracy.\n\nWhere the soft spots are: Section 4.3 reports only two real-world tasks (navigate-to-pick and pick-and-place) with five objects, no quantitative success rates, no trial counts, no error bars. Those are not the eight long-horizon benchmark tasks, so the sim-to-real claim is only demonstrated for isolated skills, not for the integrated planner-controller on long-horizon tasks. The paper itself concedes persistent failures on small objects and names robust perception and high-precision control as primary challenges, which further undercuts the strong abstract claim. Additionally, no code or benchmark assets are released, which limits reproducibility, and the long-horizon benchmark has no external baseline, making the absolute numbers hard to interpret. The policy evaluation also lacks seeds or standard deviations. These are all addressable, but they matter.\n\nThe stress-test note is on target: the real-world section does not exercise the benchmark's long-horizon structure, so it cannot validate the full-system generalization claim.\n\nThat said, the core engineering is sound, the simulation evidence is substantial, and the benchmark is a potentially useful community resource. This is exactly the kind of paper that deserves a serious referee, but with a clear request for major revision: either significantly expand the real-world evaluation to include long-horizon tasks with quantitative metrics, or soften the sim-to-real claim to match the actual demonstrations. I would also ask for code and benchmark release. Send it to review.","headline":"A solid systems paper with a useful benchmark, but the real-world validation is too thin to back the abstract's sim-to-real generalization claim.","tokens_in":18047,"tokens_out":1747,"would_cite":true,"duration_ms":20238,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A legged robot with an arm can carry out long-horizon language-guided tasks in open environments using one hierarchical planner and one whole-body learned policy.","keywords":["long-horizon mobile manipulation","quadruped robot","whole-body control","vision-language model","task planning","sim-to-real transfer","embodied AI benchmark","locomanipulation"],"falsifier":"Release the benchmark and check whether removing the world-frame height-fixed sampling step raises end-effector position error on slopes beyond the reported values; if it does not, the terrain-invariant sampling mechanism is not load-bearing. Similarly, deploying the policy on a slightly different quadruped-plus-arm platform and observing unstable grasps or falls on mild slopes would undercut the sim-to-real transfer claim.","tokens_in":16959,"feed_emoji":"🦾","tokens_out":6484,"duration_ms":61590,"temperature":0.7,"pith_summary":"ODYSSEY argues that long-horizon mobile manipulation for legged robots should be treated as one integrated problem, not separate navigation, planning, and manipulation modules. It proposes a hierarchical planner that uses a vision-language model to decompose language instructions into atomic actions (navigate, pick, place, push, pull, drag), grounded in an egocentric semantic map, and a single reinforcement-learned whole-body policy that drives all 18 joints of a quadruped-plus-arm robot. The paper also introduces a simulation benchmark with ten scenes, over a hundred assets, four short-horizon manipulation skills adapted from ARNOLD, and eight long-horizon tasks covering indoor and outdoor settings. The central claim is that this combined system transfers to a physical robot: two real-world deployments show successful task planning and execution, supporting the practical feasibility of legged mobile manipulators in unstructured environments.","feed_headline":"Sim-trained legged robot plans and executes long-horizon chores","feed_subtitle":"New benchmark measures long-horizon mobile manipulation, and the same system transfers to a real quadruped.","key_machinery":"The load-bearing mechanism is ODYSSEY's coarse-to-fine planner paired with a two-stage whole-body control policy. The planner builds an instance-level semantic graph from LiDAR and RGB data, uses a large language model to decompose the instruction into atomic actions with waypoints projected onto a SLAM occupancy map, and then uses a vision-language model to predict contact points and end-effector orientations under two geometric constraints: an axis-alignment constraint and a surface-normal constraint. The control policy is a single neural network mapping commands, end-effector target, terrain height map, gravity, previous action, and proprioception to 18 joint offsets, trained with a two-stage curriculum and extensive domain randomization. A distinctive part is the terrain-invariant end-effector sampling strategy, which fixes the target height in the world frame before transforming it to the robot's base frame, decoupling the arm target from base pitch and terrain height.","core_discovery":"The paper's central claim is that a single unified whole-body policy can coordinate quadruped locomotion with arm manipulation well enough to execute long-horizon, language-conditioned tasks on varied terrain, and that a coarse-to-fine planner built on LLM and VLM grounding can turn an egocentric observation stream into executable atomic actions without third-person cameras. In the ARNOLD-style short-horizon tests, ODYSSEY outperforms the PerAct baseline while using only a single egocentric camera, including on novel objects, scenes, and goal states. In the new long-horizon benchmark, it maintains at least 40 percent overall task success and over 60 percent per-action success across all eight tasks. Real-world trials on two long-horizon tasks with five different objects show transfer of both the planner and the control policy to physical hardware, which the paper takes as evidence of generalization and robustness.","pith_inferences":["The terrain-invariant end-effector sampling strategy is a transferable trick: any mobile manipulator whose base pitches on uneven ground could benefit from sampling targets in the world frame rather than the arm frame.","A direct next test is to run the same planner-and-policy on a larger set of real objects and scenes to quantify the sim-to-real gap beyond the two demonstrated tasks.","The geometry-constrained contact-point prediction could generalize beyond the six atomic actions to tool use or assembly, where end-effector orientation relative to a surface normal is equally important.","The benchmark could be extended into a cross-embodiment evaluation harness for vision-language models and mobile manipulators, a direction the paper itself flags as future work."],"forward_implications":["A single egocentric camera plus LiDAR is sufficient for language-guided long-horizon mobile manipulation, removing the need for third-person camera rigs.","The same whole-body control policy transfers from simulation to a physical quadruped-plus-arm platform without additional real-world fine-tuning.","The benchmark provides a common testbed with per-action success rates, letting researchers isolate whether failures come from reasoning, control, or navigation.","Object configurations outside the training distribution remain manageable, as shown by higher success rates than the PerAct baseline on novel object, scene, and state splits."],"supporting_citations":[{"why":"Supplies the large language model that decomposes long-horizon instructions into atomic actions.","marker":"(Achiam et al. 2023)"},{"why":"Supplies the vision-language model with pixel-level grounding that predicts contact points and end-effector orientations.","marker":"(Bai et al. 2025)"},{"why":"Provides the four ARNOLD short-horizon manipulation tasks and their goal-state evaluation protocol, which ODYSSEY adapts.","marker":"(Gong et al. 2023)"},{"why":"PerAct is the baseline model ODYSSEY compares against on short-horizon manipulation tasks.","marker":"(Shridhar, Manuelli, and Fox 2022)"},{"why":"RoboDuet is the whole-body control baseline for the locomotion and end-effector tracking comparison.","marker":"(Pan et al. 2025a)"},{"why":"Provides the action-as-joint-offset formulation and domain randomization precedent that the control policy builds on.","marker":"(Fu, Cheng, and Pathak 2023)"}],"fun_headline_variants":["Quadruped robot learns long-horizon chores in sim, then does them for real","Sim-to-real quadruped handles long-horizon tasks via language and vision","One policy drives legs and arm for long-horizon chores in open world","Legged manipulator nails long-horizon tasks after sim-to-real shift","Open-world quadruped picks up long-horizon chores with one egocentric camera"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The simulation benchmark and its randomized variations are a faithful proxy for open-world conditions, so that simulation success plus two qualitative real-world demos can support the claims of generalization and robustness.","fun_headline_variants_meta":{"raw":{"variants":["Quadruped robot learns long-horizon chores in sim, then does them for real","Sim-to-real quadruped handles long-horizon tasks via language and vision","One policy drives legs and arm for long-horizon chores in open world","Legged manipulator nails long-horizon tasks after sim-to-real shift","Open-world quadruped picks up long-horizon chores with one egocentric camera"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001115,"raw_usage":{"total_tokens":4677,"prompt_tokens":1012,"completion_tokens":3665,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":628,"completion_tokens_details":{"reasoning_tokens":3560}},"tokens_in":628,"tokens_out":3665,"duration_ms":23809,"temperature":1.0,"reasoning_tokens":3560,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:36:36.803398+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Release the benchmark and check whether removing the world-frame height-fixed sampling step raises end-effector position error on slopes beyond the reported values; if it does not, the terrain-invariant sampling mechanism is not load-bearing. Similarly, deploying the policy on a slightly different quadruped-plus-arm platform and observing unstable grasps or falls on mild slopes would undercut the sim-to-real transfer claim.","supporting_citations":[],"review_version":1}