Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Tasks

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A legged robot with an arm can carry out long-horizon language-guided tasks in open environments using one hierarchical planner and one whole-body learned policy.

desk verdict A solid systems paper with a useful benchmark, but the real-world validation is too thin to back the abstract's sim-to-real generalization claim. read the letter →

arxiv 2508.08240 v1 pith:JS73O37J submitted 2025-08-11 cs.RO cs.CV

Kaijun Wang , Liqin Lu , Mingyu Liu , Jianuo Jiang , Zeju Li , Bolin Zhang , Wancai Zheng , Xinyi Yu
show 2 more authors
Hao Chen Chunhua Shen
This is my paper · ORCID
classification cs.ROcs.CV
keywords long-horizonmobilemanipulationquadrupedrobotwhole-bodycontrolvision-languagemodeltaskplanningsim-to-realtransferembodiedAIbenchmarklocomanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ODYSSEY argues that long-horizon mobile manipulation for legged robots should be treated as one integrated problem, not separate navigation, planning, and manipulation modules. It proposes a hierarchical planner that uses a vision-language model to decompose language instructions into atomic actions (navigate, pick, place, push, pull, drag), grounded in an egocentric semantic map, and a single reinforcement-learned whole-body policy that drives all 18 joints of a quadruped-plus-arm robot. The paper also introduces a simulation benchmark with ten scenes, over a hundred assets, four short-horizon manipulation skills adapted from ARNOLD, and eight long-horizon tasks covering indoor and outdoor settings. The central claim is that this combined system transfers to a physical robot: two real-world deployments show successful task planning and execution, supporting the practical feasibility of legged mobile manipulators in unstructured environments.

What carries the argument

The load-bearing mechanism is ODYSSEY's coarse-to-fine planner paired with a two-stage whole-body control policy. The planner builds an instance-level semantic graph from LiDAR and RGB data, uses a large language model to decompose the instruction into atomic actions with waypoints projected onto a SLAM occupancy map, and then uses a vision-language model to predict contact points and end-effector orientations under two geometric constraints: an axis-alignment constraint and a surface-normal constraint. The control policy is a single neural network mapping commands, end-effector target, terrain height map, gravity, previous action, and proprioception to 18 joint offsets, trained with a two-stage curriculum and extensive domain randomization. A distinctive part is the terrain-invariant end-effector sampling strategy, which fixes the target height in the world frame before transforming it to the robot's base frame, decoupling the arm target from base pitch and terrain height.

What would settle it

Release the benchmark and check whether removing the world-frame height-fixed sampling step raises end-effector position error on slopes beyond the reported values; if it does not, the terrain-invariant sampling mechanism is not load-bearing. Similarly, deploying the policy on a slightly different quadruped-plus-arm platform and observing unstable grasps or falls on mild slopes would undercut the sim-to-real transfer claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that a single unified whole-body policy can coordinate quadruped locomotion with arm manipulation well enough to execute long-horizon, language-conditioned tasks on varied terrain, and that a coarse-to-fine planner built on LLM and VLM grounding can turn an egocentric observation stream into executable atomic actions without third-person cameras. In the ARNOLD-style short-horizon tests, ODYSSEY outperforms the PerAct baseline while using only a single egocentric camera, including on novel objects, scenes, and goal states. In the new long-horizon benchmark, it maintains at least 40 percent overall task success and over 60 percent per-action success across all eight tasks. Real-world trials on two long-horizon tasks with five different objects show transfer of both the planner and the control policy to physical hardware, which the paper takes as evidence of generalization and robustness.

Load-bearing premise

The simulation benchmark and its randomized variations are a faithful proxy for open-world conditions, so that simulation success plus two qualitative real-world demos can support the claims of generalization and robustness.

Editorial extensions

If this is right

  • A single egocentric camera plus LiDAR is sufficient for language-guided long-horizon mobile manipulation, removing the need for third-person camera rigs.
  • The same whole-body control policy transfers from simulation to a physical quadruped-plus-arm platform without additional real-world fine-tuning.
  • The benchmark provides a common testbed with per-action success rates, letting researchers isolate whether failures come from reasoning, control, or navigation.
  • Object configurations outside the training distribution remain manageable, as shown by higher success rates than the PerAct baseline on novel object, scene, and state splits.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The terrain-invariant end-effector sampling strategy is a transferable trick: any mobile manipulator whose base pitches on uneven ground could benefit from sampling targets in the world frame rather than the arm frame.
  • A direct next test is to run the same planner-and-policy on a larger set of real objects and scenes to quantify the sim-to-real gap beyond the two demonstrated tasks.
  • The geometry-constrained contact-point prediction could generalize beyond the six atomic actions to tool use or assembly, where end-effector orientation relative to a surface normal is equally important.
  • The benchmark could be extended into a cross-embodiment evaluation harness for vision-language models and mobile manipulators, a direction the paper itself flags as future work.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents ODYSSEY, a framework for legged mobile manipulation that combines a hierarchical vision-language planner (GPT-4.1 for long-horizon decomposition over an instance-level semantic graph, Qwen2.5-VL for geometry-constrained end-effector pose generation) with a two-stage reinforcement-learned whole-body controller for a quadruped with a mounted arm. The authors also introduce a simulation benchmark with eight long-horizon indoor/outdoor tasks and 246/58 variations, alongside four short-horizon tasks migrated from ARNOLD. Experiments report benchmark success rates, a comparison against PerAct on ARNOLD tasks and against RoboDuet on control tracking metrics, and qualitative real-world deployments of two tasks with five objects. The central claim is that the system achieves successful sim-to-real transfer and demonstrates generalization and robustness in real-world deployments.

Significance. If the claims hold, ODYSSEY would be a valuable integrated system for legged loco-manipulation, and its benchmark would be a useful resource for the community. The paper ships a substantial benchmark with many task variations, reports detailed per-action success rates, and provides an extensive failure analysis. The use of external baselines (PerAct, RoboDuet) and an external benchmark (ARNOLD) is a strength, and the two-stage curriculum with terrain-invariant end-effector sampling is a reasonable design. However, the load-bearing claim of sim-to-real generalization is currently not supported by the evidence in the manuscript, and there are unresolved inconsistencies in the reported control metrics that need addressing before the central claims can be accepted.

major comments (4)
  1. [Section 4.3 and Abstract] The claim 'Through successful sim-to-real transfer, we demonstrate the system's generalization and robustness in real-world deployments' is not supported by the evidence. Section 4.3 reports only two tasks, 'navigate to pick' and 'pick and place', tested qualitatively with five objects. These are not among the eight long-horizon benchmark tasks listed in Table 7, and they involve at most one manipulation skill preceded by navigation. No quantitative success rates, trial counts, or error bars are reported, and the text concedes that the robot 'occasionally failed at grasping small objects.' Consequently, the real-world experiments can support component-level transfer of the low-level controller and VLM grounding for two elementary behaviors, but they cannot validate the integrated planner-controller stack for long-horizon tasks. Please either add quantitative real-world evaluations on the benchmark's long-horizon tasks or substantially weaken the abstract and contribution statements.
  2. [Section 4.2, Table 3] The reported end-effector tracking errors (Dpos ≈ 10.57–11.48 and Dori ≈ 46.93–47.53) are extraordinarily large for a manipulation controller, especially if the units are centimeters and degrees. A 47° orientation error would make the 'precise end-effector control' claim and the high pick/place success rates in Table 2 difficult to reconcile. The text only states that performance is 'comparable' to RoboDuet, but it does not explain how a controller with these average errors can achieve the fine manipulation reported elsewhere. Please clarify the units, the protocol (whether errors are averaged over random commanded targets, including potentially unreachable ones), and how the pick success rates are compatible with these error magnitudes.
  3. [Section 4.1, Tables 1 and 8] The comparison to PerAct is presented as 'substantial overall improvements,' but in the Seen/Test condition ODYSSEY is substantially worse on PICKUP OBJECT (60.45 vs. 94.03 in Table 1 and Table 8). The advantage is driven entirely by novel splits, which is expected given that ODYSSEY uses a strong pretrained VLM (Qwen2.5-VL-72B) while PerAct is trained on ARNOLD demonstrations. Moreover, the ARNOLD tasks were modified (spatial layout and robot platform changed) to fit the quadruped platform. Please report aggregate mean and variance over splits, describe whether PerAct was retrained on the modified tasks or evaluated under domain shift, and discuss the fairness and interpretability of this baseline comparison.
  4. [Section 3.3 and Section 4.1] The benchmark's end-to-end evaluation is unclear about the information available to the planner. The long-horizon results in Table 2 are obtained by running the full pipeline, but it is not specified whether the instance-level semantic graph is built online from raw sensor data (as described in Appendix A.1) or from ground-truth annotations in the simulator. If the scene graph contains oracle information, the reported task success rates overestimate the performance of the perception-planning loop. Please explicitly state whether the graph is constructed online during evaluation and, if not, report an ablation with the online perception pipeline.
minor comments (6)
  1. [Abstract and Section 1] The phrases 'first comprehensive benchmark' and 'first whole-body control policy' are strong claims that are not qualified against existing work such as WildLMA (Qiu et al. 2024b) or SLIM; consider softening these claims or adding explicit comparisons.
  2. [Section 3.2, Eq. (3)] Equation (3) defines st = (qt, ṑ), but the text says st ∈ R^36; the notation should be consistent, e.g., st = (qt, ṑt).
  3. [Section 4.3] The phrase 'as illusion in Fig. 4' should be 'as illustrated in Fig. 4'.
  4. [Table 1] The abbreviations P.OBJECT, R.OBJECT, O.CABINET, and C.CABINET are not expanded in the caption; please add the full task names for readability.
  5. [Appendix B.2, Table 6] The randomization 'method' column uses 'add', 'scale', and 'interval', but 'interval' is undefined; please explain this term or replace it with a clearer description.
  6. [Appendix D.1] The main text's Table 1 reports a unified 'Novel' column, but Appendix D.1 explains that this averages over four different splits. This aggregation should be described in the main text to avoid misleading readers into thinking a single novel-split protocol was used.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; central claims rest on external benchmarks and baselines, with only non-load-bearing self-citations.

full rationale

ODYSSEY does not derive a prediction from an input that already contains it. The long-horizon planning and manipulation claims are evaluated against external anchors: the four short-horizon tasks are migrated from ARNOLD (Gong et al. 2023), the low-level controller is compared with RoboDuet (Pan et al. 2025a), and the short-horizon manipulation evaluation uses PerAct (Shridhar et al. 2022) as baseline. These comparisons give the central evaluation independent content. No fitted parameter is later renamed as a prediction: the RL policy is trained with reward terms and then measured by task success and tracking errors, which is a direct evaluation rather than a circular reduction. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via self-citation to force the chosen architecture. The paper does cite prior work with overlapping authorship (for example Visual Whole-Body Control and WildLMA in related work), but those citations are contextual and are not load-bearing for the main derivation. The self-constructed long-horizon benchmark and the qualitative, two-task real-world evaluation are legitimate evidence concerns, but they concern the strength of empirical support, not circularity of the derivation chain. Overall, no specific circular step satisfying the required quote-and-reduction standard was found; the score reflects only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central contribution is an engineering integration. The evaluation depends on hand-tuned reward weights, uncompared prompt choices in proprietary foundation models, a converted ARNOLD setup, and an assumption that simulation physics with domain randomization transfers to a physical robot. No new physical entity is postulated and no mathematical axiom is introduced beyond standard assumptions about the simulator and the learned policy.

free parameters (5)
  • End-effector pose tracking reward weights (Stage 2) = -1.20 (position), -1.50 (orientation)
    Hand-chosen weights in Table 5 define the optimization balance that produces the reported end-effector tracking errors.
  • Gait and frequency reward weights = 0.75 (gait), 12.5 (frequency)
    Hand-tuned in Table 5 to shape a stable gait; no sensitivity analysis is reported.
  • Training end-effector workspace ranges = Radius [0.30, 0.65] m, yaw [-0.17 pi, 0.33 pi], orientation ranges in Table 4
    Section 4.2 deliberately shrinks the sampling workspace to avoid self-collision; this hand choice is part of the generalization claim against RoboDuet.
  • Instance-merging thresholds = tau_sem = 0.8, tau_geo = 0.8
    Appendix A.1 uses these thresholds to fuse object detections across frames; scene-graph quality depends on them.
  • Domain randomization ranges = Friction [0.4, 2.0], end-effector mass offset [0, 0.2] kg, actuator gain [0.8, 1.2], etc.
    Table 6 lists hand-selected ranges for sim-to-real transfer; no sensitivity study is reported.
assumptions (4)
  • domain assumption GPT-4.1 and Qwen2.5-VL-72B provide correct task decomposition and contact-point grounding when prompted with a scene graph and egocentric RGB-D.
    Invoked in Section 3.1; the framework trusts the proprietary models and does not ablate planner choice or quantify prompt sensitivity.
  • domain assumption Transferring ARNOLD tasks into the ODYSSEY simulator preserves the evaluation's meaning.
    Appendix C.1 converts raw ARNOLD samples into the new simulator; the comparison with PerAct assumes equivalence despite different robot, camera, and layouts.
  • domain assumption Simulation physics plus domain randomization predicts real-world behavior.
    Sections 3.2 and 4.3 rely on sim-to-real transfer; the real-world evidence is qualitative only, and the paper itself concedes failures on small objects.
  • domain assumption The reward function is a sufficient proxy for task success.
    Table 5 defines success-related measures as tracking rewards; simulation success is measured through the same quantities the policy optimizes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Tasks." pith.science (2026). https://pith.science/paper/JS73O37J

@misc{pith2026250808240,
  author       = {Pith},
  title        = {Pith review of: ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JS73O37J}},
  note         = {Machine review of arXiv:2508.08240}
}
read the original abstract

Language-guided long-horizon mobile manipulation has long been a grand challenge in embodied semantic reasoning, generalizable manipulation, and adaptive locomotion. Three fundamental limitations hinder progress: First, although large language models have improved spatial reasoning and task planning through semantic priors, existing implementations remain confined to tabletop scenarios, failing to address the constrained perception and limited actuation ranges of mobile platforms. Second, current manipulation strategies exhibit insufficient generalization when confronted with the diverse object configurations encountered in open-world environments. Third, while crucial for practical deployment, the dual requirement of maintaining high platform maneuverability alongside precise end-effector control in unstructured settings remains understudied. In this work, we present ODYSSEY, a unified mobile manipulation framework for agile quadruped robots equipped with manipulators, which seamlessly integrates high-level task planning with low-level whole-body control. To address the challenge of egocentric perception in language-conditioned tasks, we introduce a hierarchical planner powered by a vision-language model, enabling long-horizon instruction decomposition and precise action execution. At the control level, our novel whole-body policy achieves robust coordination across challenging terrains. We further present the first benchmark for long-horizon mobile manipulation, evaluating diverse indoor and outdoor scenarios. Through successful sim-to-real transfer, we demonstrate the system's generalization and robustness in real-world deployments, underscoring the practicality of legged manipulators in unstructured environments. Our work advances the feasibility of generalized robotic assistants capable of complex, dynamic tasks. Our project page: https://kaijwang.github.io/odyssey.github.io/

Figures

Figures reproduced from arXiv: 2508.08240 by the authors.

Figure 1
Figure 1. We present ODYSSEY, a unified mobile manipulation framework for agile quadruped robots equipped with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. ODYSSEY pipeline spans the entire process of a long-horizon task, including multi-modal semantic perception, map [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. An overview of the mobile manipulator policy and [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The robot system and real-world experiments [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Instance-level graph construct pipeline. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: ARNOLD converted environments of four tasks. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: The continuous state monitoring of ARNOLD [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: ODYSSEY benchmark details. Including task type, scene, and object configuration details. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: shows the distribution of these failure types across indoor and outdoor tasks. From the analysis, we draw several key observations: • Indoor tasks are dominated by reasoning and con￾trol failures. Tasks such as CABINETSTORAGE and RE￾STOCKING, which involve complex sema…
Figure 10
Figure 10. Figure 10: Prompt examples for task-level planning using LLM. The model is conditioned on a semantic instance graph and a [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Prompt examples for local manipulation guidance using VLM. Given an egocentric image and sub-task description, [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ABot-AgentOS: A General Robotic Agent OS with Lifelong Multi-modal Memory

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A robotic agent operating system with source-grounded graph memory and split-wise self-evolution improves long-horizon embodied task success and memory QA scores over baseline controllers.

  2. VGGSounder: Audio-Visual Evaluations for Foundation Models

    cs.MM 2025-08 unverdicted novelty 6.0 of 10

    VGGSounder is a corrected, multi-label audio-visual test set with modality annotations that exposes modality interference in foundation models.

Reference graph

Works this paper leans on

7 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [1]

    Parsing each .npz sample to extract task-specific pa- rameters, including scene setup, object category, pose, and robot initial state

    work page

  2. [2]

    Transforming all coordinate frames from ARNOLD’s convention to that of our simulator

    work page

  3. [3]

    Automatically generating natural language task instruc- tions and adjusting the robot’s initial pose to ensure reachability and feasibility

    work page

  4. [4]

    object_tag

    Grouping tasks by scene and saving them as per-scene Y AML files containing layout, object initialization, eval- uation metrics, and interaction constraints. This setup ensures consistent large-scale deployment and reproducible benchmarking of ODYSSEY’s short-horizon manipulation performance across multiple scenarios. Exam- ples of multiple tasks sharing ...

    work page

  5. [6]

    Task Decomposition: Break down complex sequential tasks into a series of atomic actions

    work page

  6. [7]

    nav": Navigate to a position -

    Action Types: Only use these predefined atomic actions: - "nav": Navigate to a position - "pick": Pick up an object - "place": Place a held object - “pull": Pull an articulated object while the quadruped remains stationary. - “push": Push an articulated object while the quadruped remains stationary. - "drag": Drag an object along the ground to a target lo...

    work page

  7. [2025]

    arXiv preprint arXiv:2507.01016

    VQ-VLA: Improving Vision-Language-Action Mod- els via Scaling Vector-Quantized Action Tokenizers. arXiv preprint arXiv:2507.01016. Zhang, C.; Han, D.; Zheng, S.; Choi, J.; Kim, T.-H.; and Hong, C. S. 2023. MobileSAMv2: Faster Segment Anything to Everything. arXiv:2312.09579. Zhang, H.; Yu, H.; Zhao, L.; Choi, A.; Bai, Q.; Yang, B.; and Xu, W. 2025. Slim: ...

    arXiv 2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.