Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a voxel-based semantic occupancy grid, fused from six cameras and one LiDAR, can serve as a generalized perception module for humanoid robots, and it supplies the dataset, annotation pipeline, and lightweight…

desk verdict A potentially useful humanoid occupancy benchmark is undermined by a LiDAR-visibility ground truth and inconsistent numbers; the dataset release and a GT rework should decide its fate. read the letter →

arxiv 2507.20217 v2 pith:3JYYW54D submitted 2025-07-27 cs.RO cs.AIcs.CV

Wei Cui , Haoyu Wang , Wenkang Qin , Yijie Guo , Gang Han , Wen Zhao , Jiahang Cao , Zhang Zhang
show 14 more authors
Jiaru Zhong Jingkai Sun Pihai Sun Shuai Shi Botuo Jiang Jiahao Ma Jiaxu Wang Hao Cheng Zhichao Liu Yang Wang Zheng Zhu Guan Huang Jian Tang Qiang Zhang
This is my paper
classification cs.ROcs.AIcs.CV
keywords humanoidrobots3DsemanticoccupancymultimodalperceptionLiDAR-camerafusiondatasetbird's-eyeviewtemporalsensorlayout
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that 3D semantic occupancy, previously a dominant perception representation in autonomous driving, can be transplanted to humanoid robots as a generalized perception module. It argues that the obstacles are engineering problems: a sensor layout that survives the robot's kinematics, a data collection and annotation scheme that yields dense voxel ground truth, and a lightweight multimodal fusion network. If the paper is right, the field gains the first panoramic occupancy dataset built specifically for humanoid robots, plus a benchmark and baseline that future work can compare against. The practical payoff for a general reader is that occupancy gives a humanoid robot the dense 3D geometry and semantics it needs for foot placement, manipulation, and path planning in human-scale spaces, not just the flat ground-plane view of bird's-eye representations.

What carries the argument

The machinery is the occupancy ground-truth generation pipeline plus the bird's-eye-view (BEV) fusion network. Static background points from many frames are motion-aligned and annotated point-wise; dynamic objects are labeled with 3D bounding boxes, with extra point-level annotation for special-posture pedestrians, and stitched into each frame; the merged cloud is voxelized into a 200 by 200 by 24 grid at 0.1 meter resolution, with Poisson reconstruction explicitly skipped. The network builds LiDAR BEV features from pillars, extracts image features from the six cameras with a shared convolutional backbone, fuses the two through deformable cross-attention keyed from LiDAR queries, warps in one historical BEV feature aligned by ego-motion, and reshapes fused BEV features into 3D voxels through a channel-to-height head. Training combines focal, lovasz-softmax, and scene- and class-scaled affinity losses, with temporal information introduced after the first epoch through a continuous-frame training schedule.

What would settle it

Pick a held-out scene from the dataset and construct a dense completed reference that fills occluded space, such as a mesh or a second full-coverage scan; if the paper's voxel ground truth labels those filled regions as empty, then the reported mIoU and rayIoU measure how well models reproduce the 40-line LiDAR's visibility pattern, not true occupancy.

Watch

Extended reading notes

Core claim

The paper's central claim is that occupancy grids, in which each voxel carries both an occupied-or-free state and a semantic class, can serve as a unified environmental perception output for humanoid robots, and that a complete system requires solving sensor layout, data collection, annotation, and fusion together. It reports a full hardware and software stack: six cameras plus a 40-line 360-degree LiDAR arranged on the head to reduce kinematic occlusion, a wearable collection harness that matches the sensor height of the humanoid, and an annotation pipeline that accumulates multi-frame static points and stitches bounding-box-annotated dynamic points into one point cloud, which is then directly voxelized at 0.1 meter resolution into a 200 by 200 by 24 grid without volumetric completion. The network, named HumanoidOcc, fuses LiDAR and camera features through deformable cross-attention, warps in one historical bird's-eye-view feature frame, and reshapes the fused features into a 3D occupancy prediction. On the new dataset the two-frame model reports 55.73 mIoU and 61.32 rayIoU with 40.5 million parameters, higher than the camera-only and LiDAR-only variants and higher than the bird's-eye-view baselines it is compared against.

Load-bearing premise

The labels are built from multi-frame accumulated point clouds with no volumetric completion, so a voxel counts as empty whenever the 40-line laser did not put a point there; if real occupied space is silent to the sensor, every number in the tables inherits that blind spot.

Editorial extensions

If this is right

  • A humanoid-specific occupancy benchmark with decimeter resolution over a 10 meter near-field range now exists, filling the gap left by autonomous-driving occupancy datasets that operate at 50 to 200 meters with meter-level resolution.
  • Multimodal fusion is worth its complexity on this task: the camera-plus-LiDAR model reaches 55.73 mIoU versus 50.37 for camera-only and 48.61 for LiDAR-only, confirming that geometry and semantics are complementary.
  • Temporal fusion of exactly one previous frame improves both mIoU and rayIoU, but windows of three or more frames degrade performance because pose errors accumulate during robotic motion.
  • A distortion-aware projection into BEV outperforms both raw pinhole projection and traditional image undistortion on the calibrated subset, so a latency-sensitive robot can avoid the cost of undistorting images.
  • The 40.5 million-parameter network establishes a lightweight baseline that future humanoid occupancy methods can be measured against on a common dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the wearable data-collection harness could be reused on other robot morphologies by matching sensor height and adding neck stabilization, letting the dataset grow without operating a humanoid in every scene.
  • Editorial inference: because the ground truth is defined by visible laser points, the reported mIoU and rayIoU likely reward models that reproduce the 40-line LiDAR's visibility pattern; a benchmark with explicit unknown or occluded labels would measure true geometric understanding more directly.
  • Editorial inference: the finding that the optimal temporal window is exactly two frames points to pose-error accumulation as the binding constraint, so better odometry or learned alignment, rather than more historical frames, is the lever that would extend the temporal benefit.
  • Editorial inference: if the dataset is released, the immediate test is cross-platform generalization, training on the wearable-collected data and evaluating on the humanoid's own sensors, which would quantify the domain gap the height-matching design was intended to close.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Humanoid Occupancy, a multimodal occupancy perception system for humanoid robots. It describes a sensor layout for the Tienkung humanoid robot (6 cameras and a 40-line LiDAR), a wearable data-collection device to avoid the cost of robot-based collection, an annotation pipeline that produces voxelized semantic occupancy labels, and a fusion network called HumaniodOcc. The network combines PointPillar-based LiDAR BEV features, camera CNN features, deformable cross-attention fusion, temporal feature warping, and a FlashOcc-style prediction head. The authors collected a dataset of 180 training and 20 validation clips with three scene types and evaluate against BEVDet, FBOcc, and BEVFusion, with ablations on distortion handling, temporal frames, and modality inputs. The central claims are that this is the first panoramic occupancy dataset specifically for humanoid robots and that the proposed network achieves superior metrics with fewer parameters.

Significance. If the dataset and benchmark are valid, the work would address a real gap: there is no established humanoid-specific occupancy benchmark, and the integrated hardware-software system could be a useful template for future robot perception platforms. The lightweight network design and the explicit treatment of sensor placement and kinematic interference are practical contributions. The paper also creditably includes a temporal fusion ablation and a distortion-aware projection strategy. However, the benchmark validity rests on the ground-truth definition, and the current evaluation does not establish that the voxel labels represent occupancy rather than LiDAR-visible surfaces. The dataset is not released, no code is provided, and all results are single-run, so the 'generalized' and 'standardized benchmark' claims are not yet supported.

major comments (4)
  1. [§3.3, Fig. 5] The occupancy ground truth is generated by directly voxelizing the superimposed static and stitched dynamic point clouds, and the text explicitly states that Poisson reconstruction is not performed. As a result, every labeled occupied voxel is a point hit by a LiDAR ray after multi-frame accumulation, with no separation of free space from unknown space and no completion of surfaces behind occluders. The mIoU and rayIoU metrics in Tables 2-5 therefore measure how well a model reproduces the LiDAR-visible surface shell, not geometric occupancy or scene completion. This is load-bearing for the paper's claim that the representation captures vertical structure and semantic attributes, and for the dataset's value as a benchmark. Please provide statistics on unobserved volume, compare against a completion-based label such as TSDF or Poisson reconstruction, or both, and adjust the claims accordingly.
  2. [§4.2, Table 2 and Fig. 7] The claim that the model 'achieves superior metrics' and the Fig. 7 statement that it 'significantly outperforms other methods in both mIoU and ray IoU' are contradicted by the 1-frame row: BEVFusion reports mIoU 53.98 versus HumaniodOcc 52.79, while rayIoU is essentially tied (60.24 vs 60.49). The advantage appears only in the 2-frame setting (55.73 vs 53.21). Please report per-configuration comparisons and correct the overclaim; with a single run and no error bars, 'significantly outperforms' is not supported.
  3. [§4.1–§4.2] All results come from a single training run on a self-collected dataset that is not released and is compared only against baselines trained in-house on the same data. No standard deviations, no multiple seeds, and no external benchmark are provided. Since the dataset is the paper's main novel contribution, this protocol is insufficient to support the 'standardized benchmark' and 'generalized' claims. At minimum, report variance across several seeds, describe the planned dataset release, and provide an external reference point such as evaluation on an established occupancy benchmark.
  4. [Table 3] The distortion ablation reports mIoU 47.92 for the final distortion-aware projection method, whereas Table 2 reports 52.79 for the same 1-frame model with raw pinhole projection. The note 'for this comparison we use data with precisely calibrated projection alignment to reveal the distinct characteristics of each distortion handling method' does not specify what differs in data, alignment, or training. Without that clarification, the ablation cannot be compared with the main results, and the conclusion that the proposed method 'ensures that the performance is not affected' is not established.
minor comments (5)
  1. [Throughout] There are frequent typos and inconsistent spellings, including 'HumaniodOcc' (should be 'HumanoidOcc'), 'evalidation' (§4.1), 'occulsion' and 'occlusion reasoning' (§4.3), 'A ablation' (Table 4), and 'the different number of temporal frames' (Table 4). The manuscript needs a careful proofreading pass.
  2. [§3.2] The wearable collection device is worn by humans, but no calibration or validation is reported to show that the wearable device's extrinsic calibration matches the robot-mounted configuration. Please specify how the transfer from human collector to robot is realized and what calibration checks are performed.
  3. [§4.1] The rayIoU metric is said to follow SparseOcc (Liu et al., 2023), but the cited reference [28] is 'Fully sparse 3d panoptic occupancy prediction.' Please clarify the exact ray sampling procedure and the correct citation.
  4. [§4.1, Table 2] The semantic category sets differ across scenes (Table 1), yet Table 2 reports per-class numbers for a single pooled class list. Please specify how classes are aligned across scenes and how the per-class metrics are aggregated.
  5. [References] Several references are incomplete or malformed, such as [1], [2], and [30], and some citations have trailing commas (e.g., 'Sun et al., 2025,'). Please normalize the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the system is a dataset-plus-network paper with no analytic derivation that reduces to its own inputs; the voxelized-LiDAR ground truth is an explicit labeling choice, and the reported comparisons are against externally developed baselines.

full rationale

This paper does not contain a derivation chain in which a predicted quantity is definitionally equal to a fitted input. The occupancy ground truth is explicitly constructed by superimposing multi-frame static LiDAR points, stitching dynamic-object bounding-box points, and directly voxelizing the merged cloud, with Poisson reconstruction deliberately skipped (Sec. 3.3, Fig. 5). That is a transparent labeling pipeline, not a circular step: the network is trained to predict this target from current sensor inputs, and the target is not itself a function of the network's weights or of the model's output. The explicit limitation that unobserved space is not labeled affects what the benchmark measures (surface visibility versus completed geometry), which is a correctness/validity concern rather than a circularity one. Self-citations to RoboOcc, HumanoidPano, and Occupancy World Model appear only in related-work and survey context and are not load-bearing for the central claims. The main experimental results compare against externally developed baselines (BEVDet, FBOcc, BEVFusion) retrained with identical settings, so the reported mIoU and rayIoU improvements are not guaranteed by construction. No equation, fitted parameter, or architectural choice in the paper was found to be equivalent to its own input under the definitions of circularity used here.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the annotation pipeline's validity, the representativeness of human-wearable data collection, and the accuracy of ego-motion and calibration assumptions. No new physical entities are introduced, and there is no parameter-free derivation; the model is a learned system whose weights are trained rather than fitted constants.

free parameters (3)
  • Number of historical frames (k) = 1 (2 total frames)
    Selected by validation ablation in Table 4; 3 and 4 frames degrade performance due to pose error accumulation, so the temporal memory size is a hand-chosen tuning decision that affects all multi-frame results.
  • Occupancy range and voxel resolution = X,Y in [-10,10] m, Z in [-1.5,0.9] m, 0.1 m voxels, 200x200x24 grid
    Design choices that define the task and evaluation space; the truncated Z-range means objects above 0.9 m or below -1.5 m are not evaluated, which shapes the reported mIoU and rayIoU.
  • Human collector height = About 160 cm
    Hand-picked to approximate the robot's sensor height; any height mismatch shifts the data distribution and the paper provides no sensitivity analysis.
assumptions (5)
  • domain assumption Voxelized, multi-frame accumulated LiDAR point clouds constitute valid occupancy ground truth without volumetric completion.
    Section 3.3 explicitly skips Poisson reconstruction, so occupancy is defined only at sensor-visible surfaces. This may bias evaluation toward matching LiDAR visibility rather than true 3D occupancy.
  • domain assumption Human wearable data collection with a 160 cm tall wearer approximates the sensor pose of the Tienkung robot.
    Section 3.2 states the height matching is intended to keep the distribution close, but no quantitative domain-gap analysis is provided.
  • domain assumption Accurate ego-motion and camera-LiDAR calibration are available.
    Equations 1-2 rely on relative ego-motion T for temporal warping, and the distortion-aware projection relies on pinhole distortion parameters; Table 4 attributes degradation at 3-4 frames to pose errors, showing sensitivity.
  • domain assumption Dynamic objects can be captured by bounding-box stitching during ground-truth generation.
    Section 3.3 merges dynamic foreground points via bounding box alignment; annotation errors in boxes or alignment propagate directly to the occupancy labels.
  • standard math Standard deep learning tooling (mmdetection3d, pretrained ResNet50) is correct.
    Experiments are built on the mmdetection3d framework and pretrained backbones; the paper does not verify these components.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots." pith.science (2026). https://pith.science/paper/3JYYW54D

@misc{pith2026250720217,
  author       = {Pith},
  title        = {Pith review of: Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3JYYW54D}},
  note         = {Machine review of arXiv:2507.20217}
}
read the original abstract

Humanoid robot technology is advancing rapidly, with manufacturers introducing diverse heterogeneous visual perception modules tailored to specific scenarios. Among various perception paradigms, occupancy-based representation has become widely recognized as particularly suitable for humanoid robots, as it provides both rich semantic and 3D geometric information essential for comprehensive environmental understanding. In this work, we present Humanoid Occupancy, a generalized multimodal occupancy perception system that integrates hardware and software components, data acquisition devices, and a dedicated annotation pipeline. Our framework employs advanced multi-modal fusion techniques to generate grid-based occupancy outputs encoding both occupancy status and semantic labels, thereby enabling holistic environmental understanding for downstream tasks such as task planning and navigation. To address the unique challenges of humanoid robots, we overcome issues such as kinematic interference and occlusion, and establish an effective sensor layout strategy. Furthermore, we have developed the first panoramic occupancy dataset specifically for humanoid robots, offering a valuable benchmark and resource for future research and development in this domain. The network architecture incorporates multi-modal feature fusion and temporal information integration to ensure robust perception. Overall, Humanoid Occupancy delivers effective environmental perception for humanoid robots and establishes a technical foundation for standardizing universal visual modules, paving the way for the widespread deployment of humanoid robots in complex real-world scenarios.

Figures

Figures reproduced from arXiv: 2507.20217 by the authors.

Figure 1
Figure 1. Schematic diagram of the Humanoid Occupancy system. © 2025 X-HUMANOID and GigaAI. All rights reserved. arXiv:2507.20217v2 [cs.RO] 29 Jul 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Overview of our proposed HumaniodOcc. The proposed network processes image and LiDAR inputs through separate encoders to extract features. The image features and LiDAR features are then fused via cross-attention mechanisms, enabling adaptive interaction between visual and geometric cues. The fused features undergo temporal fusion to aggregate sequential information. Finally, the network predicts the 3D occupancy gri… view at source ↗
Figure 2
Figure 2. Equipment and Schematic Diagram of Data Acquisition Process. In this work, we implement the Humanoid Occupancy concept system on the Tienkung humanoid robot. The system features a modular, selectable RGB-D camera with two degrees of freedom (pitch and yaw), supporting both manipulation and terrain per￾ception tasks. Additionally, 6 cameras and one LiDAR sensor are deployed to enable robust mapping and semantic under… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Structural visualization of humanoid occupancy hardware. We equipped the humanoid robot head with 6 cameras and a lidar to acquire sensory information. More details in Sec. 3.2 use humanoid robotics to collect data in the environments, due to the extremely high cost an…
Figure 5
Figure 5. Figure 5: The occupancy generation pipeline. The ground truth generation pipeline begins with 3D bounding box annotation and semantic segmentation labeling. Multi-frame LiDAR scans are then processed by separately stitching dynamic objects through bounding box alignment and accu…
Figure 6
Figure 6. Figure 6: The results of our humanoid perception system. The left side of each row shows the occupancy ground truth, the middle is the 6 images and point cloud input to the model, and the right side shows the result of model inference. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Comparisons of the mIoU and ray IoU of various 3D occupancy prediction methods. Our model significantly outperforms other methods in both mIoU and ray IoU with the smallest number of parameters [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Panoramic stitching of 6 cameras and densified visualization of LiDAR point clouds. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: 3D scene visualization and LiDAR point clouds visualization. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Urban Autonomous Agents

    cs.CV 2026-03 conditional novelty 6.5 of 10

    O3N is the first open-vocabulary occupancy prediction method that takes a single omnidirectional RGB image and labels 3D voxels with both seen and unseen semantic classes.

  2. GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory

    cs.RO 2026-07 conditional novelty 6.0 of 10

    GEM-Occ converts transient visual geometry into semantic Gaussian and free-space ray evidence, fuses them into hierarchical occupancy memory, and beats prior indoor occupancy baselines on the new HIOcc benchmark.

Reference graph

Works this paper leans on

72 extracted references · 39 canonical work pages · cited by 2 Pith papers

  1. [1]

    Video friday: Atlas robot sees the world

    Evan Ackerman. Video friday: Atlas robot sees the world. URL https://spectrum.ieee.org/ video-friday-atlas-robot-sees-world . 3

    work page

  2. [2]

    agilityrobotics website

    AgilityRobotics. agilityrobotics website. URLhttps://www.agilityrobotics.com/. 3

    work page

  3. [3]

    Homie: Humanoid loco-manipulation with isomorphic exoskeleton cockpit.arXiv preprint arXiv:2502.13013, 2025

    Qingwei Ben, Feiyu Jia, Jia Zeng, Junting Dong, Dahua Lin, and Jiangmiao Pang. Homie: Humanoid loco-manipulation with isomorphic exoskeleton cockpit.arXiv preprint arXiv:2502.13013, 2025. 3

    arXiv 2025

  4. [4]

    Human- robot perception in industrial environments: A survey.Sensors, 21(5):1571, 2021

    Andrea Bonci, Pangcheng David Cen Cheng, Marina Indri, Giacomo Nabissi, and Fiorella Sibona. Human- robot perception in industrial environments: A survey.Sensors, 21(5):1571, 2021. 2

    work page 2021

  5. [5]

    nuscenes: A multimodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11621–11631,

    work page

  6. [6]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. InEuropean conference on computer vision, pages 213–229. Springer, 2020. 3

    work page 2020

  7. [7]

    Open-television: Teleoperation with immersive active visual feedback.arXiv preprint arXiv:2407.01512, 2024

    Xuxin Cheng, Jialong Li, Shiqi Yang, Ge Yang, and Xiaolong Wang. Open-television: Teleoperation with immersive active visual feedback.arXiv preprint arXiv:2407.01512, 2024. 3

    arXiv 2024

  8. [8]

    All robots are not created equal: the design and perception of humanoid robot heads

    Carl F DiSalvo, Francine Gemperle, Jodi Forlizzi, and Sara Kiesler. All robots are not created equal: the design and perception of humanoid robot heads. InProceedings of the 4th conference on Designing interactive systems: processes, practices, methods, and techniques , pages 321–326, 2002. 2

    work page 2002

Show all 72 references
  1. [9]

    Learning vision-based bipedal locomotion for challenging terrain

    Helei Duan, Bikram Pandit, Mohitvishnu S Gadde, Bart Van Marum, Jeremy Dao, Chanho Kim, and Alan Fern. Learning vision-based bipedal locomotion for challenging terrain. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 56–62. IEEE, 2024. 3

    2024

  2. [10]

    Occupancy grids: A stochastic spatial representation for active robot perception.arXiv preprint arXiv:1304.1098, 2013

    Alberto Elfes. Occupancy grids: A stochastic spatial representation for active robot perception.arXiv preprint arXiv:1304.1098, 2013. 3

    2013 arXiv

  3. [11]

    Humanplus: Humanoid shadowing and imitation from humans

    Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Humanoid shadowing and imitation from humans. InConference on Robot Learning (CoRL) , 2024. 3

    2024

  4. [12]

    Humanoid locomotion and manipulation: Current progress and challenges in control, planning, and learning.arXiv preprint arXiv:2501.02116, 2025

    Zhaoyuan Gu, Junheng Li, Wenlan Shen, Wenhao Yu, Zhaoming Xie, Stephen McCrory, Xianyi Cheng, Abdulaziz Shamsah, Robert Griffin, C Karen Liu, et al. Humanoid locomotion and manipulation: Current progress and challenges in control, planning, and learning.arXiv preprint arXiv:25...

    2025 arXiv

  5. [13]

    Attention- based map encoding for learning generalized legged locomotion.arXiv preprint arXiv:2506.09588, 2025

    Junzhe He, Chong Zhang, Fabian Jenelten, Ruben Grandia, Moritz BÄcher, and Marco Hutter. Attention- based map encoding for learning generalized legged locomotion.arXiv preprint arXiv:2506.09588, 2025. 3

    2025 arXiv

  6. [14]

    The development of honda humanoid robot

    Kazuo Hirai, Masato Hirose, Yuji Haikawa, and Toru Takenaka. The development of honda humanoid robot. InProceedings. 1998 IEEE international conference on robotics and automation ( Cat. No. 98CH36146), volume 2, pages 1321–1326. IEEE, 1998. 2

    1998

  7. [15]

    Honda humanoid robots development.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences , 365(1850):11–19, 2007

    Masato Hirose and Kenichi Ogawa. Honda humanoid robots development.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences , 365(1850):11–19, 2007. 3

    2007

  8. [16]

    Bevdet4d: Exploit temporal cues in multi-camera 3d object detection

    Junjie Huang and Guan Huang. Bevdet4d: Exploit temporal cues in multi-camera 3d object detection. arXiv preprint arXiv:2203.17054, 2022. 9 15 Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots

    2022 arXiv

  9. [17]

    Bevdet: High-performance multi-camera 3d object detection in bird-eye-view.arXiv preprint arXiv:2112.11790, 2021

    Junjie Huang, Guan Huang, Zheng Zhu, Ye Yun, and Dalong Du. Bevdet: High-performance multi-camera 3d object detection in bird-eye-view.arXiv preprint arXiv:2112.11790, 2021. 10

    2021 arXiv

  10. [18]

    Tri-perspective view for vision-based 3d semantic occupancy prediction

    Yuanhui Huang, Wenzhao Zheng, Yunpeng Zhang, Jie Zhou, and Jiwen Lu. Tri-perspective view for vision-based 3d semantic occupancy prediction. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9223–9232, 2023. 5

    2023

  11. [19]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023. 4

    2023

  12. [20]

    Armor: Egocentric perception for humanoid robot collision avoidance and motion planning.arXiv preprint arXiv:2412.00396, 2024

    Daehwa Kim, Mario Srouji, Chen Chen, and Jian Zhang. Armor: Egocentric perception for humanoid robot collision avoidance and motion planning.arXiv preprint arXiv:2412.00396, 2024. 3

    2024 arXiv

  13. [21]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision , pages 4015–4026, 2023. 3

    2023

  14. [22]

    Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom

    Alex H. Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Point- pillars: Fast encoders for object detection from point clouds. In IEEE Conference on Com- puter Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019 , page...

    2019 doi

  15. [23]

    Voxformer: Sparsevoxeltransformerforcamera-based3dsemanticscenecompletion

    Yiming Li, Zhiding Yu, Christopher Choy, Chaowei Xiao, Jose M Alvarez, Sanja Fidler, Chen Feng, and AnimaAnandkumar. Voxformer: Sparsevoxeltransformerforcamera-based3dsemanticscenecompletion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition ...

  16. [24]

    Le, Alan L

    Yingwei Li, Adams Wei Yu, Tianjian Meng, Benjamin Caine, Jiquan Ngiam, Daiyi Peng, Junyang Shen, Yifeng Lu, Denny Zhou, Quoc V. Le, Alan L. Yuille, and Mingxing Tan. Deepfusion: Lidar-camera deep fusion for multi-modal 3d object detection. InIEEE/CVF Conference on Computer Vis...

    2022

  17. [25]

    Fb-occ: 3d occupancy prediction based on forward-backward view transformation.arXiv preprint arXiv:2307.01492,

    Zhiqi Li, Zhiding Yu, David Austin, Mingsheng Fang, Shiyi Lan, Jan Kautz, and Jose M Alvarez. Fb-occ: 3d occupancy prediction based on forward-backward view transformation.arXiv preprint arXiv:2307.01492,

    arXiv

  18. [26]

    Bevfusion: A simple and robust lidar-camera fusion framework.Advances in Neural Information Processing Systems, 35:10421–10434, 2022

    Tingting Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia, Zhiwei Lin, Yongtao Wang, Tao Tang, Bing Wang, and Zhi Tang. Bevfusion: A simple and robust lidar-camera fusion framework.Advances in Neural Information Processing Systems, 35:10421–10434, 2022. 5

    2022

  19. [27]

    Girshick, Kaiming He, Bharath Hariharan, and Serge J

    Tsung-Yi Lin, Piotr Dollár, Ross B. Girshick, Kaiming He, Bharath Hariharan, and Serge J. Belongie. Feature pyramid networks for object detection. In2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017 , pages 936–944....

    2017

  20. [28]

    Fully sparse 3d panoptic occupancy prediction.arXiv preprint arXiv:2312.17118, 2023

    Haisong Liu, Haiguang Wang, Yang Chen, Zetong Yang, Jia Zeng, Li Chen, and Limin Wang. Fully sparse 3d panoptic occupancy prediction.arXiv preprint arXiv:2312.17118, 2023. 10

    2023 arXiv

  21. [29]

    Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation.arXiv preprint arXiv:2205.13542, 2022

    Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela Rus, and Song Han. Bevfusion: Multi-task multi-sensor fusion with unified bird’s-eye view representation.arXiv preprint arXiv:2205.13542, 2022. 5, 10 16 Humanoid Occupancy: Enabling A Generalized Multim...

    2022 arXiv

  22. [30]

    Learning humanoid locomotion with perceptive internal model, 2024.URL https://arxiv

    Junfeng Long, Junli Ren, Moji Shi, Zirui Wang, Tao Huang, Ping Luo, and Jiangmiao Pang. Learning humanoid locomotion with perceptive internal model, 2024.URL https://arxiv. org/abs/2411.14386. 3

    2024 arXiv

  23. [31]

    Mobile-television: Predictive motion priors for humanoid whole-body control.arXiv preprint arXiv:2412.07773, 2024

    Chenhao Lu, Xuxin Cheng, Jialong Li, Shiqi Yang, Mazeyu Ji, Chengjing Yuan, Ge Yang, Sha Yi, and Xiaolong Wang. Mobile-television: Predictive motion priors for humanoid whole-body control.arXiv preprint arXiv:2412.07773, 2024. 3

    2024 arXiv

  24. [32]

    Pie: Parkour with implicit- explicit learning framework for legged robots.IEEE Robotics and Automation Letters , 2024

    Shixin Luo, Songbo Li, Ruiqi Yu, Zhicheng Wang, Jun Wu, and Qiuguo Zhu. Pie: Parkour with implicit- explicit learning framework for legged robots.IEEE Robotics and Automation Letters , 2024. 4

    2024

  25. [33]

    Vision-centric bev perception: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Yuexin Ma, Tai Wang, Xuyang Bai, Huitong Yang, Yuenan Hou, Yaming Wang, Yu Qiao, Ruigang Yang, and Xinge Zhu. Vision-centric bev perception: A survey.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 5

    2024

  26. [34]

    Impact of ros 2 node composition in robotic systems.IEEE Robotics and Automation Letters , 8(7):3996–4003, 2023

    Steve Macenski, Alberto Soragna, Michael Carroll, and Zhenpeng Ge. Impact of ros 2 node composition in robotic systems.IEEE Robotics and Automation Letters , 8(7):3996–4003, 2023. 4

    2023

  27. [35]

    Robot operating system 2: Design, architecture, and uses in the wild.Science robotics, 7(66):eabm6074, 2022

    Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. Robot operating system 2: Design, architecture, and uses in the wild.Science robotics, 7(66):eabm6074, 2022. 4

    2022

  28. [36]

    Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106, 2021

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106, 2021. 4

    2021

  29. [37]

    Occfusion: Multi-sensor fusion framework for 3d semantic occupancy prediction.IEEE Transactions on Intelligent Vehicles, 2024

    Zhenxing Ming, Julie Stephany Berrio, Mao Shan, and Stewart Worrall. Occfusion: Multi-sensor fusion framework for 3d semantic occupancy prediction.IEEE Transactions on Intelligent Vehicles, 2024. 5

    2024

  30. [38]

    Perceptions of asimo: an exploration on co-operation and competition with humans and humanoid robots

    Bilge Mutlu, Steven Osman, Jodi Forlizzi, Jessica Hodgins, and Sara Kiesler. Perceptions of asimo: an exploration on co-operation and competition with humans and humanoid robots. InProceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction , pages 351–352, 2006. 3

    2006

  31. [39]

    Lovon: Legged open-vocabulary object navigator

    Daojie Peng, Jiahang Cao, Qiang Zhang, and Jun Ma. Lovon: Legged open-vocabulary object navigator. arXiv preprint arXiv:2507.06747, 2025. 3

    2025 arXiv

  32. [40]

    Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d

    Jonah Philion and Sanja Fidler. Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. InEuropean conference on computer vision, pages 194–210. Springer, 2020. 2

    2020

  33. [41]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

  34. [42]

    Perception for humanoid robots

    Arindam Roychoudhury, Shahram Khorshidi, Subham Agrawal, and Maren Bennewitz. Perception for humanoid robots. Current Robotics Reports, 4(4):127–140, 2023. 2

    2023

  35. [43]

    Parkour in the wild: Learning a general and extensible agile locomotion policy using multi-expert distillation and rl fine-tuning.arXiv preprint arXiv:2505.11164, 2025

    Nikita Rudin, Junzhe He, Joshua Aurand, and Marco Hutter. Parkour in the wild: Learning a general and extensible agile locomotion policy using multi-expert distillation and rl fine-tuning.arXiv preprint arXiv:2505.11164, 2025. 3

    2025 arXiv

  36. [44]

    3d is here: Point cloud library (pcl)

    Radu Bogdan Rusu and Steve Cousins. 3d is here: Point cloud library (pcl). In2011 IEEE international conference on robotics and automation, pages 1–4. IEEE, 2011. 3

    2011

  37. [45]

    Asimo and humanoid robot research at honda

    Satoshi Shigemi. Asimo and humanoid robot research at honda. InHumanoid robotics: A reference, pages 1–36. Springer, 2017. 3 17 Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots

    2017

  38. [46]

    Scene as occupancy.arXiv preprint arXiv:2306.02851, 2023

    Chonghao Sima, Wenwen Tong, Tai Wang, Li Chen, Silei Wu, Hanming Deng, Yi Gu, Lewei Lu, Ping Luo, Dahua Lin, et al. Scene as occupancy.arXiv preprint arXiv:2306.02851, 2023. 4

    2023 arXiv

  39. [47]

    Robotic operating system

    Stanford Artificial Intelligence Laboratory et al. Robotic operating system. URLhttps://www.ros.org. 4

  40. [48]

    Trinity: A modular humanoid robot ai system.arXiv preprint arXiv:2503.08338, 2025

    Jingkai Sun, Qiang Zhang, Gang Han, Wen Zhao, Zhe Yong, Yan He, Jiaxu Wang, Jiahang Cao, Yijie Guo, and Renjing Xu. Trinity: A modular humanoid robot ai system.arXiv preprint arXiv:2503.08338, 2025. 3

    2025 arXiv

  41. [49]

    Learning perceptive humanoid locomotion over challenging terrain.arXiv preprint arXiv:2503.00692, 2025

    Wandong Sun, Baoshi Cao, Long Chen, Yongbo Su, Yang Liu, Zongwu Xie, and Hong Liu. Learning perceptive humanoid locomotion over challenging terrain.arXiv preprint arXiv:2503.00692, 2025. 3

    2025 arXiv

  42. [50]

    Sparseocc: Rethinking sparse latent representation for vision-based semantic occupancy prediction

    Pin Tang, Zhongdao Wang, Guoqing Wang, Jilai Zheng, Xiangxuan Ren, Bailan Feng, and Chao Ma. Sparseocc: Rethinking sparse latent representation for vision-based semantic occupancy prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

  43. [51]

    Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36:64318–64330, 2023

    Xiaoyu Tian, Tao Jiang, Longfei Yun, Yucheng Mao, Huitong Yang, Yue Wang, Yilun Wang, and Hang Zhao. Occ3d: A large-scale 3d occupancy prediction benchmark for autonomous driving.Advances in Neural Information Processing Systems, 36:64318–64330, 2023. 5

    2023

  44. [52]

    Embodiedocc++: Boosting embodied 3d occupancy prediction with plane regularization and uncertainty sampler, 2025

    Hao Wang, Xiaobao Wei, Xiaoan Zhang, Jianing Li, Chengyu Bai, Ying Li, Ming Lu, Wenzhao Zheng, and Shanghang Zhang. Embodiedocc++: Boosting embodied 3d occupancy prediction with plane regularization and uncertainty sampler, 2025. URLhttps://arxiv.org/abs/2504.09540. 5

    2025 arXiv

  45. [53]

    Reinforcement learning with generalizable gaussian splatting

    Jiaxu Wang, Qiang Zhang, Jingkai Sun, Jiahang Cao, Gang Han, Wen Zhao, Weining Zhang, Yecheng Shao, Yijie Guo, and Renjing Xu. Reinforcement learning with generalizable gaussian splatting. In2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , page...

    2024

  46. [54]

    Query-based semantic gaussian field for scene representation in reinforcement learning.arXiv preprint arXiv:2406.02370, 2024

    Jiaxu Wang, Ziyi Zhang, Qiang Zhang, Jia Li, Jingkai Sun, Mingyuan Sun, Junhao He, and Renjing Xu. Query-based semantic gaussian field for scene representation in reinforcement learning.arXiv preprint arXiv:2406.02370, 2024. 4

    2024 arXiv

  47. [55]

    Exploring object-centric temporal modeling for efficient multi-view 3d object detection.arXiv preprint arXiv:2303.11926, 2023

    Shihao Wang, Yingfei Liu, Tiancai Wang, Ying Li, and Xiangyu Zhang. Exploring object-centric temporal modeling for efficient multi-view 3d object detection.arXiv preprint arXiv:2303.11926, 2023. 10

    2023 arXiv

  48. [56]

    Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception

    Xiaofeng Wang, Zheng Zhu, Wenbo Xu, Yunpeng Zhang, Yi Wei, Xu Chi, Yun Ye, Dalong Du, Jiwen Lu, and Xingang Wang. Openoccupancy: A large scale benchmark for surrounding semantic occupancy perception. In Proceedings of the IEEE/CVF International Conference on Computer Vision , ...

    2023

  49. [57]

    Omni-perception: Omnidirectional collision avoidance for legged locomotion in dynamic environments

    ZifanWang,TeliMa,YufeiJia,XunYang,JiamingZhou,WenlongOuyang,QiangZhang,andJunweiLiang. Omni-perception: Omnidirectional collision avoidance for legged locomotion in dynamic environments. arXiv preprint arXiv:2505.19214, 2025. 3, 4

    2025 arXiv

  50. [58]

    Embodiedocc: Embodied 3d occupancy prediction for vision-based online scene understanding.arXiv preprint arXiv:2412.04380,

    YuqiWu, WenzhaoZheng, SichengZuo, YuanhuiHuang, JieZhou, andJiwenLu. Embodiedocc: Embodied 3d occupancy prediction for vision-based online scene understanding.arXiv preprint arXiv:2412.04380,

    arXiv

  51. [59]

    Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin

    Zichen Yu, Changyong Shu, Jiajun Deng, Kangjie Lu, Zongdai Liu, Jiangyong Yu, Dawei Yang, Hui Li, and Yan Chen. Flashocc: Fast and memory-efficient occupancy prediction via channel-to-height plugin. arXiv preprint arXiv:2311.12058, 2023. 5, 9

    2023 arXiv

  52. [60]

    Gnfactor: Multi-task real robot learning with generalizable neural feature fields

    Yanjie Ze, Ge Yan, Yueh-Hua Wu, Annabella Macaluso, Yuying Ge, Jianglong Ye, Nicklas Hansen, Li Erran Li, and Xiaolong Wang. Gnfactor: Multi-task real robot learning with generalizable neural feature fields. In Conference on robot learning, pages 284–301. PMLR, 2023. 4 18 Huma...

    2023

  53. [61]

    Generalizable humanoid manipulation with 3d diffusion policies.arXiv preprint arXiv:2410.10803, 2024

    Yanjie Ze, Zixuan Chen, Wenhao Wang, Tianyi Chen, Xialin He, Ying Yuan, Xue Bin Peng, and Jiajun Wu. Generalizable humanoid manipulation with 3d diffusion policies.arXiv preprint arXiv:2410.10803, 2024. 3

    2024 arXiv

  54. [62]

    Occfusion: Depthestimationfreemulti-sensorfusionfor3doccupancy prediction

    JiZhang, YiranDing, andZixinLiu. Occfusion: Depthestimationfreemulti-sensorfusionfor3doccupancy prediction. In Proceedings of the Asian Conference on Computer Vision , pages 3587–3604, 2024. 5

    2024

  55. [63]

    Distillation-ppo: A novel two-stage reinforcement learning framework for humanoid robot perceptive locomotion.arXiv preprint arXiv:2503.08299, 2025

    Qiang Zhang, Gang Han, Jingkai Sun, Wen Zhao, Chenghao Sun, Jiahang Cao, Jiaxu Wang, Yijie Guo, and Renjing Xu. Distillation-ppo: A novel two-stage reinforcement learning framework for humanoid robot perceptive locomotion.arXiv preprint arXiv:2503.08299, 2025. 3

    2025 arXiv

  56. [64]

    Humanoidpano: Hybrid spherical panoramic-lidar cross-modal perception for humanoid robots.arXiv preprint arXiv:2503.09010, 2025

    Qiang Zhang, Zhang Zhang, Wei Cui, Jingkai Sun, Jiahang Cao, Yijie Guo, Gang Han, Wen Zhao, Jiaxu Wang, Chenghao Sun, et al. Humanoidpano: Hybrid spherical panoramic-lidar cross-modal perception for humanoid robots.arXiv preprint arXiv:2503.09010, 2025. 3, 5

    2025 arXiv

  57. [65]

    Learning robust motion skills via critical adversarial attacks for humanoid robots.arXiv preprint arXiv:2507.08303, 2025

    Yang Zhang, Zhanxiang Cao, Buqing Nie, Haoyang Li, and Yue Gao. Learning robust motion skills via critical adversarial attacks for humanoid robots.arXiv preprint arXiv:2507.08303, 2025. 3

    2025

  58. [66]

    Occformer: Dual-path transformer for vision-based 3d semantic occupancy prediction

    Yunpeng Zhang, Zheng Zhu, and Dalong Du. Occformer: Dual-path transformer for vision-based 3d semantic occupancy prediction. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9433–9443, 2023. 5

    2023

  59. [67]

    Roboocc: Enhancing the geometric and semantic scene understanding for robots

    Zhang Zhang, Qiang Zhang, Wei Cui, Shuai Shi, Yijie Guo, Gang Han, Wen Zhao, Hengle Ren, Renjing Xu, and Jian Tang. Roboocc: Enhancing the geometric and semantic scene understanding for robots. arXiv preprint arXiv:2504.14604, 2025. 4

    2025 arXiv

  60. [68]

    Occupancy world model for robots.arXiv preprint arXiv:2505.05512, 2025

    Zhang Zhang, Qiang Zhang, Wei Cui, Shuai Shi, Yijie Guo, Gang Han, Wen Zhao, Jingkai Sun, Jiahang Cao, Jiaxu Wang, et al. Occupancy world model for robots.arXiv preprint arXiv:2505.05512, 2025. 4

    2025 arXiv

  61. [69]

    Open3d: A modern library for 3d data processing.arXiv preprint arXiv:1801.09847, 2018

    Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3d: A modern library for 3d data processing.arXiv preprint arXiv:1801.09847, 2018. 4

    2018 arXiv

  62. [70]

    Deformable DETR: deformable transformers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR: deformable transformers for end-to-end object detection. In9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021...

    2021

  63. [71]

    Humanoid parkour learning

    Ziwen Zhuang, Shenzhe Yao, and Hang Zhao. Humanoid parkour learning. arXiv preprint arXiv:2406.10759, 2024. 3 19

    2024 arXiv

  64. [2017]

    URLhttps://doi.org/10.1109/CVPR.2017.106

    doi: 10.1109/CVPR.2017.106. URLhttps://doi.org/10.1109/CVPR.2017.106. 9

    2017 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.