Pith. sign in

REVIEW 5 major objections 5 minor 94 references

O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Urban Autonomous Agents

T0 review · 5 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read O3N claims that a single omnidirectional RGB image plus arbitrary text class names can produce a full 3D occupancy map labeling classes never seen in training, and reports state-of-the-art results on two benchmarks.

desk verdict Solid first step on omni open-vocab occupancy, but the novel-class numbers lean on the most frequent, easiest classes. read the letter →

arxiv 2603.12144 v4 pith:DZDD3LTD submitted 2026-03-12 cs.CV cs.ROeess.IV

Mengfei Duan , Hao Shi , Fei Teng , Guoqiang Zhao , Yuheng Zhang , Zhiyong Li , Kailun Yang This is my paper
classification cs.CVcs.ROeess.IV
keywords open-vocabularyoccupancypredictionomnidirectionalperceptionpanoramicimages3Dsemanticscenecompletionpixel-voxel-textalignmentstatespacemodelsvision-languageembodiedagents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

O3N attempts to establish that a single 360-degree RGB image, together with free-form text class names, is enough to produce a full 3D occupancy map of the surrounding scene, including semantic categories that were never labeled during training. If true, embodied agents—robots, quadruped platforms, and future urban vehicles—could reconstruct and understand the 3D world from one cheap camera instead of relying on multi-view sensor rigs or fixed category lists. The paper reports state-of-the-art results on two omnidirectional occupancy benchmarks, with the biggest gains on novel classes (e.g., +3.01 mean IoU over the previous open-vocabulary occupancy baseline on QuadOcc), and shows the approach transfers to simulation and indoor scenes. The framework is built around a 'pixel-voxel-text' triad: a polar-spiral scanning module for panoramic geometry, an occupancy cost aggregation module for geometric-semantic consistency, and a gradient-free alignment mechanism that closes the image-text domain gap without backpropagating through the text encoder. The central bet is that open-world 3D perception can be decomposed into these three problems and solved in an end-to-end, purely visual pipeline.

What carries the argument

The load-bearing mechanism is the 'pixel-voxel-text' triad, realized by three intertwined modules. The Polar-spiral Mamba (PsM) module compresses cylindrical voxels into a bird's-eye-view feature and scans it in an outward spiral from the pole, fusing cylindrical and Cartesian voxel representations to preserve continuity under equirectangular distortion. The Occupancy Cost Aggregation (OCA) module forms a voxel–text cost volume by cosine similarity and refines it with per-class spatial aggregation, class-wise transformer aggregation, and a scene-affinity loss, turning raw similarities into geometrically consistent semantic masks. The Natural Modality Alignment (NMA) module treats text embedd

What would settle it

Re-run the same training protocol on QuadOcc with an inverted split: keep road, building, and vehicle as base classes and hold out person, vegetation, and terrain as novel. If O3N's novel mIoU falls to near its base mIoU, or matches a frozen-encoder similarity baseline with no learned alignment, then the reported generalization is an artifact of the chosen class split rather than evidence of open-vocabulary occupancy reasoning.

Watch

Extended reading notes

Core claim

The paper's central claim is that the omnidirectional open-vocabulary occupancy prediction task is well posed and that O3N is the first purely visual, end-to-end solution. O3N takes one equirectangular RGB image, extracts 2D features and text embeddings from a frozen vision-language encoder, lifts features into a hybrid Cartesian-cylindrical voxel grid, and uses a spiral scan that starts at the pole and moves outward to fuse the two grids while respecting panoramic density. The occupancy cost aggregation module builds a voxel-text similarity cost volume and refines it with spatial and class-wise aggregation plus an affinity loss, so geometry and semantics are supervised jointly. The natural

Load-bearing premise

The load-bearing assumption is that holding out the most frequent and visually salient classes (road, building, vehicle) and training only on rare classes measures the framework's ability to generalize to unseen semantics, rather than the frozen vision-language encoder's existing ability to recognize those easy categories.

Editorial extensions

If this is right

  • A single 360° camera, rather than multi-camera or LiDAR rigs, can in principle support open-vocabulary 3D occupancy for robots and vehicles.
  • Classes absent from training data can be queried at inference time by typing a class name, so deployed agents could adapt to new object categories without retraining.
  • The modular design transfers to multiple 3D occupancy backbones, delivering consistent gains over the previous baseline, suggesting the modules are not tied to one architecture.
  • Because the method runs at about 9.4 frames per second with roughly 5 GB memory at inference, real-time embodied use is within reach.
  • Cross-dataset results on simulation (H3O) and an indoor dataset (NYUv2) indicate the framework generalizes beyond the real-world campus split it was tuned on.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The benchmark's class split makes the strongest generalization claim hard to separate from the frozen encoder's priors: road, building, and vehicle cover ~68% of voxels in QuadOcc, and these are exactly the categories a web-scale vision-language encoder knows well. A held-out split using rare classes would test the mechanism itself.
  • Editorial inference: Because NMA decouples the text branch from training gradients, the same trained model should accept an arbitrarily large vocabulary at inference; measuring mIoU with 50–100 class names, including fine-grained categories, would stress this property.
  • Editorial inference: The spiral scan ordering is not specific to occupancy; the same PsM-style ordering could be plugged into panoramic BEV mapping or segmentation models, where polar density varies the same way.
  • Editorial inference: The FoV robustness result (performance degrades gracefully from 360° down to 90°) suggests the framework may transfer directly to fisheye and pinhole cameras without architectural change, which is testable by fine-tuning on standard automotive datasets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. O3N proposes a single-image omnidirectional open-vocabulary occupancy prediction framework. It introduces a Polar-spiral Mamba (PsM) module for cylindrical-voxel modeling, an Occupancy Cost Aggregation (OCA) module for refining voxel–text cost volumes, and a gradient-free Natural Modality Alignment (NMA) mechanism that iteratively aligns text embeddings with semantic prototypes. Experiments on QuadOcc and Human360Occ report mIoU of 16.54 (21.16 novel mIoU) on QuadOcc and 24.25 on H3O, outperforming the OVO baseline, and ablations show consistent gains from each component. The paper claims to introduce the omnidirectional open-vocabulary occupancy task for the first time and to achieve state-of-the-art performance.

Significance. If the empirical claims are robust, this is a useful contribution: it extends open-vocabulary occupancy prediction to 360° single-image input, proposes a polar-spiral scanning mechanism adapted to ERP geometry, and reports internally consistent ablation improvements. The framework is modular, the comparisons with OVO are reproducible in principle, and the source code is promised. However, the headline generalization results rest on an evaluation protocol whose "novel" classes are dominated by the largest, easiest semantic regions, and the key hyperparameter β is selected on the validation benchmark. Until these concerns are addressed, the central open-vocabulary claim is not fully established.

major comments (5)
  1. [Sec. 4.1 (Class Split)] The QuadOcc novel set is vehicle (0.90%), road (52.34%), and building (14.00%), i.e., about 67% of all voxels; the H3O novel set is about 75%, dominated by road and sidewalk. The model is trained only on rare base classes such as person (2.09%) in QuadOcc. The reported novel mIoU is therefore largely a score on large, low-intra-class-variance structures that are heavily represented in CLIP's pretraining distribution. Following OVO's split does not remove the confound: per-class novel mIoU and an additional split with rare or genuinely hard held-out classes are needed to support the open-vocabulary generalization claim.
  2. [Sec. 3.3 / Fig. 7] β is swept over {0.01, 0.05, 0.1, 0.2, 0.3} and the value maximizing QuadOcc validation novel mIoU is selected (β = 0.1). Since the same validation set is used for the headline results, the reported gains may partly reflect hyperparameter tuning on the evaluation benchmark. A fixed β chosen on a separate split, or a sensitivity analysis with error bars, is required before the +2.21/+3.01 improvements can be taken at face value.
  3. [Sec. 3.3 (novel prototypes)] The paper states: "we also introduce learnable prototypes for novel classes and initialize them to implicitly capture the unseen semantics." If novel-class text embeddings or prototypes participate in training, the held-out classes are not truly unseen during training. If they do not, the mechanism needs a precise description: initialization, supervision signal, and whether any novel-class text labels are used. As written, the risk of unseen-semantic leakage is unresolved and directly affects the open-vocabulary claim.
  4. [Tables 1–2 vs. Related Work] Among open-vocabulary occupancy methods, only OVO is compared numerically, while the related work lists several other open-vocabulary methods (POP-3D, OpenOcc, VEON, AGO, etc.). If those methods are not directly comparable due to input modality or benchmark scope, the state-of-the-art claim should be qualified. In addition, no error bars or multi-seed results are reported, so the statistical significance of the improvements over OVO is unknown.
  5. [Sec. D.2 (Failure Case Analysis)] The supplementary failure analysis concedes that the adopted FLoSP projection is "insensitive to overall spatial depth" and causes spatial misalignment, as shown in Fig. S7. Since O3N relies on FLoSP, the paper's emphasis on accurate spatial geometry and geometric-semantic consistency should be tempered, or supplemented with quantitative evidence on depth-sensitive metrics. This is an honest limitation, but it weakens one of the central claims if left unqualified in the main text.
minor comments (5)
  1. [Table 1 caption] Typo: "undeline" should be "underline".
  2. [Eq. (8)] The summation uses k both as the index and in the upper limit; the index should be renamed. Also state the convergence condition for the Neumann series in Eq. (10) (e.g., norm of β²A < 1).
  3. [Tables 3–4] Table 3 reports memory in GB during inference (4.97 GB), while Table 4 reports 17.53 GB, apparently training memory. Label the columns as inference/training memory to avoid confusion.
  4. [Sec. 4.2] The phrase "trained with only 30% of base-class annotations" is unclear: base classes are 3 of 6 classes, roughly 32.8% of voxels, not 30% of annotations. Please clarify what the percentage refers to.
  5. [Sec. 3.3] NMA is described as involving a "walking probability" β, but Eqs. (6)–(7) are deterministic interpolations. Rename or explain the stochastic interpretation.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation chain; novel-class evaluation is tunable but not by-construction self-referential.

full rationale

The central derivation is not circular. PsM, OCA, and NMA are trained with base-class labels only (Sec. 3.4: "For voxels belonging to novel classes, we solely possess their geometric positions without semantic information... we aggregate all novel classes into a new class"), and novel inference combines CLIP text similarities with OCA probabilities. NMA's closed form (Eq. 10) is a Neumann-series rearrangement of its own update equations, not an encoding of the headline metric. The QuadOcc/H3O class split follows external setups [25,48] rather than an O3N-derived split; comparisons include external NYUv2 and cross-city H3O-Heter, so the benchmark is not merely self-citation. Beta is swept on QuadOcc validation (Fig. 7), which is selection bias, but the same model is evaluated on held-out splits (H3O-Heter, NYUv2), so the result does not reduce to the sweep. Self-citations to OneOcc [43] provide datasets and the FLoSP projection choice; they are not invoked as a uniqueness theorem and are not load-bearing. The limitation section D.1 explicitly acknowledges dependence on the vision-language model, which is consistent with the intended use of CLIP priors rather than a hidden circular step.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The central method relies on standard CLIP embeddings, a fixed projection strategy, and several hand-tuned hyperparameters. The most significant ledger entries are the benchmark split (dominant classes as 'novel') and the underspecified novel-class prototypes, which carry the open-vocabulary generalization claim.

free parameters (6)
  • β (NMA walking probability) = 0.1
    Selected as the global peak of novel-class mIoU in Fig. 7; controls the random-walk alignment strength in Eqs. 6–10 and directly gates the reported novel-class gains.
  • α (EMA update rate) = 0.9
    Set by hand for prototype EMA update in Eq. 4; affects stability of seen-class prototypes.
  • λ (affinity scaling factor) = 1.0
    Set by hand in Eq. 5; scales the text-prototype affinity matrix used by NMA.
  • d_o (OCA cost embedding dimension) = 128
    Dimension of the occupancy cost embedding in Sec. 3.2; chosen by convention following CAT-Seg.
  • Cylindrical voxel resolution (R, P, Z) = 32 × 90 × 8
    Chosen via ablation in Table 4 as the best trade-off between novel mIoU and memory; directly affects PsM and final metrics.
  • Weight decay coefficient = 1e-3
    Adjusted from 1e-4 to 1e-3 'to mitigate overfitting' (Sec. B.2); a hand-tuned regularizer that affects final performance.
assumptions (5)
  • domain assumption Frozen CLIP and LSeg embeddings provide a reliable shared pixel-voxel-text semantic space.
    The entire open-vocabulary machinery (Eqs. 2, 4–10) assumes cosine similarity in this embedding space transfers from 2D pixels to 3D voxels; the paper acknowledges dependence on vision-language models in Sec. D.1.
  • domain assumption FLoSP 2D-to-3D projection preserves enough geometry for occupancy prediction.
    The method uses FLoSP rather than learnable depth-based lifting; Sec. D.2 concedes that FLoSP is insensitive to spatial depth and causes structural errors.
  • standard math The Neumann series inverse in Eq. 10 is well-defined.
    Eq. 10 requires (I − β²A) to be invertible; the paper does not state conditions on β or the affinity matrix A for convergence.
  • domain assumption The dataset labels and the novel/base split constitute a valid zero-shot generalization test.
    Sec. 4.1 defines novel classes as unseen; all headline evaluation depends on this protocol, which assigns the most frequent classes to the novel set.
  • domain assumption Spatial-Mamba's structure-aware SSM generalizes to the polar-spiral scan path.
    PsM uses Spatial-Mamba on a custom polar scan; the paper provides no proof that the spiral path preserves spatial continuity, only empirical ablations.
invented entities (1)
  • Learnable novel-class prototypes P_l
    purpose: Used in NMA to 'implicitly capture unseen semantics' and align text embeddings for novel classes.
    No loss function or supervision for these prototypes is specified in Sec. 3.3. If they are optimized using novel-class information, the open-vocabulary claim is weakened; this is an underspecified internal construct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Urban Autonomous Agents." pith.science (2026). https://pith.science/paper/DZDD3LTD

@misc{pith2026260312144,
  author       = {Pith},
  title        = {Pith review of: O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Urban Autonomous Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DZDD3LTD}},
  note         = {Machine review of arXiv:2603.12144}
}
read the original abstract

Understanding and reconstructing the 3D world through omnidirectional perception is becoming increasingly important for intelligent transportation and urban autonomous agents operating in shared spaces. However, existing vision-based 3D occupancy prediction methods are constrained by limited perspective inputs and a predefined training distribution, making them difficult to apply to embodied agents that require comprehensive and safe perception of scenes in open-world exploration. To address this, we present O3N, the first framework for open-vocabulary occupancy prediction from a single omnidirectional RGB image. O3N embeds omnidirectional voxels in a polar-spiral topology via the Polar-spiral Mamba (PsM) module, enabling continuous spatial representation and long-range context modeling across 360{\deg}. The Occupancy Cost Aggregation (OCA) module introduces a principled mechanism for unifying geometric and semantic supervision within the voxel space, ensuring consistency between the reconstructed geometry and the underlying semantic structure. Moreover, Natural Modality Alignment (NMA) establishes a gradient-free alignment pathway that harmonizes visual features, voxel embeddings, and text semantics, forming a consistent ``pixel-voxel-text'' representation triad. Extensive experiments on multiple models demonstrate that our method not only achieves state-of-the-art performance on QuadOcc and Human360Occ benchmarks but also exhibits remarkable cross-scene generalization and semantic scalability, highlighting the potential of O3N for scalable open-world 3D scene understanding. The source code will be made publicly available at https://github.com/MengfeiD/O3N

Figures

Figures reproduced from arXiv: 2603.12144 by the authors.

Figure 1
Figure 1. (a) Omnidirectional Open-vocabulary Occupancy Prediction. The pro￾posed O3N is capable of predicting semantic categories that were not labeled during training by providing only an omnidirectional RGB image as input, along with the required class name text. For example, a closed-set semantic occupancy prediction model might misclassify a box as road or a dog as a bicycle, whereupon the cor￾responding class names can … view at source ↗
Figure 2
Figure 2. (a) Omnidirectional open-vocabulary occupancy prediction aims to perform open-vocabulary 3D occupancy prediction solely based on pure omnidirectional visual perception; (b) Regions distant from the viewpoint occupy a smaller proportion of pixels in the image due to the perspective effect and latitude distortion inherent in the equirectangular (ERP) projection; (c) Projection of voxel centers in 3D space onto an omni… view at source ↗
Figure 3
Figure 3. O3N architecture. O3N takes equirectangular omnidirectional image as in￾put and is fully end-to-end trained. The image features and text embeddings are pre￾extracted via the language-image encoder. The 3D decoder, enhanced with the proposed Polar-spiral Mamba (PsM) module, captures both geometric and semantic dependen￾cies across directions. The Occupancy Cost Aggregation (OCA) and Natural Modality Alignment (NMA) m… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Polar-spiral Mamba module uti￾lizes a dual-branch architecture to effec￾tively model the spatial structure of om￾nidirectional images. P-SMamba scans the space in an outward spiral pattern, pre￾cisely capturing variations in information density within polar regions. Vo…
Figure 5
Figure 5. Figure 5: Qualitative results. O3N more effectively maintains the clarity and continu￾ity of global geometry and semantics, and achieves significant improvements over the baseline in terms of perception and generalization to unknown semantics. 4.3 Ablation Studies To validate th…
Figure 6
Figure 6. Figure 6: Distribution of similarity scores between voxels and text em￾beddings with and without NMA during inference. 13.82 15.69 16.54 14.84 14.01 15.86 20.54 21.16 17.93 16.93 0.01 0.05 0.1 0.2 0.3 0 5 10 15 20 25 % β mIoU mIoU-Novel mIoU-Base [PITH_FULL_IMAGE:figures/full_f…

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

94 extracted references · 10 linked inside Pith

  1. [1]

    Ai, H., Cao, Z., Wang, L.: A survey of representation learning, optimization strate- gies,andapplicationsforomnidirectionalvision.InternationalJournalofComputer Vision (2025) 2

    2025

  2. [2]

    In: 3DV (2025) 2, 4

    Boeder, S., Gigengack, F., Risse, B.: LangOcc: Open vocabulary occupancy esti- mation via volume rendering. In: 3DV (2025) 2, 4

    2025

  3. [3]

    In: CVPR (2021) 8

    Cai, Z., Ravichandran, A., Maji, S., Fowlkes, C., Tu, Z., Soatto, S.: Exponential moving average normalization for self-supervised and semi-supervised learning. In: CVPR (2021) 8

    2021

  4. [4]

    In: CVPR (2022) 2, 4, 7, 9, 10, 11, 23, 27

    Cao, A.Q., de Charette, R.: MonoScene: Monocular 3D semantic scene completion. In: CVPR (2022) 2, 4, 7, 9, 10, 11, 23, 27

    2022

  5. [5]

    In: NeurIPS (2024) 4

    Cao Dinh, D., Kim, S.J., Cho, K.: Geometric exploitation for indoor panoramic semantic segmentation. In: NeurIPS (2024) 4

    2024

  6. [6]

    In: ECCV (2018) 7, 22

    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: ECCV (2018) 7, 22

    2018

  7. [7]

    In: CVPR (2024) 2, 3, 7

    Cho, S., Shin, H., Hong, S., Arnab, A., Seo, P.H., Kim, S.: CAT-Seg: Cost aggre- gation for open-vocabulary semantic segmentation. In: CVPR (2024) 2, 3, 7

    2024

  8. [8]

    arXiv preprint arXiv:2507.20217 (2025) 4 16 M

    Cui, W., Wang, H., Qin, W., Guo, Y., Han, G., Zhao, W., Cao, J., Zhang, Z., Zhong, J., Sun, J., Sun, P., Shi, S., Jiang, B., Ma, J., Wang, J., Cheng, H., Liu, Z., Wang, Y., Zhu, Z., Huang, G., Tang, J., Zhang, Q.: Humanoid occupancy: Enabling a generalized multimodal occupancy perception system on humanoid robots. arXiv preprint arXiv:2507.20217 (2025) 4 ...

    arXiv 2025

Show all 94 references
  1. [9]

    In: CVPR (2025) 4

    Duan, Z., Dang, C., Hu, X., An, P., Ding, J., Zhan, J., Xu, Y., Ma, J.: SDGOCC: Semantic and depth-guided bird’s-eye view transformation for 3D multimodal oc- cupancy prediction. In: CVPR (2025) 4

    2025

  2. [10]

    In: ICRA (2025) 4

    E, W., Yuan, C., Li, L., Sun, Y., Gaus, Y.F.A., Atapour-Abarghouei, A., Breckon, T.P.: Dur360BEV: A real-world 360-degree single camera dataset and benchmark for bird-eye view mapping in autonomous driving. In: ICRA (2025) 4

    2025

  3. [11]

    Gao, Y., Xiang, X., Zhong, S., Wang, G.: LOC: A general language-guided frame- workforopen-set3Doccupancyprediction.arXivpreprintarXiv:2510.22141(2025) 4, 5

    2025

  4. [12]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2006) 8

    Grady, L.: Random walks for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2006) 8

    2006

  5. [13]

    In: WACV (2024) 4

    Guttikonda, S., Rambach, J.: Single frame semantic segmentation using multi- modal spherical images. In: WACV (2024) 4

    2024

  6. [14]

    IEEE Transactions on Instrumentation and Measurement (2022) 4

    Hu, X., An, Y., Shao, C., Hu, H.: Distortion convolution module for semantic segmentation of panoramic images based on the image-forming principle. IEEE Transactions on Instrumentation and Measurement (2022) 4

    2022

  7. [15]

    In: CVPR (2024) 4

    Huang, Y., Zheng, W., Zhang, B., Zhou, J., Lu, J.: SelfOcc: Self-supervised vision- based 3D occupancy prediction. In: CVPR (2024) 4

    2024

  8. [16]

    In: CVPR (2023) 2, 4

    Huang, Y., Zheng, W., Zhang, Y., Zhou, J., Lu, J.: Tri-perspective view for vision- based 3D semantic occupancy prediction. In: CVPR (2023) 2, 4

    2023

  9. [17]

    In: ECCV (2024) 4

    Huang, Y., Zheng, W., Zhang, Y., Zhou, J., Lu, J.: GaussianFormer: Scene as gaussians for vision-based 3D semantic occupancy prediction. In: ECCV (2024) 4

    2024

  10. [18]

    In: IROS (2024) 2, 4, 5

    Jiang, H., Xu, Y., Zeng, Y., Xu, H., Zhang, W., Feng, J., Zhang, L.: OpenOcc: Open vocabulary 3D scene reconstruction via occupancy representation. In: IROS (2024) 2, 4, 5

    2024

  11. [19]

    In: ICASSP (2025) 4

    Jiang, J., Zhu, J., Xu, Z., Chen, X., Zhao, S., Yao, H.: Gaussian constrained diffeo- morphic deformation network for panoramic semantic segmentation. In: ICASSP (2025) 4

    2025

  12. [20]

    In: ICML (2020) 7, 22

    Katharopoulos, A., Vyas, A., Pappas, N., Fleuret, F.: Transformers are RNNs: Fast autoregressive transformers with linear attention. In: ICML (2020) 7, 22

    2020

  13. [21]

    In: ICIP (2022) 4

    Kim, J., Jeong, S., Sohn, K.: PASTS: Toward effective distilling transformer for panoramic semantic segmentation. In: ICIP (2022) 4

    2022

  14. [22]

    In: ICIP (2025) 4

    Lan, B., Yang, L., Xu, M., Jiang, L., Wang, Y.: Deformable spherical geometry transformer for panoramic semantic segmentation. In: ICIP (2025) 4

    2025

  15. [23]

    In: ICLR (2022) 2, 24

    Li, B., Weinberger, K.Q., Belongie, S.J., Koltun, V., Ranftl, R.: Language-driven semantic segmentation. In: ICLR (2022) 2, 24

    2022

  16. [24]

    arXiv preprint arXiv:2509.13681 (2025) 4

    Li, H., Sheng, D., Dong, Q., Wang, Z., Xu, Z., Li, T.: FishBEV: Distortion-resilient bird’s eye view segmentation with surround-view fisheye cameras. arXiv preprint arXiv:2509.13681 (2025) 4

    2025

  17. [25]

    arXiv preprint arXiv:2504.10117 (2025) 4, 10

    Li, P., Ding, S., Zhou, Y., Zhang, Q., Inak, O., Triess, L., Hanselmann, N., Cordts, M., Zell, A.: AGO: Adaptive grounding for open world 3D occupancy prediction. arXiv preprint arXiv:2504.10117 (2025) 4, 10

    2025

  18. [26]

    In: IJCAI (2023) 4

    Li, X., Wu, T., Qi, Z., Wang, G., Shan, Y., Li, X.: SGAT4PASS: Spherical geometry-aware transformer for panoramic semantic segmentation. In: IJCAI (2023) 4

    2023

  19. [27]

    In: CVPR (2023) 4, 10, 11, 27

    Li, Y., Yu, Z., Choy, C.B., Xiao, C., Álvarez, J.M., Fidler, S., Feng, C., Anandku- mar, A.: VoxFormer: Sparse voxel transformer for camera-based 3D semantic scene completion. In: CVPR (2023) 4, 10, 11, 27

    2023

  20. [28]

    IEEE Transactions on Pattern Analysis and Machine Intelligence47(3), 2020–2036 (2024) 27

    Li, Z., Wang, W., Li, H., Xie, E., Sima, C., Lu, T., Yu, Q., Dai, J.: Bevformer: learningbird’s-eye-viewrepresentationfromlidar-cameraviaspatiotemporaltrans- O3N 17 formers. IEEE Transactions on Pattern Analysis and Machine Intelligence47(3), 2020–2036 (2024) 27

    2020

  21. [29]

    arXiv preprint arXiv:2509.04444 (2025) 1, 2, 4

    Lin, X., Ge, X., Zhang, D., Wan, Z., Wang, X., Li, X., Jiang, W., Du, B., Tao, D., Yang, M.H., Qi, L.: One flight over the gap: A survey from perspective to panoramic vision. arXiv preprint arXiv:2509.04444 (2025) 1, 2, 4

    2025 arXiv

  22. [30]

    IEEE Robotics and Automation Letters (2025) 4

    Liu, W., Wang, W.: ArticuBEVSeg: Road semantic understanding and its applica- tion in bird’s eye view from panoramic vision system of long combination vehicles. IEEE Robotics and Automation Letters (2025) 4

    2025

  23. [31]

    In: ITSC (2021) 4

    Ma, C., Zhang, J., Yang, K., Roitberg, A., Stiefelhagen, R.: DensePASS: Dense panoramic semantic segmentation via unsupervised domain adaptation with attention-augmented context exchange. In: ITSC (2021) 4

    2021

  24. [32]

    In: CVPR (2024) 2, 4

    Ma, Q., Tan, X., Qu, Y., Ma, L., Zhang, Z., Xie, Y.: COTR: Compact occupancy transformer for vision-based 3D occupancy prediction. In: CVPR (2024) 2, 4

    2024

  25. [33]

    IEEE Transactions on Image Processing (2024) 9, 10, 11, 23, 27

    Mei, J., Yang, Y., Wang, M., Zhu, J., Ra, J., Ma, Y., Li, L., Liu, Y.: Camera-based 3D semantic scene completion with sparse guidance network. IEEE Transactions on Image Processing (2024) 9, 10, 11, 23, 27

    2024

  26. [34]

    SIAM (2023) 9

    Meyer, C.D.: Matrix analysis and applied linear algebra. SIAM (2023) 9

    2023

  27. [35]

    arXiv preprint arXiv:2505.03284 (2025) 3

    Ming, Z., Berrio, J.S., Shan, M., Huang, Y., Lyu, H., Tran, N.H.K., Tseng, T.Y., Worrall, S.: OccCylindrical: Multi-modal fusion with cylindrical representation for 3D semantic occupancy prediction. arXiv preprint arXiv:2505.03284 (2025) 3

    2025 arXiv

  28. [36]

    In: CVPR (2025) 4

    Oh, G., Kim, S., Ko, H., Chi, H.g., Kim, J., Lee, D., Ji, D., Choi, S., Jang, S., Kim, S.: 3D occupancy prediction with low-resolution queries via prototype-aware view transformation. In: CVPR (2025) 4

    2025

  29. [37]

    Signal, Image and Video Processing (2022) 4

    Orhan, S., Bastanlar, Y.: Semantic segmentation of outdoor panoramic images. Signal, Image and Video Processing (2022) 4

    2022

  30. [38]

    In: IROS (2024) 4

    Pan, X., Du, J., Su, S., Zong, W., Wang, X., Liu, C., Chen, Q.: GenerOcc: Self- supervised framework of real-time 3D occupancy prediction for monocular generic cameras. In: IROS (2024) 4

    2024

  31. [39]

    In: European conference on computer vision

    Philion, J., Fidler, S.: Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d. In: European conference on computer vision. pp. 194–210. Springer (2020) 27

    2020

  32. [40]

    In: ICML (2021) 4, 24

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: ICML (2021) 4, 24

    2021

  33. [41]

    In: 3DV (2020) 10, 27

    Roldão, L., de Charette, R., Verroust-Blondet, A.: LMSCNet: Lightweight multi- scale 3D semantic completion. In: 3DV (2020) 10, 27

    2020

  34. [42]

    In: IROS (2023) 4

    Samani, E.U., Tao, F., Reddy, D.H., Ding, S., Banerjee, A.G.: F2BEV: Bird’s eye view generation from surround-view fisheye camera images for automated driving. In: IROS (2023) 4

    2023

  35. [43]

    In: CVPR (2026) 2, 3, 4, 9, 10, 11, 23, 25, 27

    Shi, H., Wang, Z., Guo, S., Duan, M., Wang, S., Chen, T., Yang, K., Wang, L., Wang, K.: OneOcc: Semantic occupancy prediction for legged robots with a single panoramic camera. In: CVPR (2026) 2, 3, 4, 9, 10, 11, 23, 25, 27

    2026

  36. [44]

    In: ECCV (2024) 4

    Shi, Y., Cheng, T., Zhang, Q., Liu, W., Wang, X.: Occupancy as set of points. In: ECCV (2024) 4

    2024

  37. [45]

    In: CVPR (2017) 10, 11

    Song, S., Yu, F., Zeng, A., Chang, A.X., Savva, M., Funkhouser, T.A.: Semantic scene completion from a single depth image. In: CVPR (2017) 10, 11

    2017

  38. [46]

    In: NeurIPS (2025) 2

    Sun, Y., Contreras, J., Ortiz, J.: Dynamic focused masking for autoregressive em- bodied occupancy prediction. In: NeurIPS (2025) 2

    2025

  39. [47]

    In: WACV (2025) 4 18 M

    Tan, T., Chen, B., Cao, H., Yan, C., Ma, Y., Dai, F.: DASC-SPT: Towards self- supervised panoramic semantic segmentation. In: WACV (2025) 4 18 M. Duan, H. Shiet al

    2025

  40. [48]

    arXiv preprint arXiv:2305.16133 (2023) 2, 3, 4, 8, 9, 10, 11, 23, 24, 25, 27, 28

    Tan, Z., Dong, Z., Zhang, C., Zhang, W., Ji, H., Li, H.: OVO: Open-vocabulary occupancy. arXiv preprint arXiv:2305.16133 (2023) 2, 3, 4, 8, 9, 10, 11, 23, 24, 25, 27, 28

    2023 arXiv

  41. [49]

    In: CVPR (2024) 4

    Tang, P., Wang, Z., Wang, G., Zheng, J., Ren, X., Feng, B., Ma, C.: SparseOcc: Rethinking sparse latent representation for vision-based semantic occupancy pre- diction. In: CVPR (2024) 4

    2024

  42. [50]

    In: WACV (2024) 4

    Teng, Z., Zhang, J., Yang, K., Peng, K., Shi, H., Reiß, S., Cao, K., Stiefelhagen, R.: 360BEV: Panoramic semantic mapping for indoor bird’s-eye view. In: WACV (2024) 4

    2024

  43. [51]

    In: NeurIPS (2023) 2

    Tian, X., Jiang, T., Yun, L., Mao, Y., Yang, H., Wang, Y., Wang, Y., Zhao, H.: Occ3D: A large-scale 3D occupancy prediction benchmark for autonomous driving. In: NeurIPS (2023) 2

    2023

  44. [52]

    In: ICCV (2023) 4

    Tong, W., Sima, C., Wang, T., Chen, L., Wu, S., Deng, H., Gu, Y., Lu, L., Luo, P., Lin, D., Li, H.: Scene as occupancy. In: ICCV (2023) 4

    2023

  45. [53]

    In: NeurIPS (2023) 2, 4, 5

    Vobecky, A., Siméoni, O., Hurych, D., Gidaris, S., Bursuc, A., Pérez, P., Sivic, J.: POP-3D: Open-vocabulary 3D occupancy prediction from images. In: NeurIPS (2023) 2, 4, 5

    2023

  46. [54]

    In: ECCV (2024) 2

    Wang, G., Wang, Z., Tang, P., Zheng, J., Ren, X., Feng, B., Ma, C.: OccGen: Gen- erative multi-modal 3D occupancy prediction for autonomous driving. In: ECCV (2024) 2

    2024

  47. [55]

    In: MM (2025) 2

    Wang, H., Wei, X., Zhang, X., Li, J., Bai, C., Li, Y., Lu, M., Zheng, W., Zhang, S.: EmbodiedOcc++: Boosting embodied 3D occupancy prediction with plane regu- larization and uncertainty sampler. In: MM (2025) 2

    2025

  48. [56]

    In: ICRA (2025) 10

    Wang, J., Yin, W., Long, X., Zhang, X., Xing, Z., Guo, X., Zhang, Q.: OccRWKV: Rethinking efficient 3D semantic occupancy prediction with linear complexity. In: ICRA (2025) 10

    2025

  49. [57]

    In: ICCV (2023) 2

    Wang, X., Zhu, Z., Xu, W., Zhang, Y., Wei, Y., Chi, X., Ye, Y., Du, D., Lu, J., Wang, X.: OpenOccupancy: A large scale benchmark for surrounding semantic occupancy perception. In: ICCV (2023) 2

    2023

  50. [58]

    In: ACCV (2024) 4

    Wei, J., Zheng, J., Liu, R., Hu, J., Zhang, J., Stiefelhagen, R.: OneBEV: Using one panoramic image for bird’s-eye-view semantic mapping. In: ACCV (2024) 4

    2024

  51. [59]

    In: ICCV (2023) 2, 4

    Wei, Y., Zhao, L., Zheng, W., Zhu, Z., Zhou, J., Lu, J.: SurroundOcc: Multi-camera 3D occupancy prediction for autonomous driving. In: ICCV (2023) 2, 4

    2023

  52. [60]

    IEEE Access (2025) 3, 4

    Wu, C., Li, J., Cao, J., Li, M., Du, S., Li, Y.: OmniOcc: Cylindrical voxel-based semantic occupancy prediction for omnidirectional vision systems. IEEE Access (2025) 3, 4

    2025

  53. [61]

    In: ICCV (2025) 2

    Wu, Y., Zheng, W., Zuo, S., Huang, Y., Zhou, J., Lu, J.: EmbodiedOcc: Embodied 3D occupancy prediction for vision-based online scene understanding. In: ICCV (2025) 2

    2025

  54. [62]

    In: ICLR (2025) 6, 12

    Xiao, C., Li, M., Zhang, Z., Meng, D., Zhang, L.: Spatial-Mamba: Effective visual state space models via structure-aware state fusion. In: ICLR (2025) 6, 12

    2025

  55. [63]

    In: CVPR (2024) 2, 3

    Xie, B., Cao, J., Xie, J., Khan, F.S., Pang, Y.: SED: A simple encoder-decoder for open-vocabulary semantic segmentation. In: CVPR (2024) 2, 3

    2024

  56. [64]

    Displays (2025) 4

    Xu, J., Xu, C., Zhao, J., Han, C., Li, H.: Mamba4PASS: Vision mamba for panoramic semantic segmentation. Displays (2025) 4

    2025

  57. [65]

    arXiv preprint arXiv:2510.04759 (2025) 4

    Yan, C., Xu, D.: Progressive gaussian transformer with anisotropy-aware sampling for open vocabulary occupancy prediction. arXiv preprint arXiv:2510.04759 (2025) 4

    2025

  58. [66]

    IEEE Transactions on Intelligent Transportation Systems (2020) 4 O3N 19

    Yang, K., Hu, X., Bergasa, L.M., Romera, E., Wang, K.: PASS: Panoramic annular semantic segmentation. IEEE Transactions on Intelligent Transportation Systems (2020) 4 O3N 19

    2020

  59. [67]

    In: IV (2020) 4

    Yang, K., Hu, X., Chen, H., Xiang, K., Wang, K., Stiefelhagen, R.: DS-PASS: Detail-sensitive panoramic annular semantic segmentation through SwaftNet for surrounding sensing. In: IV (2020) 4

    2020

  60. [68]

    arXiv preprint arXiv:2501.15384 (2025) 4

    Yang,L.,Zheng,L.,Ai,W.,Liu,M.,Li,S.,Lin,Q.,Yan,S.,Bai,J.,Ma,Z.,Zhu,X.: MetaOcc: Spatio-temporal fusion of surround-view 4D radar and camera for 3D oc- cupancy prediction with dual training strategies. arXiv preprint arXiv:2501.15384 (2025) 4

    2025 arXiv

  61. [69]

    IEEE Transactions on Circuits and Systems for Video Technology (2025) 2

    Yang, T., Qian, Y., Yan, W., Wang, C., Yang, M.: AdaptiveOcc: Adaptive octree- based network for multi-camera 3D semantic occupancy prediction in autonomous driving. IEEE Transactions on Circuits and Systems for Video Technology (2025) 2

    2025

  62. [70]

    In: CVPRW (2024) 4

    Yogamani, S., Unger, D., Narayanan, V., Kumar, V.R.: FisheyeBEVSeg: Surround view fisheye cameras based bird’s-eye view segmentation for autonomous driving. In: CVPRW (2024) 4

    2024

  63. [71]

    In: ICCV (2025) 4, 5

    Yu, Z., Pang, B., Liu, L., Zhang, R., Peng, Q., Luo, M., Yang, S., Chen, M., Cao, S.Y., Shen, H.L.: Language driven occupancy prediction. In: ICCV (2025) 4, 5

    2025

  64. [72]

    IEEE Signal Processing Letters (2023) 4

    Yuan, Z., Wang, J., Lv, Y., Wang, D., Fang, Y.: Laformer: Vision transformer for panoramic image semantic segmentation. IEEE Signal Processing Letters (2023) 4

    2023

  65. [73]

    In: WACV (2025) 4

    Zhang, J., Ding, Y., Liu, Z.: OccLoff: Learning optimized feature fusion for 3D occupancy prediction. In: WACV (2025) 4

    2025

  66. [74]

    In: CVPR (2022) 4

    Zhang, J., Yang, K., Ma, C., Reiß, S., Peng, K., Stiefelhagen, R.: Bending reality: Distortion-aware transformers for adapting to panoramic semantic segmentation. In: CVPR (2022) 4

    2022

  67. [75]

    IEEE Transac- tions on Pattern Analysis and Machine Intelligence (2024) 2, 4

    Zhang, J., Yang, K., Shi, H., Reiß, S., Peng, K., Ma, C., Fu, H., Torr, P.H.S., Wang, K., Stiefelhagen, R.: Behind every domain there is a shift: Adapting distortion- aware vision transformers for panoramic semantic segmentation. IEEE Transac- tions on Pattern Analysis and Mac...

    2024

  68. [76]

    arXiv preprint arXiv:2503.09010 (2025) 4

    Zhang, Q., Zhang, Z., Cui, W., Sun, J., Cao, J., Guo, Y., Han, G., Zhao, W., Wang, J., Sun, C., Zhang, L., Cheng, H., Chen, Y., Wang, L., Tang, J., Xu, R.: HumanoidPano: Hybrid spherical panoramic-LiDAR cross-modal perception for humanoid robots. arXiv preprint arXiv:2503.0901...

    2025 arXiv

  69. [77]

    In: CVPR (2024) 4

    Zhang, W., Liu, Y., Zheng, X., Wang, L.: GoodSAM: Bridging domain and ca- pacity gaps via segment anything model for distortion-aware panoramic semantic segmentation. In: CVPR (2024) 4

    2024

  70. [78]

    arXiv preprint arXiv:2408.09115 (2024) 4

    Zhang, W., Liu, Y., Zheng, X., Wang, L.: GoodSAM++: Bridging domain and capacity gaps via segment anything model for panoramic semantic segmentation. arXiv preprint arXiv:2408.09115 (2024) 4

    2024 arXiv

  71. [79]

    In: ICCV (2023) 4, 10, 11, 27

    Zhang, Y., Zhu, Z., Du, D.: OccFormer: Dual-path transformer for vision-based 3D semantic occupancy prediction. In: ICCV (2023) 4, 10, 11, 27

    2023

  72. [80]

    arXiv preprint arXiv:2504.14604 (2025) 2

    Zhang, Z., Zhang, Q., Cui, W., Shi, S., Guo, Y., Han, G., Zhao, W., Ren, H., Xu, R., Tang, J.: RoboOcc: Enhancing the geometric and semantic scene understanding for robots. arXiv preprint arXiv:2504.14604 (2025) 2

    2025 arXiv

  73. [81]

    arXiv preprint arXiv:2505.05512 (2025) 2

    Zhang, Z., Zhang, Q., Cui, W., Shi, S., Guo, Y., Han, G., Zhao, W., Sun, J., Cao, J., Wang, J., Cheng, H., Ju, X., Che, Z., Xu, R., Tang, J.: Occupancy world model for robots. arXiv preprint arXiv:2505.05512 (2025) 2

    2025 arXiv

  74. [82]

    In: CVPR (2025) 3, 8

    Zhao, Z., Li, X., Shi, L., Imanpour, N., Wang, S.: DPSeg: Dual-prompt cost volume learning for open-vocabulary semantic segmentation. In: CVPR (2025) 3, 8

    2025

  75. [83]

    In: ECCV (2024) 2, 4, 5 20 M

    Zheng, J., Tang, P., Wang, Z., Wang, G., Ren, X., Feng, B., Ma, C.: VEON: Vocabulary-enhanced occupancy prediction. In: ECCV (2024) 2, 4, 5 20 M. Duan, H. Shiet al

    2024

  76. [84]

    arXiv preprint arXiv:2501.15394 (2025) 4

    Zheng, L., Liu, J., Guan, R., Yang, L., Lu, S., Li, Y., Bai, X., Bai, J., Ma, Z., Shen, H., Zhu, X.: Doracamom: Joint 3D detection and occupancy prediction with multi-view 4D radars and cameras for omnidirectional perception. arXiv preprint arXiv:2501.15394 (2025) 4

    2025

  77. [85]

    In: ICCV (2023) 4

    Zheng, X., Pan, T., Luo, Y., Wang, L.: Look at the neighbor: Distortion-aware unsupervised domain adaptation for panoramic semantic segmentation. In: ICCV (2023) 4

    2023

  78. [86]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2025) 4

    Zheng, X., Zhou, P.Y., Vasilakos, A.V., Wang, L.: 360SFUDA++: Towards source- free UDA for panoramic segmentation by learning reliable category prototypes. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025) 4

    2025

  79. [87]

    In: CVPR (2024) 4

    Zheng, X., Zhou, P., Vasilakos, A.V., Wang, L.: Semantics distortion and style matter: Towards source-free UDA for panoramic segmentation. In: CVPR (2024) 4

    2024

  80. [88]

    In: CVPR (2023) 4

    Zheng, X., Zhu, J., Liu, Y., Cao, Z., Fu, C., Wang, L.: Both style and distor- tion matter: Dual-path unsupervised domain adaptation for panoramic semantic segmentation. In: CVPR (2023) 4

    2023

  81. [89]

    In: ICCV (2025) 4

    Zhong, D., Zheng, X., Liao, C., Lyu, Y., Chen, J., Wu, S., Zhang, L., Hu, X.: Om- niSAM: Omnidirectional segment anything model for UDA in panoramic semantic segmentation. In: ICCV (2025) 4

    2025

  82. [90]

    In: CVPR (2026) 4

    Zhou, C., Luo, Y., Zhang, H., Jiang, Z., Chen, C.: Monocular open vocabulary occupancy prediction for indoor scenes. In: CVPR (2026) 4

    2026

  83. [91]

    In: ICCV (2025) 4

    Zhou,X.,Wang,J.,Wang,Y.,Wei,Y.,Dong,N.,Yang,M.H.:AutoOcc:Automatic open-ended semantic occupancy annotation via vision-language guided gaussian splatting. In: ICCV (2025) 4

    2025

  84. [92]

    In: ECCV (2024) 4

    Zhu, B., Wang, Z., Li, H.: nuCraft: Crafting high resolution 3D semantic occupancy for unified 3D scene understanding. In: ECCV (2024) 4

    2024

  85. [93]

    In: NeurIPS (2025) 2

    Zuo, S., Zheng, W., Han, X., Yang, L., Pan, Y., Lu, J.: QuadricFormer: Scene as superquadrics for 3D semantic occupancy prediction. In: NeurIPS (2025) 2

    2025

  86. [94]

    pixel-voxel-text

    Zuo, S., Zheng, W., Huang, Y., Zhou, J., Lu, J.: PointOcc: Cylindrical tri- perspective view for point-based 3D semantic occupancy prediction. arXiv preprint arXiv:2308.16896 (2023) 3, 4 O3N 21 In the supplementary materials, we provide comprehensive descriptions of frameworks...

    2023 arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.