Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes 3D-MOOD as the first end-to-end monocular open-set 3D object detector, lifting language-prompted 2D detections into metric 3D boxes.

desk verdict First real end-to-end open-set monocular 3D detector, with strong closed-set numbers and a genuinely new benchmark; the open-set SOTA claim is only as good as the paper's own ODS metric, so add IoU AP and seeds. read the letter →

arxiv 2507.23567 v2 pith:6PVCNI6Q submitted 2025-07-31 cs.CV

Yung-Hsu Yang , Luigi Piccinelli , Mattia Segu , Siyuan Li , Rui Huang , Yuqian Fu , Marc Pollefeys , Hermann Blum
show 1 more author
Zuria Bauer
This is my paper
classification cs.CV
keywords monocular3Dobjectdetectionopen-setopen-vocabularyend-to-endcross-datasetgeneralizationcanonicalimagespacemetricdepthestimationODSevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

3D-MOOD tackles monocular 3D object detection in the open-set regime: one RGB image and a language prompt for the objects of interest should return oriented 3D bounding boxes, including categories and scenes never seen during training. The paper's mechanism is to lift a language-conditioned 2D open-set detector into 3D with a dedicated 3D bounding box head, so the 2D and 3D tasks are trained jointly and end-to-end rather than stitched together by pseudo-labels. It reports that this design is the first end-to-end open-set monocular 3D detector, that it reaches new state-of-the-art accuracy on the closed-set Omni3D benchmark, and that it consistently beats closed-set baselines and an earlier pseudo-label pipeline on new Argoverse 2 and ScanNet open-set benchmarks. The practical stake is that robots and AR/VR systems could localize arbitrary prompted objects in unfamiliar scenes from one cheap camera, without per-dataset retraining.

What carries the argument

The load-bearing mechanism is the 3D bounding box head and its Lift operation. For every 2D object query, the head outputs a 12-dimensional prediction, comprising a projected-center offset, log-scaled metric depth, log-scaled dimensions, and a 6D rotation, and Lift combines those outputs with the camera intrinsics to produce an oriented 3D box in camera coordinates. Around that head sit two supporting mechanisms: geometry-aware 3D query generation, which replaces plain 2D queries with queries conditioned on camera embeddings and metric depth features so the 3D estimate inherits scene geometry; and the canonical image space, which resize-and-center-pads every training and test image to one fixed resolution while adjusting the intrinsics identically, removing the image-size, intrinsic, and depth ambiguity the paper identifies as the main obstacle to cross-dataset monocular 3D estimation.

What would settle it

Take the released 3D-MOOD and baseline weights, run them on the same Argoverse 2 and ScanNet validation images, and compute class-wise AP3D with IoU3D thresholds 0.05, 0.1, ..., 0.5 as Omni3D does. If Cube R-CNN or OVM3D-Det matches or beats 3D-MOOD under that matching, or if most of 3D-MOOD's distance-matched true positives fail the IoU criterion, then the paper's open-set superiority claim is not robust.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that open-vocabulary classification can be carried from 2D into 3D by geometry regression: a 3D head consumes the decoder queries of a language-conditioned 2D detector and regresses the twelve numbers that define a 3D box. The head predicts the offset from the 2D box center to the projected 3D center, a scaled logarithmic metric depth, log-scaled width, length, and height, and a 6D rotation, then lifts the box into camera coordinates with the camera intrinsics. To transfer across datasets, all images are mapped into a canonical image space with fixed resolution, ratio-preserving resize, center padding, and correspondingly adjusted intrinsics, and the object queries are conditioned on camera embeddings and metric depth features from an auxiliary depth head. Evaluated on Omni3D the model reports around 30% 3D AP with a Swin-B backbone; on the newly proposed open-set benchmarks it reports ODS 23.8 on Argoverse 2 and 31.5 on ScanNet, and it detects novel classes on which the closed-set baselines score zero.

Load-bearing premise

The open-set superiority claim rests on the paper's own ODS metric, which matches predictions to ground truth by center distance with ground-truth-radius thresholds, and on its own Argoverse 2 and ScanNet benchmarks; if the field insists on standard IoU-based 3D AP for open-set evaluation, the reported margins over baselines could shrink or disappear, as the paper itself explains in Sec. 4.2 and Supp. G.

Editorial extensions

If this is right

  • If the central claim is right, open-set 3D detection works from a single image: no LiDAR, stereo pair, or per-scene retraining is required.
  • A model trained once on Omni3D can return 3D boxes for novel classes in unseen indoor and outdoor scenes, which closed-set baselines cannot do.
  • The canonical image space lowers the cost of cross-dataset training while improving transfer, cutting GPU memory from 21–23 GB to 17 GB in the paper's comparison.
  • Distance-based matching with ground-truth-radius thresholds gives small and thin objects a fairer evaluation than IoU3D, so the proposed ODS score is the metric under which the open-set gains are stated.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the architecture splits category recognition from geometry regression, so a stronger 2D open-set detector could likely be plugged into the same 3D head without rebuilding the geometry stack.
  • Beyond the paper: the open-set ranking is metric-dependent; the paper's own supplementary material shows large AP swings between IoU and distance matching, so standard IoU3D evaluation on the new benchmarks could put the baselines closer.
  • Beyond the paper: since the auxiliary depth head gives only a small open-set gain and the paper notes its depth training data are limited, feeding it denser or more diverse metric depth supervision is a direct testable route to larger gains.
  • Beyond the paper: because the 3D head consumes 2D object queries, the same lifting should transfer to video or multi-frame settings, where temporal consistency could stabilize depth and rotation estimates.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces 3D-MOOD, an end-to-end monocular open-set 3D object detector that extends the Grounding DINO 2D open-set detector with a 3D bounding box head, geometry-aware 3D query generation, an auxiliary camera-conditioned metric depth head, and a canonical image space for cross-dataset training. The method is trained on Omni3D and evaluated in closed-set settings on Omni3D and in open-set settings on newly proposed Argoverse 2 and ScanNet benchmarks, where it reports state-of-the-art results under a newly introduced Open Detection Score (ODS). The authors also provide code and models.

Significance. The paper addresses a real gap: existing monocular 3D detectors are closed-set, and the main open-vocabulary alternative, OVM3D-Det, is not end-to-end. The proposed architecture is coherent, and the ablations in Table 4 support each component individually. The closed-set results on Omni3D are strong, with Swin-B reaching 30.0% AP_omni3D and surpassing prior work. The new open-set benchmarks and the ODS metric are potentially useful contributions to the community. However, the headline open-set superiority claim rests entirely on the paper's own distance-based ODS metric and on threshold and weight choices that are not checked against the standard IoU-based AP criterion; the significance of the open-set claim therefore depends on the community's acceptance of ODS and on additional validation that the current manuscript does not provide.

major comments (3)
  1. [Sec. 4.2, Table 1, Supp. G] The open-set state-of-the-art claim is supported only by APdist_3D and ODS, which match predictions to ground truth by 3D center distance with thresholds in [0.5, 1.0] times the GT box radius; the paper never reports standard IoU-based 3D AP on Argoverse 2 or ScanNet. Supp. G, Table 10 shows that the same detections yield very different AP under IoU matching (e.g., Monitor: 0.5 IoU-based vs 9.4 distance-based; Construction cone: 0.5 vs 6.5). Because the distance criterion tolerates depth and extent errors that IoU3D would reject, the ranking against Cube R-CNN and OVM3D-Det could change if IoU3D were applied. Please report IoU-based AP_3D for all methods on both open-set benchmarks and discuss any discrepancies.
  2. [Sec. 4.2, Eq. (6), Table 1, Table 4] ODS is a new composite metric whose weights (3:1:1:1), distance thresholds (0.5–1.0 times GT radius), and choice of true-positive errors are chosen by the authors, but no sensitivity analysis is provided. The paper also reports single runs for all experiments; in Table 1 the Swin-T vs Swin-B difference on Argoverse 2 APdist_3D is 14.8 vs 14.7, and in Table 4 rows 3 and 4 differ by only 1.3 ODS points on the open-set average, so run-to-run variability could alter conclusions. Please provide multiple seeds or bootstrap confidence intervals for the key comparisons and a sensitivity analysis of ODS with respect to its matching thresholds and weighting.
  3. [Sec. 3.5, Eq. (3), Supp. D] The auxiliary depth head reports 9.1% AbsRel on KITTI Eigen, versus 4.2% for UniDepth, yet the geometry-aware 3D query generation of Eq. (3) conditions object queries on the depth features F_d^8|E. The paper does not analyze how this relatively weak auxiliary depth affects the open-set gains attributed to the geometry prior. An ablation that replaces or freezes the depth features with a strong external depth map (e.g., from UniDepth) would clarify whether the ODS improvements come specifically from the geometric conditioning or from other components, and would strengthen the attribution of the open-set gains.
minor comments (6)
  1. [Sec. 1] The term 'Detecter' in '3D Monocular Open-set Object Detecter (3D-MOOD)' should be 'Detector'.
  2. [Sec. 3.7] The phrase 'in conjugation' should be 'in conjunction'.
  3. [Sec. 2.1] The citation 'GLIP [27]' is incorrect because [27] refers to Grounding DINO; GLIP is reference [21], and both are cited with the same number in that paragraph.
  4. [Sec. 4.2, Eq. (6)] The notation 'mTPE ∈ [mATE, mASE, mAOE]' is unconventional; please define mTPE explicitly as the tuple of the three errors and write the summation accordingly.
  5. [Table 1 caption] ODS (B) and ODS (N) are used in the table but are not defined in Sec. 4.2; please define the base/novel class subsets in the main text.
  6. [Supp. C] For reproducibility of the proposed benchmark, the exact frame indices used for 'every 5 frames' (Argoverse 2) and 'maximum 20 frames per scan' (ScanNet) should be released, or a fixed seed specified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the open-set SOTA claim is an empirical benchmark result; the new ODS metric and benchmark are disclosed evaluation choices, not derivation steps built from the model's own outputs.

full rationale

I checked the derivation chain against the seven circularity patterns. The central pipeline is G-DINO 2D detection plus a 3D bounding-box head that lifts 2D boxes using predicted depth, dimensions, and rotation (Eqs. 2-4), trained jointly with Omni3D 2D/3D losses (Eq. 5). The headline claims are supported by external benchmarking: closed-set AP3D on Omni3D against Cube R-CNN and Uni-MODE (Tab. 3), and open-set results against Cube R-CNN and OVM3D-Det on Argoverse 2 and ScanNet (Tab. 1). No parameter is fitted to the open-set labels and then renamed a prediction. The ODS metric (Eq. 6) is indeed proposed by the paper and is the criterion for the open-set SOTA claim, so benchmark acceptance is a legitimate risk, but the metric is defined transparently, borrows from NDS/CDS, and is applied uniformly to all methods; a metric choice is not a derivation that reduces to the model's own outputs. The self-citations to UniDepth (camera-conditioned depth head) and Vis4D (framework) are code-released building blocks, and the paper's Supp. D shows its own depth head is substantially worse than UniDepth (9.1% vs 4.2% AbsRel), so the cited work is not being used to assert the component's effectiveness. The limited benefit of the auxiliary depth head is also acknowledged in the ablation (open-set gain of only 0.2 ODS). I therefore find no circular step that can be exhibited as Eq. X = Eq. Y by construction; concerns about ODS being non-standard or favorable to the method belong to benchmark/correctness risk, not circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. The free parameters are training and evaluation hyperparameters. The ODS metric, while new, is an evaluation instrument rather than a postulated entity. The axioms are standard domain assumptions for monocular 3D detection.

free parameters (4)
  • depth scale s_depth
    Scale used to convert predicted log depth to metric depth (exp(d/s_depth)). Value not stated; chosen by hand, affects all depth and 3D box predictions.
  • dimension scale s_dim
    Scale used for log dimension outputs for width, length, height; not reported.
  • canonical image resolution = 800x1333
    Fixed input shape chosen for training and inference; affects intrinsics padding and depth ambiguity.
  • ODS metric weights and matching thresholds = 1/6, [0.5,0.55,...,1.0] radius
    The open detection score weights and distance thresholds are defined by the authors; the SOTA claims on the open-set benchmark depend on these choices.
assumptions (4)
  • domain assumption Monocular 3D detection is feasible from a single image with known intrinsics.
    The entire task assumes metric depth and 3D boxes can be estimated from RGB; this is standard in the field.
  • domain assumption Open-set 2D detection (G-DINO) provides reliable pseudo-lifting targets.
    The 3D head lifts 2D detections; the paper assumes the 2D detector's boxes, classes, and queries are accurate enough to supervise 3D estimation.
  • domain assumption Depth features conditioned on camera embeddings generalize across domains.
    Geometry-aware queries rely on depth features from the auxiliary head; the paper assumes these features, trained on Omni3D, transfer to Argoverse 2 and ScanNet.
  • domain assumption Omni3D depth ground truth (LiDAR or SfM projections) is reliable for auxiliary loss.
    The scale-invariant log loss on depth uses projected depth from multiple datasets; noisy SfM depth could bias training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of 3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection." pith.science (2026). https://pith.science/paper/6PVCNI6Q

@misc{pith2026250723567,
  author       = {Pith},
  title        = {Pith review of: 3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6PVCNI6Q}},
  note         = {Machine review of arXiv:2507.23567}
}
read the original abstract

Monocular 3D object detection is valuable for various applications such as robotics and AR/VR. Existing methods are confined to closed-set settings, where the training and testing sets consist of the same scenes and/or object categories. However, real-world applications often introduce new environments and novel object categories, posing a challenge to these methods. In this paper, we address monocular 3D object detection in an open-set setting and introduce the first end-to-end 3D Monocular Open-set Object Detector (3D-MOOD). We propose to lift the open-set 2D detection into 3D space through our designed 3D bounding box head, enabling end-to-end joint training for both 2D and 3D tasks to yield better overall performance. We condition the object queries with geometry prior and overcome the generalization for 3D estimation across diverse scenes. To further improve performance, we design the canonical image space for more efficient cross-dataset training. We evaluate 3D-MOOD on both closed-set settings (Omni3D) and open-set settings (Omni3D to Argoverse 2, ScanNet), and achieve new state-of-the-art results. Code and models are available at royyang0714.github.io/3D-MOOD.

Figures

Figures reproduced from arXiv: 2507.23567 by the authors.

Figure 1
Figure 1. Open-set Monocular 3D Object Detection. Unlike previous methods focusing on achieving good results in the closed￾set setting, we aim to resolve the open-set monocular 3D object detection problem. This challenge requires the model to classify arbitrary objects while precisely localizing them in unseen scenes. from specialized models to more unified frameworks. How￾ever, as shown in [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figure 2
Figure 2. 3D-MOOD. We propose an end-to-end 3D monocular open-set object detector that takes a monocular image and the language prompts of the interested objects as input and classifies and localizes the 3D objects in the scenes. Our design will transform the input image and camera intrinsics into the proposed canonical image space and achieve the open-set ability for diverse scenes. open-set benchmarks, the performance is li… view at source ↗
Figure 3
Figure 3. Canonical Image Space. We compare the difference be￾tween different resizing and padding strategies. It is worth noting that the same image will have the same camera intrinsic K despite having very different image resolutions for previous methods. 3.4. Canonical Image Space To train the model across datasets that contain images with different resolutions from various datasets, previous works [4, 23, 27] either resiz… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Matching function. Different matching criteria over three methods on three different classes on AV2 and ScanNet. CD stands for matching prediction and GT using our proposed normal￾ized center distance matching, while IoU stands for using IoU3D. average orientation erro…
Figure 5
Figure 5. Figure 5: In-the-wild Qualitative Results. We show the visual￾ization of 3D-MOOD for in-the-wild images. The red boxes in the 3D visualization (last row) are the GT annotations. 5. Conclusion In this work, we introduce 3D-MOOD, the first end-to￾end 3D monocular open-set object d…
Figure 6
Figure 6. Figure 6: Comparison with Gemini 2. We qualitatively compare with Gemini 2 given the novel classes. We also propose the normalized true positive errors (TPE) to further analyze the matched prediction. First, we compute the 3D Euclidean distance between prediction and GT, and we …
Figure 7
Figure 7. Figure 7: Open-set Qualitative Results. We show more visualization on Argoverse 2 [54] and ScanNet [8]. 4 [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Closed-set Qualitative Results. We show the qualitative results for 3D-MOOD on Omni3D [4] test set. 5 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PLOT: Pseudo-Labeling via Object Tracking for Monocular 3D Object Detection

    cs.CV 2025-07 conditional novelty 6.0 of 10

    PLOT generates 3D bounding-box labels for objects from monocular video alone, using point tracking, estimated camera motion, and temporal fusion of pseudo-LiDAR, without auxiliary sensors or model training.

Reference graph

Works this paper leans on

67 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    Objectron: A large scale dataset of object-centric videos in the wild with pose anno- tations.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

    Adel Ahmadyan, Liangkai Zhang, Artsiom Ablavatski, Jian- ing Wei, and Matthias Grundmann. Objectron: A large scale dataset of object-centric videos in the wild with pose anno- tations.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021. 1, 2, 6

    work page 2021

  2. [2]

    ARK- itscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data

    Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, and Elad Shulman. ARK- itscenes - a diverse real-world dataset for 3d indoor scene understanding using mobile RGB-d data. InThirty-fifth Con- ference on Neural Information Processing Systems Datasets and Benchmarks Trac...

    work page 2021

  3. [3]

    Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024

    Aleksei Bochkovskii, Ama ¨el Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second.arXiv preprint arXiv:2410.02073, 2024. 2, 8

    arXiv 2024

  4. [4]

    Omni3d: A large benchmark and model for 3d object detection in the wild

    Garrick Brazil, Abhinav Kumar, Julian Straub, Nikhila Ravi, Justin Johnson, and Georgia Gkioxari. Omni3d: A large benchmark and model for 3d object detection in the wild. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 13154–13164, 2023. 1, 2, 4, 5, 6, 7

    work page 2023

  5. [5]

    Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom

    Holger Caesar, Varun Bankiti, Alex H. Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A mul- timodal dataset for autonomous driving.arXiv preprint arXiv:1903.11027, 2019. 1, 2, 5, 6

    arXiv 1903

  6. [6]

    End-to- end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to- end object detection with transformers. InEuropean confer- ence on computer vision, pages 213–229. Springer, 2020. 2, 3, 4

    work page 2020

  7. [7]

    Yolo-world: Real- time open-vocabulary object detection.arXiv preprint arXiv:2401.17270, 2024

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xinggang Wang, and Ying Shan. Yolo-world: Real- time open-vocabulary object detection.arXiv preprint arXiv:2401.17270, 2024. 2

    arXiv 2024

  8. [8]

    Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 2017. 2, 6, 7, 3, 4

    work page 2017

Show all 67 references
  1. [9]

    Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding.arXiv preprint arXiv:1810.04805, 2018. 2, 3

    2018 arXiv

  2. [10]

    Towards real-time monocular depth estimation for robotics: A survey.IEEE Transactions on Intelligent Transportation Systems, 23(10):16940–16961,

    Xingshuai Dong, Matthew A Garratt, Sreenatha G Ana- vatti, and Hussein A Abbass. Towards real-time monocular depth estimation for robotics: A survey.IEEE Transactions on Intelligent Transportation Systems, 23(10):16940–16961,

  3. [11]

    Depth map prediction from a single image using a multi-scale deep net- work.Advances in Neural Information Processing Systems (NeurIPS), 27, 2014

    David Eigen, Christian Puhrsch, and Rob Fergus. Depth map prediction from a single image using a multi-scale deep net- work.Advances in Neural Information Processing Systems (NeurIPS), 27, 2014. 5

    2014

  4. [12]

    Cc-3dt: Panoramic 3d object tracking via cross-camera fusion

    Tobias Fischer, Yung-Hsu Yang, Suryansh Kumar, Min Sun, and Fisher Yu. Cc-3dt: Panoramic 3d object tracking via cross-camera fusion. In6th Annual Conference on Robot Learning, 2022. 2, 4

    2022

  5. [13]

    Cross-domain few-shot object detection via enhanced open-set object detector

    Yuqian Fu, Yu Wang, Yixuan Pan, Lian Huai, Xingyu Qiu, Zeyu Shangguan, Tong Liu, Yanwei Fu, Luc Van Gool, and Xingqun Jiang. Cross-domain few-shot object detection via enhanced open-set object detector. InEuropean Conference on Computer Vision, pages 247–264. Springer, 2024. 2

    2024

  6. [14]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. InConference on Computer Vision and Pattern Recog- nition (CVPR), 2012. 1, 2, 5

    2012

  7. [15]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2

    2016

  8. [16]

    Sequential multi-object grasping with one dexterous hand.IROS, 2025

    Sicheng He, Zeyu Shangguan, Kuanning Wang, Yongchong Gu, Yuqian Fu, Yanwei Fu, and Daniel Seita. Sequential multi-object grasping with one dexterous hand.IROS, 2025. 1

    2025

  9. [17]

    Monocular quasi-dense 3d object tracking.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):1992–2008, 2022

    Hou-Ning Hu, Yung-Hsu Yang, Tobias Fischer, Trevor Dar- rell, Fisher Yu, and Min Sun. Monocular quasi-dense 3d object tracking.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):1992–2008, 2022. 2, 4

    1992

  10. [18]

    Training an open-vocabulary monocular 3d detection model without 3d data

    Rui Huang, Henry Zheng, Yan Wang, Zhuofan Xia, Marco Pavone, and Gao Huang. Training an open-vocabulary monocular 3d detection model without 3d data. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 2, 7

    2024

  11. [19]

    Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll ´ar, and Ross Girshick. Segment anything.arXiv:2304.02643, 2023. 2, 7

    2023 arXiv

  12. [20]

    3d-rcnn: Instance-level 3d object reconstruction via render-and- compare

    Abhijit Kundu, Yin Li, and James M Rehg. 3d-rcnn: Instance-level 3d object reconstruction via render-and- compare. InProceedings of the IEEE conference on com- puter vision and pattern recognition, pages 3559–3568,

  13. [21]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages...

    2022

  14. [22]

    Bevformer: 9 Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers.arXiv preprint arXiv:2203.17270, 2022

    Zhiqi Li, Wenhai Wang, Hongyang Li, Enze Xie, Chong- hao Sima, Tong Lu, Yu Qiao, and Jifeng Dai. Bevformer: 9 Learning bird’s-eye-view representation from multi-camera images via spatiotemporal transformers.arXiv preprint arXiv:2203.17270, 2022. 1, 2

    2022 arXiv

  15. [23]

    Unimode: Unified monocular 3d object detection

    Zhuoling Li, Xiaogang Xu, SerNam Lim, and Hengshuang Zhao. Unimode: Unified monocular 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 16561–16570,

  16. [24]

    Feature pyra- mid networks for object detection

    Tsung-Yi Lin, Piotr Doll ´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyra- mid networks for object detection. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2117–2125, 2017. 5

    2017

  17. [25]

    Sparse4d: Multi-view 3d object detec- tion with sparse spatial-temporal fusion.arXiv preprint arXiv:2211.10581, 2022

    Xuewu Lin, Tianwei Lin, Zixiang Pei, Lichao Huang, and Zhizhong Su. Sparse4d: Multi-view 3d object detec- tion with sparse spatial-temporal fusion.arXiv preprint arXiv:2211.10581, 2022. 2

    2022 arXiv

  18. [26]

    Sparse4d v3: Advancing end-to-end 3d detec- tion and tracking.arXiv preprint arXiv:2311.11722, 2023

    Xuewu Lin, Zixiang Pei, Tianwei Lin, Lichao Huang, and Zhizhong Su. Sparse4d v3: Advancing end-to-end 3d detec- tion and tracking.arXiv preprint arXiv:2311.11722, 2023. 4

    2023 arXiv

  19. [27]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection.arXiv preprint arXiv:2303.05499, 2023. 2, 3, 4, 7, 1

    2023 arXiv

  20. [28]

    Petr: Position embedding transformation for multi-view 3d object detection.arXiv preprint arXiv:2203.05625, 2022

    Yingfei Liu, Tiancai Wang, Xiangyu Zhang, and Jian Sun. Petr: Position embedding transformation for multi-view 3d object detection.arXiv preprint arXiv:2203.05625, 2022. 1, 2

    2022 arXiv

  21. [29]

    Smoke: Single- stage monocular 3d object detection via keypoint estimation

    Zechen Liu, Zizhang Wu, and Roland T ´oth. Smoke: Single- stage monocular 3d object detection via keypoint estimation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 996–997,

  22. [30]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 3, 5, 7, 1

    2021

  23. [31]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 11976–11986,

  24. [32]

    Shift r-cnn: Deep monocular 3d object detection with closed-form geometric constraints

    Andretti Naiden, Vlad Paunescu, Gyeongmo Kim, Byeong- Moon Jeon, and Marius Leordeanu. Shift r-cnn: Deep monocular 3d object detection with closed-form geometric constraints. In2019 IEEE international conference on im- age processing (ICIP), pages 61–65. IEEE, 2019. 2

    2019

  25. [33]

    Scalable parallel programming with cuda: Is cuda the parallel programming model that application developers have been waiting for?Queue, 6(2):40–53, 2008

    John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. Scalable parallel programming with cuda: Is cuda the parallel programming model that application developers have been waiting for?Queue, 6(2):40–53, 2008. 5

    2008

  26. [34]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 3

    2023 arXiv

  27. [35]

    Locate anything on earth: Advancing open-vocabulary ob- ject detection for remote sensing community.arXiv preprint arXiv:2408.09110, 2024

    Jiancheng Pan, Yanxing Liu, Yuqian Fu, Muyuan Ma, Jiahao Li, Danda Pani Paudel, Luc Van Gool, and Xiaomeng Huang. Locate anything on earth: Advancing open-vocabulary ob- ject detection for remote sensing community.arXiv preprint arXiv:2408.09110, 2024. 2

    2024 arXiv

  28. [36]

    Is pseudo-lidar needed for monocular 3d object detection? InIEEE/CVF International Conference on Computer Vision (ICCV), 2021

    Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon. Is pseudo-lidar needed for monocular 3d object detection? InIEEE/CVF International Conference on Computer Vision (ICCV), 2021. 1, 2, 4

    2021

  29. [37]

    Py- torch: An imperative style, high-performance deep learning library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zem- ing Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai- son, Andreas Kopf, Edward Yang, Zachary DeVito, Mar- tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steine...

  30. [38]

    UniDepth: Universal monocular metric depth estimation

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. UniDepth: Universal monocular metric depth estimation. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 5, 7, 8

    2024

  31. [39]

    UniK3D: Universal camera monocular 3d estimation

    Luigi Piccinelli, Christos Sakaridis, Mattia Segu, Yung- Hsu Yang, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniK3D: Universal camera monocular 3d estimation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), 2025. 5

    2025

  32. [40]

    UniDepthV2: Universal monocular metric depth estimation made simpler.arXiv:2502.20110, 2025

    Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mat- tia Segu, Siyuan Li, Wim Abbeloos, and Luc Van Gool. UniDepthV2: Universal monocular metric depth estimation made simpler.arXiv:2502.20110, 2025. 2, 5

    2025 arXiv

  33. [41]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  34. [42]

    Generalized in- tersection over union: A metric and a loss for bounding box regression

    Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized in- tersection over union: A metric and a loss for bounding box regression. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 658–666,

  35. [43]

    Susskind

    Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic syn- thetic dataset for holistic indoor scene understanding. In International Conference on Computer Vision (ICCV) 2021,

    2021

  36. [44]

    Imvoxelnet: Image to voxels projection for monocular and multi-view general-purpose 3d object detection

    Danila Rukhovich, Anna V orontsova, and Anton Konushin. Imvoxelnet: Image to voxels projection for monocular and multi-view general-purpose 3d object detection. InProceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2397–2406, 2022. 1, 7 10

    2022

  37. [45]

    Structure-from-motion revisited

    Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. InConference on Com- puter Vision and Pattern Recognition (CVPR), 2016. 5

    2016

  38. [46]

    Pixelwise view selection for un- structured multi-view stereo

    Johannes Lutz Sch ¨onberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for un- structured multi-view stereo. InEuropean Conference on Computer Vision (ECCV), 2016. 5

    2016

  39. [47]

    Sun rgb-d: A rgb-d scene understanding benchmark suite

    Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 567–576, 2015. 1, 2, 5

    2015

  40. [48]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023. 3

    2023 arXiv

  41. [49]

    Im- geonet: Image-induced geometry-aware voxel representation for multi-view 3d object detection

    Tao Tu, Shun-Po Chuang, Yu-Lun Liu, Cheng Sun, Ke Zhang, Donna Roy, Cheng-Hao Kuo, and Min Sun. Im- geonet: Image-induced geometry-aware voxel representation for multi-view 3d object detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 69...

    2023

  42. [50]

    Attention is all you need.Advances in Neural Information Processing Systems, 2017

    A Vaswani. Attention is all you need.Advances in Neural Information Processing Systems, 2017. 3, 5

    2017

  43. [51]

    Fcos3d: Fully convolutional one-stage monocular 3d object detection

    Tai Wang, Xinge Zhu, Jiangmiao Pang, and Dahua Lin. Fcos3d: Fully convolutional one-stage monocular 3d object detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 913–922, 2021. 1, 7

    2021

  44. [52]

    Probabilistic and geometric depth: Detecting objects in per- spective

    Tai Wang, ZHU Xinge, Jiangmiao Pang, and Dahua Lin. Probabilistic and geometric depth: Detecting objects in per- spective. InConference on Robot Learning, pages 1475–

  45. [53]

    Pseudo- lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving

    Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hari- haran, Mark Campbell, and Kilian Q Weinberger. Pseudo- lidar from visual depth estimation: Bridging the gap in 3d object detection for autonomous driving. InProceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2019

  46. [54]

    Argoverse 2: Next generation datasets for self-driving perception and fore- casting

    Benjamin Wilson, William Qi, Tanmay Agarwal, John Lam- bert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Rat- nesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays. Argoverse 2: Next generation datasets for self-driving perception an...

    2021

  47. [55]

    Qiao, Lewei Lu, Jie Zhou, and Jifeng Dai

    Chenyu Yang, Yuntao Chen, Haofei Tian, Chenxin Tao, Xizhou Zhu, Zhaoxiang Zhang, Gao Huang, Hongyang Li, Y . Qiao, Lewei Lu, Jie Zhou, and Jifeng Dai. Bevformer v2: Adapting modern image backbones to bird’s-eye-view recognition via perspective supervision.ArXiv, 2022. 1, 2, 4

    2022

  48. [56]

    Huang, Ren ´e Zurbr¨ugg, Tao Sun, and Fisher Yu

    Yung-Hsu Yang, Tobias Fischer, Thomas E. Huang, Ren ´e Zurbr¨ugg, Tao Sun, and Fisher Yu. Vis4D.https:// github.com/SysCV/vis4d, 2024. 5

    2024

  49. [57]

    Center- based 3d object detection and tracking

    Tianwei Yin, Xingyi Zhou, and Philipp Krahenbuhl. Center- based 3d object detection and tracking. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11784–11793, 2021. 1

    2021

  50. [58]

    Metric3d: Towards zero-shot metric 3d prediction from a single image

    Wei Yin, Chi Zhang, Hao Chen, Zhipeng Cai, Gang Yu, Kaixuan Wang, Xiaozhi Chen, and Chunhua Shen. Metric3d: Towards zero-shot metric 3d prediction from a single image. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 9043–9053, 2023. 2, 4, 5, 8

    2023

  51. [59]

    Deep layer aggregation

    Fisher Yu, Dequan Wang, Evan Shelhamer, and Trevor Darrell. Deep layer aggregation. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2403–2412, 2018. 3

    2018

  52. [60]

    Open-vocabulary detr with conditional matching

    Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Open-vocabulary detr with conditional matching. InEuropean Conference on Computer Vision, pages 106–122. Springer, 2022. 2

    2022

  53. [61]

    Open-vocabulary object detection using captions

    Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih- Fu Chang. Open-vocabulary object detection using captions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14393–14402, 2021. 2, 3

    2021

  54. [62]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection.arXiv preprint arXiv:2203.03605, 2022

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection.arXiv preprint arXiv:2203.03605, 2022. 2, 3

    2022 arXiv

  55. [63]

    Monodetr: Depth- guided transformer for monocular 3d object detection.ICCV 2023, 2022

    Renrui Zhang, Han Qiu, Tai Wang, Xuanzhuo Xu, Ziyu Guo, Yu Qiao, Peng Gao, and Hongsheng Li. Monodetr: Depth- guided transformer for monocular 3d object detection.ICCV 2023, 2022. 1

    2023

  56. [64]

    An open and com- prehensive pipeline for unified object grounding and detec- tion.arXiv preprint arXiv:2401.02361, 2024

    Xiangyu Zhao, Yicheng Chen, Shilin Xu, Xiangtai Li, Xin- jiang Wang, Yining Li, and Haian Huang. An open and com- prehensive pipeline for unified object grounding and detec- tion.arXiv preprint arXiv:2401.02361, 2024. 2, 5

    2024 arXiv

  57. [65]

    Does computer vision matter for action?Science Robotics, 4,

    Brady Zhou, Philipp Kr ¨ahenb¨uhl, and Vladlen Koltun. Does computer vision matter for action?Science Robotics, 4,

  58. [66]

    Detecting twenty-thousand classes using image-level supervision

    Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Kr¨ahenb¨uhl, and Ishan Misra. Detecting twenty-thousand classes using image-level supervision. InECCV, 2022. 2

    2022

  59. [67]

    Deformable detr: Deformable trans- formers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable trans- formers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020. 3 11 3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection Supplementary Materi...

    2010 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.