REVIEW 3 major objections 4 minor 35 references
ToosiCubix: Monocular 3D Cuboid Labeling via Vehicle Part Annotations
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read About ten user clicks on a calibrated car photo can recover the vehicle's 3D cuboid up to a global scale.
desk verdict Core 8DoF cuboid annotation from monocular part clicks is a real and useful contribution, but the abstract overclaims 9DoF quality and the wheel-flush assumption needs a direct test. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the linear-auxiliary cuboid parameterization: every annotation is a 3D point $X_i = A_i p$ written as an affine function of a vector $p$ that contains the dimensions $d_x, d_y, d_z$, the front and rear wheel x-coordinates, and per-constraint heights or offsets. Substituting $X_i = A_i p$ into the projection equation (1) yields the system (8) $\lambda_i u_i = R A_i p + t$, whose cost (9)–(11) is minimized by alternating SQPnP for $(R,t)$ and least squares for $(p,t)$, then refined by pixel-domain Levenberg-Marquardt. Gaussian size priors with robust median-based moments enter as the regularizer $\lambda (d-\mu_c)^T \Sigma_c^{-1}(d-\mu_c)$, resolving scale and missing-dimension ambiguities, and the yaw angle is initialized from line correspondences through the constraint $l^T K R D = 0$.
What would settle it
Reconstruct cuboids for vehicles whose true body width and track width are measured (for example, from CAD models or LiDAR), using only wheel-based annotations, and test whether estimated width error grows with the overhang of the body beyond the wheels.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that vehicle cuboids can be recovered from a single view by modeling each annotation type as a point whose 3D position is an affine function of the cuboid dimensions and a few auxiliary variables (wheel front and rear x-coordinates, heights along edges, lateral offsets of symmetric pairs). Substituted into the projection equation $\lambda_i u_i = R X_i + t$, these constraints form a joint system that is linear in the shape variables $p$ given a rotation, and an instance of PnP given $p$, so the two stages can be alternated until convergence. Direction annotations are handled by reparameterizing the arrow as the vehicle length $d_x$ rather than a free vector, which avoids degeneracies when the annotation plane is nearly parallel to the ground. The paper's central assertion is that this pipeline achieves accurate 8 DoF cuboids (mean rotation error about 3 degrees, scaled IoU 0.77–0.82 on KITTI and Cityscapes3D respectively) fast enough for practical annotation, and that full 9 DoF accuracy additionally requires geometric scale cues beyond soft size priors.
Load-bearing premise
The results assume the outer edge of the wheel where it touches the ground lies exactly on the vertical side face of the vehicle's cuboid, which biases width and lateral position for vehicles with fenders, mirrors, or bodies wider than their track.
Editorial extensions
If this is right
- Existing monocular datasets captured without LiDAR or stereo can be upgraded with 3D cuboid ground truth at an annotation cost of roughly 10 clicks and about 70 seconds per vehicle.
- 8 DoF cuboids—rotation, scaled position, and scaled dimensions—are reliable enough for downstream tasks such as orientation estimation and relative distance, since rotation error stays near 3 degrees and scaled IoU reaches 0.77–0.82.
- Full 9 DoF placement needs more than size priors; accurate cuboid annotation should combine the geometric solver with explicit scale cues like license-plate width, tire diameter, or ground-plane measurements.
- Pairing the geometric pipeline with a 2D keypoint detector yields an interpretable, geometry-based monocular 3D detector that relies less on end-to-end learning.
- Robust Gaussian priors learned per vehicle category can disambiguate unobserved dimensions (for example, length from a rear view) without assigning fixed prototypes.
Reading between the lines
- Since the wheel-flush assumption ($Y_i=\pm d_y/2$) is the main link between wheel clicks and cuboid width, vehicles with bodies wider than their track (fenders, running boards, mirror housings) will get systematically biased width and lateral position; adding an estimated side-offset parameter or a second width annotation would test and fix this.
- The reported per-vehicle time of roughly 70 seconds is dominated by choosing and labelling features; a semi-automatic keypoint suggestion system, which the paper only sketches as future work, could plausibly cut the cost to a few seconds of verification.
- The priors are fitted on Cityscapes3D prototype classes; applying the method to a new dataset would likely need priors re-estimated from that dataset or from CAD size databases, since vehicle size distributions differ across regions and eras.
- If the 8 DoF results transfer to automatic keypoint detection, the same formulation could serve as a training signal for monocular 3D detection with only image-level keypoint supervision; this is an implication the paper raises but does not develop.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ToosiCubix, a monocular 3D cuboid annotation method that takes about 10 2D user clicks on vehicle parts (wheels, centerline points, vertical edges, corners, left-right symmetric pairs, and directional arrows) and estimates the vehicle cuboid. The problem is formulated as a system of projection equations in which the 3D points are linear functions of the cuboid dimensions and auxiliary variables, and the authors solve it by coordinate descent alternating between SQPnP and least-squares subproblems, followed by a pixel-domain fine-tuning step with a probabilistic size prior. Evaluation on subsets of KITTI and Cityscapes3D reports strong 8DoF results (rotation error around 3 degrees, scaled IoU 0.77–0.82) but comparatively weak 9DoF results (IoU 0.29–0.32), which the paper itself acknowledges are 'not sufficient for accurate cuboid annotation'.
Significance. The paper presents a clean and flexible geometric formulation that extends PnP to settings where 3D landmarks are not fixed but are affine in the cuboid dimensions and auxiliary variables. The coordinate descent with a globally optimal PnP solver and probabilistic size priors is sensible, and the honest discussion of the 9DoF limitation is a strength. If the 8DoF claim survives scrutiny, the method is a practical low-cost tool for up-to-scale cuboid annotation, with potential use in weakly supervised training and dataset augmentation. The main risks are the geometric assumption about wheel placement and the overstatement of the abstract relative to the actual 9DoF performance.
major comments (3)
- [§3.1.1 (Wheels)] The assumption that the outer ground-contact point of a wheel lies exactly on the cuboid side plane, Y_i = ±d_y/2, is load-bearing for the estimation of vehicle width d_y and lateral position t_y. For most vehicles the body and fenders are wider than the wheel track, and the paper explicitly ignores only side mirrors, not body overhang. If the true contact point is inset by ε, every wheel constraint is systematically biased and the least-squares solution will underestimate d_y and shift t_y; the sIoU metric, which only rescales the solution, cannot correct this aspect-ratio error. The authors should validate this assumption against ground-truth contact-point offsets (e.g., by comparing the estimated d_y from wheel constraints with the GT cuboid width) or introduce a per-class correction term. Without such a validation, the reported sIoU and dimension errors may be optimistic.
- [Abstract and §4.4] The abstract claims 'high-quality 3D cuboid annotation' and accurate estimation of position, orientation, and dimensions, but §4.4 explicitly states that the 9DoF results (IoU 0.29/0.32, E_d 0.14/0.04) are 'not sufficient for accurate cuboid annotation' and that 'size priors alone do not fully resolve the scale ambiguity.' Since an annotation tool's output is a cuboid with absolute scale, the 8DoF up-to-scale results do not support the abstract's strong claim. The abstract and introduction should be reworded to clearly describe the contribution as 8DoF up-to-scale cuboid estimation, with 9DoF placement requiring additional geometric cues.
- [§4.1 (sIoU)] The sIoU metric is computed with a scale factor s* = ||t_gt||/||t||, which removes the global scale error by construction. The reported sIoU values (0.77–0.82) therefore do not reflect the absolute scale error, which is substantial as seen in the 9DoF metrics (IoU 0.29–0.32, E_t 0.06–0.10). The paper should report the distribution of s* and the scale error separately, and clarify in the discussion that a high sIoU does not imply accurate absolute dimensions. This is important for interpreting the central 8DoF claim.
minor comments (4)
- [Throughout] There are several typos and formatting issues, including 'Symmetris' in §3.2, garbled notation in Eq. (9) where the norm signs are missing, and inconsistent use of superscripts/subscripts for auxiliary variables (e.g., X_back^i and X_front^i in §3.3).
- [§3.3] The description of the Forward direction annotation is written somewhat confusingly: the notation X_back^i is used both as a 3D point and as an auxiliary variable, and the reader has to infer that X_i is reused for a pair. Please rename the pair variables for clarity.
- [§4.3] The paper reports annotation time but not the actual number of feature points used per vehicle, despite the abstract's claim of 'about 10 user clicks'. Reporting this number would help readers assess the practicality.
- [§4.4] The evaluation is based on only 145 and 135 vehicles, and the paper does not report per-class results or confidence intervals. Given the variance in vehicle size and pose, the 3° rotation error and sIoU differences between datasets may not be statistically robust.
Circularity Check
No circularity found: cuboid parameters are solved from projection equations and checked against external ground truth; the fitted priors are standard training-set statistics, not relabeled predictions.
full rationale
The derivation chain is self-contained. Section 3.1-3.3 converts 2D annotations into 3D vehicle-coordinate points whose coordinates are linear in the cuboid dimensions and auxiliary variables; equation (7), λ_i u_i = R X_i + t, is the standard projection equation, and equations (8)-(11) are direct algebraic rewrites. The optimization alternates between a PnP stage and a least-squares stage, with no output quantity defined in terms of the fitted priors or the evaluation ground truth. The wheel-flush assumption Y_i = ±d_y/2 in Section 3.1.1 is a geometric modeling assumption that can bias width and lateral position if violated, but it is not circular: it is an input hypothesis, not a restatement of the estimated cuboid. Similarly, the d_x used in the direction constraints in Section 3.3 is a parameterization of the vehicle length, not a pre-supplied answer. The size priors μ_c and Σ_c are computed from Cityscapes3D training data, and λ and λ_p are tuned on 57 training vehicles, while the reported evaluation is explicitly on the Cityscapes3D validation set and on KITTI (Section 4.4); this is standard train/validation separation rather than a fitted input being called a prediction. The paper even states that size priors alone do not fully resolve the monocular scale ambiguity, further confirming that the 9-DoF results are not forced by the priors. All cited support, including Cityscapes3D [20], SQPnP [29], and the direction-annotation prior work [30], is external; there is no load-bearing self-citation chain or imported uniqueness theorem. No step meets the required bar of reducing, by the paper's own equations or citations, to its own inputs, so the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- lambda (prior weight in Eq. 17) =
not reported
- lambda_p (prior weight in Eq. 18) =
not reported
- per-class size prior means mu_c and covariances Sigma_c =
computed from Cityscapes3D training set via geometric median
assumptions (4)
- domain assumption Camera intrinsics K are known and lens distortion is corrected
- domain assumption Vehicle is represented as a 3D cuboid with wheel ground-contact points on the side faces (Yi = +/- dy/2)
- domain assumption Initial rotation has zero pitch and roll relative to the camera
- ad hoc to paper Vehicle dimensions per prototype class follow a Gaussian distribution (robustly estimated)
Cite this review
Pith. "Pith review of ToosiCubix: Monocular 3D Cuboid Labeling via Vehicle Part Annotations." pith.science (2026). https://pith.science/paper/PVHW4EJU
@misc{pith2026250621358,
author = {Pith},
title = {Pith review of: ToosiCubix: Monocular 3D Cuboid Labeling via Vehicle Part Annotations},
year = {2026},
howpublished = {\url{https://pith.science/paper/PVHW4EJU}},
note = {Machine review of arXiv:2506.21358}
}
read the original abstract
Many existing methods for 3D cuboid annotation of vehicles rely on expensive and carefully calibrated camera-LiDAR or stereo setups, limiting their accessibility for large-scale data collection. We introduce ToosiCubix, a simple yet powerful approach for annotating ground-truth cuboids using only monocular images and intrinsic camera parameters. Our method requires only about 10 user clicks per vehicle, making it highly practical for adding 3D annotations to existing datasets originally collected without specialized equipment. By annotating specific features (e.g., wheels, car badge, symmetries) across different vehicle parts, we accurately estimate each vehicle's position, orientation, and dimensions up to a scale ambiguity (8 DoF). The geometric constraints are formulated as an optimization problem, which we solve using a coordinate descent strategy, alternating between Perspective-n-Points (PnP) and least-squares subproblems. To handle common ambiguities such as scale and unobserved dimensions, we incorporate probabilistic size priors, enabling 9 DoF cuboid placements. We validate our annotations against the KITTI and Cityscapes3D datasets, demonstrating that our method offers a cost-effective and scalable solution for high-quality 3D cuboid annotation.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
G. Ros, L. Sellart, J. Materzynska, D. Vazquez, and A. M. Lopez, “The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3234– 3243
work page 2016
-
[2]
Playing for data: Ground truth from computer games,
S. R. Richter, V . Vineet, S. Roth, and V . Koltun, “Playing for data: Ground truth from computer games,” inComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. Springer, 2016, pp. 102–118
work page 2016
-
[3]
Synscapes: A photorealistic synthetic dataset for street scene parsing,
M. Wrenninge and J. Unger, “Synscapes: A photorealistic synthetic dataset for street scene parsing,”arXiv preprint arXiv:1810.08705, 2018
arXiv 2018
-
[4]
Y. Cabon, N. Murray, and M. Humenberger, “Virtual kitti 2,”arXiv preprint arXiv:2001.10773, 2020
arXiv 2001
-
[5]
Skope3d: A synthetic dataset for vehicle keypoint perception in 3d from traffic monitoring cameras,
H. Pahadia, D. Lu, B. Chakravarthi, and Y. Yang, “Skope3d: A synthetic dataset for vehicle keypoint perception in 3d from traffic monitoring cameras,” in2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2023, pp. 4367– 4372
work page 2023
-
[6]
Unisim: A neural closed-loop sensor simulator,
Z. Yang, Y. Chen, J. Wang, S. Manivasagam, W.-C. Ma, A. J. Yang, and R. Urtasun, “Unisim: A neural closed-loop sensor simulator,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 1389–1399
work page 2023
-
[7]
Neuroncap: Photorealistic closed- loop safety testing for autonomous driving,
W. Ljungbergh, A. Tonderski, J. Johnander, H. Caesar, K. Astrom, M. Felsberg, and C. Petersson, “Neuroncap: Photorealistic closed- loop safety testing for autonomous driving,” inEuropean Confer- ence on Computer Vision. Springer, 2024, pp. 161–177
work page 2024
-
[8]
3drealcar: An in-the-wild rgb-d car dataset with 360-degree views,
X. Du, H. Sun, S. Wang, Z. Wu, H. Sheng, J. Ying, M. Lu, T. Zhu, K. Zhan, and X. Yu, “3drealcar: An in-the-wild rgb-d car dataset with 360-degree views,”arXiv preprint arXiv:2406.04875, 2024
arXiv 2024
Show all 35 references
-
[9]
Unraveling the effects of synthetic data on end-to-end autonomous driving,
J. Ge, Z. Liu, L. Fan, Y. Jiang, J. Su, Y. Li, Z. Zhang, and S. Chen, “Unraveling the effects of synthetic data on end-to-end autonomous driving,”arXiv preprint arXiv:2503.18108, 2025
2025 arXiv
-
[10]
Are we ready for autonomous driving? the kitti vision benchmark suite,
A. Geiger, P . Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in2012 IEEE conference on computer vision and pattern recognition. IEEE, 2012, pp. 3354– 3361
2012
-
[11]
The apolloscape dataset for autonomous driving,
X. Huang, X. Cheng, Q. Geng, B. Cao, D. Zhou, P . Wang, Y. Lin, and R. Yang, “The apolloscape dataset for autonomous driving,” inProceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, pp. 954–960
2018
-
[12]
Argoverse: 3d tracking and forecasting with rich maps,
M.-F. Chang, J. Lambert, P . Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P . Carr, S. Lucey, D. Ramananet al., “Argoverse: 3d tracking and forecasting with rich maps,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 8748–8757
2019
-
[13]
Lyft level 5 av dataset 2019,
R. Kesten, M. Usman, J. Houston, T. Pandya, K. Nadhamuni, A. Ferreira, M. Yuan, B. Low, A. Jain, P . Ondruskaet al., “Lyft level 5 av dataset 2019,”urlhttps://level5. lyft. com/dataset, vol. 1, p. 3, 2019
2019
-
[14]
Waymo open dataset: An autonomous driving dataset,
W. LLC, “Waymo open dataset: An autonomous driving dataset,” 2019
2019
-
[15]
The h3d dataset for full-surround 3d multi-object detection and tracking in crowded urban scenes,
A. Patil, S. Malla, H. Gang, and Y.-T. Chen, “The h3d dataset for full-surround 3d multi-object detection and tracking in crowded urban scenes,” in2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019, pp. 9552–9557
2019
-
[16]
One million scenes for autonomous driving: Once dataset,
J. Mao, M. Niu, C. Jiang, H. Liang, J. Chen, X. Liang, Y. Li, C. Ye, W. Zhang, Z. Liet al., “One million scenes for autonomous driving: Once dataset,”arXiv preprint arXiv:2106.11037, 2021
2021 arXiv
-
[17]
Rope3d: The roadside perception dataset for autonomous driving and monocular 3d object detection task,
X. Ye, M. Shu, H. Li, Y. Shi, Y. Li, G. Wang, X. Tan, and E. Ding, “Rope3d: The roadside perception dataset for autonomous driving and monocular 3d object detection task,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 21 341–21 350
2022
-
[18]
A* 3d dataset: Towards autonomous driving in challenging environments,
Q.-H. Pham, P . Sevestre, R. S. Pahwa, H. Zhan, C. H. Pang, Y. Chen, A. Mustafa, V . Chandrasekhar, and J. Lin, “A* 3d dataset: Towards autonomous driving in challenging environments,” in2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 2267–2273
2020
-
[19]
Zenseact open dataset: A large-scale and diverse multimodal dataset for autonomous driving,
M. Alibeigi, W. Ljungbergh, A. Tonderski, G. Hess, A. Lilja, C. Lindstrom, D. Motorniuk, J. Fu, J. Widahl, and C. Petersson, “Zenseact open dataset: A large-scale and diverse multimodal dataset for autonomous driving,” inProceedings of the IEEE/CVF International Conference on ...
2023
-
[20]
Cityscapes 3d: Dataset and benchmark for 9 dof vehicle detec- tion,
N. G ¨ahlert, N. Jourdan, M. Cordts, U. Franke, and J. Denzler, “Cityscapes 3d: Dataset and benchmark for 9 dof vehicle detec- tion,”arXiv preprint arXiv:2006.07864, 2020
2006 arXiv
-
[21]
Boxy vehicle detection in large images,
K. Behrendt, “Boxy vehicle detection in large images,” inPro- ceedings of the IEEE International Conference on Computer Vision Workshops, 2019
2019
-
[22]
3d pose estimation for fine-grained object categories,
Y. Wang, X. Tan, Y. Yang, X. Liu, E. Ding, F. Zhou, and L. S. Davis, “3d pose estimation for fine-grained object categories,” in Proceedings of the European Conference on Computer Vision (ECCV) Workshops, 2018, pp. 0–0
2018
-
[23]
Beyond pascal: A bench- mark for 3d object detection in the wild,
Y. Xiang, R. Mottaghi, and S. Savarese, “Beyond pascal: A bench- mark for 3d object detection in the wild,” inIEEE winter conference on applications of computer vision. IEEE, 2014, pp. 75–82
2014
-
[24]
Apollocar3d: A large 3d car instance un- derstanding benchmark for autonomous driving,
X. Song, P . Wang, D. Zhou, R. Zhu, C. Guan, Y. Dai, H. Su, H. Li, and R. Yang, “Apollocar3d: A large 3d car instance un- derstanding benchmark for autonomous driving,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 5452–5462
2019
-
[25]
A morphable model for the synthesis of 3d faces,
V . Blanz and T. Vetter, “A morphable model for the synthesis of 3d faces,” inProceedings of the 26th annual conference on Computer graphics and interactive techniques. ACM Press, 1999, pp. 187–194
1999
-
[26]
Jointly optimiz- ing 3d model fitting and fine-grained classification,
Y.-L. Lin, V . I. Morariu, W. Hsu, and L. S. Davis, “Jointly optimiz- ing 3d model fitting and fine-grained classification,” inComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part IV 13. Springer, 2014, pp. 466–480
2014
-
[27]
Mono3d++: Monocular 3d vehicle detection with two-scale 3d hypotheses and task priors,
T. He and S. Soatto, “Mono3d++: Monocular 3d vehicle detection with two-scale 3d hypotheses and task priors,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 8409–8416
2019
-
[28]
Optimal and robust category-level perception: Object pose and shape estimation from 2-d and 3-d semantic keypoints,
J. Shi, H. Yang, and L. Carlone, “Optimal and robust category-level perception: Object pose and shape estimation from 2-d and 3-d semantic keypoints,”IEEE Transactions on Robotics, vol. 39, no. 5, pp. 4131–4151, 2023
2023
-
[29]
A consistently fast and globally optimal solution to the perspective-n-point problem,
G. Terzakis and M. Lourakis, “A consistently fast and globally optimal solution to the perspective-n-point problem,” inEuropean Conference on Computer Vision. Springer, 2020, pp. 478–494
2020
-
[30]
Weakly supervised monocular 3d object detection using multi-view projection and direction consistency,
R. Tao, W. Han, Z. Qiu, C.-Z. Xu, and J. Shen, “Weakly supervised monocular 3d object detection using multi-view projection and direction consistency,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17 482–17 492
2023
-
[31]
Pose estimation from line correspondences: A complete analysis and a series of solutions,
C. Xu, L. Zhang, L. Cheng, and R. Koch, “Pose estimation from line correspondences: A complete analysis and a series of solutions,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 6, pp. 1209–1222, 2016
2016
-
[32]
The weber problem,
Z. Drezner, K. Klamroth, A. Sch ¨obel, and G. O. Wesolowsky, “The weber problem,”Facility location: Applications and theory, pp. 1–36, 2002
2002
-
[33]
Gauss-newton/levenberg-marquardt optimization,
E. Eade, “Gauss-newton/levenberg-marquardt optimization,” Tech. Rep., 2013
2013
-
[34]
Accelerating 3d deep learning with py- torch3d,
N. Ravi, J. Reizenstein, D. Novotny, T. Gordon, W.-Y. Lo, J. John- son, and G. Gkioxari, “Accelerating 3d deep learning with py- torch3d,”arXiv:2007.08501, 2020
2007 arXiv
-
[35]
Rotation averaging,
R. Hartley, J. Trumpf, Y. Dai, and H. Li, “Rotation averaging,” International journal of computer vision, vol. 103, pp. 267–305, 2013. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 14 Behrooz Nasihatkonreceived his Ph.D. degree from the Australian National Universi...
2013
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.