REVIEW 3 major objections 4 minor 31 references
MoCap2GT: A High-Precision Ground Truth Estimator for SLAM Benchmarking Based on Motion Capture and IMU Fusion
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read MoCap2GT claims that fusing marker-based motion capture with the device's own IMU produces ground-truth trajectories accurate enough to benchmark rotation and inter-frame SLAM errors, meeting target errors below 2 mm/0.2 deg and 0.4…
desk verdict Solid engineering paper with a plausible but unproven headline accuracy claim; the method is a sensible combination of known tools, worth reviewing, but the real-world validation is too thin to certify the 2 mm/0.2 deg targets. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a cumulative cubic B-spline pose parameterization on the $\mathrm{SE}(3)$ manifold with a variable time offset, embedded as MoCap factors in a factor-graph maximum-likelihood problem. The B-spline interpolates the discrete, jittery MoCap poses with $C^2$ continuity, so each IMU-state residual queries a smooth pose; the time offset is itself a linear B-spline whose control points are estimated states, absorbing clock-scale drift between the two devices. Convergence is prepared by a linear initializer that uses screw-theory weighting and RANSAC to recover extrinsic rotation, translation, velocity, and gravity alignment without good priors. A degeneracy detector computes the maximum rotation in sliding windows and suppresses gradients of the spatiotemporal calibration parameters during low-excitation motion, preventing ill-conditioned updates.
What would settle it
Collect a new sequence where a room-scale trajectory is measured simultaneously by the MoCap+IMU setup and by an independent reference with known sub-0.05 mm accuracy, such as a laser tracker; if MoCap2GT's ATE/ARE/RTE/RRE against that reference exceed the stated targets, the central claim is falsified.
Extended reading notes
Core claim
The paper's central claim is that a joint maximum-likelihood estimator over MoCap and IMU measurements can simultaneously solve spatiotemporal calibration and MoCap jitter mitigation, so the resulting ground truth is accurate enough to score rotation and inter-frame SLAM errors. The authors position this as the first consumer-grade setup to meet their stated accuracy targets, reporting ATE of 1.466 mm, ARE of 0.178 deg, RTE of 0.177 mm, and RRE of 0.013 deg in the sufficient-motion real-world scenario, with the estimator staying within the target bounds even under motion degradation. They further show that re-benchmarking a state-of-the-art visual-inertial SLAM system against MoCap2GT instead of official dataset ground truth changes reported relative rotation errors dramatically, which they take as evidence that official MoCap-based GT in public datasets is not accurate enough for rotation and inter-frame evaluation.
Load-bearing premise
The evaluation against real-world reference trajectories assumes that a global bundle adjustment with reprojection error below 0.1 pixels yields reference poses accurate to about 0.05 mm and 0.003 deg at 1 m viewing distance; that conversion is theoretical, and the small test volume may not represent the motions real SLAM benchmarks require.
Editorial extensions
If this is right
- If MoCap2GT's accuracy holds, SLAM benchmarks can report rotation and inter-frame errors (ARE/RTE/RRE) with the same confidence they now give absolute translation error (ATE).
- Existing public datasets that publish raw MoCap and IMU data can be re-processed to produce cleaner ground truth; the paper does this for two widely used datasets and shows that official ground truth with jitter inflates relative rotation error.
- The estimator's variable time-offset model keeps accuracy when the MoCap and IMU clocks drift, a regime in which fixed-offset baselines degrade sharply.
- A visual-inertial SLAM system re-benchmarked on MoCap2GT ground truth yields materially different error numbers (e.g., relative rotation error on one room sequence drops from 0.362 deg to 0.064 deg), implying comparisons published against noisy GT may need revisiting.
Reading between the lines
- The accuracy target is demonstrated only inside a short-range, calibration-board volume; whether the same error levels persist over meter-scale, high-speed motion typical of SLAM evaluation is not tested here and would be the natural extension.
- Because the method needs only raw MoCap and IMU streams, it could be applied retroactively to any existing dataset that stores both, giving an immediate upgrade path for benchmark ground truth without new data collection.
- The variable time-offset model suggests a general recipe for any sensor pair with unsynchronized free-running clocks: estimate the mapping between clocks as parameters of the trajectory representation rather than a single scalar.
- A reader wanting independent confirmation could replace the camera-based reference with a laser tracker or a second MoCap system; if the reference uncertainty is not at least an order of magnitude below the claimed errors, the headline numbers should be read as an upper bound.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents MoCap2GT, a batch maximum-likelihood estimator that fuses 6-DoF marker-based MoCap pose measurements with IMU readings from the device under test to produce a high-precision ground-truth trajectory for SLAM benchmarking. The method consists of a linear initialization stage that recovers extrinsic rotation, translation, gravity, velocities, and initial time offset using screw-theory robust kernels and RANSAC, followed by nonlinear optimization over a factor graph. The MoCap factor is modeled via cubic B-spline interpolation on SE(3) with a time offset that is itself represented as a linear B-spline with free control points, allowing for clock-drift compensation. A degeneracy-aware rejection strategy gates the MoCap factor's contribution to spatiotemporal calibration parameters in low-rotation windows. The authors claim that, using a consumer-grade IMU and MoCap system, MoCap2GT is the first method to satisfy the accuracy targets ATE/ARE < 2 mm/0.2 deg and RTE/RRE < 0.4 mm/0.02 deg. Validation includes simulated data with controlled noise and a real-world experiment against a monocular bundle-adjustment reference, plus benchmarking applications on TUM-VI and EuRoC.
Significance. If the central accuracy claim is upheld, the paper would make a useful contribution to SLAM benchmarking by enabling reliable evaluation of rotation and inter-frame errors, which existing MoCap-based ground-truth pipelines do not provide at the stated precision. The formulation is well grounded: it builds on standard IMU preintegration and continuous-time B-splines, and the variable time-offset model is a sensible response to clock-scale drift. The manuscript also provides an open-source implementation and uses concrete, falsifiable accuracy targets. The main weakness is at the validation layer: the real-world experiment is a single short-range trajectory with a reference that is not independently calibrated, and the simulation uses an ORB-SLAM3 trajectory as the 'true' motion. These gaps, together with empirically set thresholds, leave the headline claim not yet firmly established.
major comments (3)
- [Section V-B, Table III, Fig. 8] The claimed first demonstration of meeting the Section I-A accuracy targets rests on a single real-world trajectory compared against a bundle-adjustment reference whose accuracy is inferred from a 0.1 px reprojection error via a theoretical conversion (0.05 mm and 0.003 deg at a viewing distance of 1 m). This conversion does not account for correlated error sources in monocular global BA, such as biases in camera intrinsics, calibration-board flatness, and corner detection. The trajectory shown in Fig. 8 spans only a few centimeters, and only single trial values are reported in Table III. With ATE margin of about 0.53 mm and ARE margin of about 0.022 deg, an independent reference (for example, a laser tracker or a second, higher-fidelity MoCap system) or repeated trials at multiple ranges and motion types is needed before the absolute-accuracy claim is established.
- [Section V-A] The simulation uses an ORB-SLAM3 trajectory as the basis ground truth for generating synthetic IMU and MoCap measurements. Since an ORB-SLAM3 estimate has its own unknown trajectory error, the simulation cannot establish the absolute accuracy of MoCap2GT; it can only compare relative performance and noise robustness. To support the central claim that the method meets the specific error targets, the simulator should be driven by a trajectory with known ground truth, such as an analytically generated spline motion or a real trajectory measured by an independent high-accuracy device. The current simulation is therefore not sufficient evidence for the headline accuracy claim.
- [Section IV-B3, Eq. (10), Eq. (25)] Several key parameters are set empirically without a reported sensitivity analysis: the kernel amplification factor mu = 5 in Eq. (10), the degeneracy window size w = 5 s, and the rotation threshold theta = 10 deg in Eq. (25). Since the method's robustness to motion degeneracy is presented as a contribution, it is important to show how the four output error metrics vary with these parameters, or to provide a data-driven justification for the chosen values. Without this, the reader cannot tell whether the single successful real-world result in Table III depends on parameters tuned to that particular short-range motion.
minor comments (4)
- [Section V-B] The paper explicitly acknowledges that the real-world setup limits the testable motion range to keep the board in view, but the conclusion states without qualification that the method meets the accuracy targets. The limitation should be reflected in the conclusion, or the authors should add experiments that cover longer and more varied trajectories.
- [Fig. 5] The visualization of Jacobian directional errors would benefit from a colorbar and axis labels; in the current figure it is not clear how the 'directional error' is computed or what the color scale represents.
- [Table III] The typographic device of green highlighting for target-satisfying cells will not be visible in grayscale print; please use a pattern or an explicit marker, and also state the margins with respect to the targets in the caption.
- [Section IV-B2, Eq. (19)] The cubic B-spline interpolates the raw MoCap positions, which still contain measurement noise; the claim that it 'reduces MoCap jitter' depends on the smoothing behavior of the spline basis. It would be helpful to report the residual statistics between the fitted spline and the raw MoCap data to quantify how much smoothing is actually applied.
Circularity Check
No significant circularity: MoCap2GT's trajectory is produced by a standard MLE over independent MoCap and IMU factors and is validated against an external BA reference; the only self-citation is a non-load-bearing initializer.
full rationale
The derivation chain is self-contained rather than circular. The state vector in Eq. (1) and the objective in Eq. (16) are defined directly from MoCap pose residuals (18), IMU preintegration residuals (17), and bias random-walk terms; none of these quantities is defined in terms of ATE/ARE/RTE/RRE or in terms of the Section I-A thresholds, so the reported trajectory errors are not forced by construction. The real-world validation in Section V-B uses an external monocular global-bundle-adjustment reference trajectory, not the proposed estimator's own output, so the Table III comparison does not rename a fitted value as a prediction. The paper itself admits the physical-range limitation of that validation, and the conversion from 0.1 px reprojection error to 0.05 mm/0.003 deg pose error is a theoretical estimate rather than an independently verified reference; these are evidence-quality concerns, not circularity. The only overlapping-author citation, [22], is used for a coarse angular-velocity correlation time-alignment initializer and for a literature critique of existing datasets; it does not assume the target accuracy and is not the source of the central claim. No equation or fitted parameter in the paper reduces to the claimed accuracy target by definition, so the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- Kernel amplification factor mu =
5
- Degeneracy window size w =
5 s
- Rotation threshold theta =
10 deg
- Number of time alignment control points m
assumptions (5)
- domain assumption MoCap and IMU noise are independent, white, and Gaussian with known covariances.
- domain assumption The MoCap trajectory can be represented by a uniform cubic B-spline on SE(3) with C2 continuity.
- ad hoc to paper Time offset drift can be modeled as a linear B-spline with control points as free states.
- domain assumption Global BA with 0.1 px reprojection error yields a reference pose error of 0.05 mm and 0.003 deg at 1 m viewing distance.
- standard math Gravity alignment can be reduced to 2-DoF roll-pitch (yaw unobservable).
Cite this review
Pith. "Pith review of MoCap2GT: A High-Precision Ground Truth Estimator for SLAM Benchmarking Based on Motion Capture and IMU Fusion." pith.science (2026). https://pith.science/paper/4UOQANQ7
@misc{pith2026250712920,
author = {Pith},
title = {Pith review of: MoCap2GT: A High-Precision Ground Truth Estimator for SLAM Benchmarking Based on Motion Capture and IMU Fusion},
year = {2026},
howpublished = {\url{https://pith.science/paper/4UOQANQ7}},
note = {Machine review of arXiv:2507.12920}
}
read the original abstract
Marker-based optical motion capture (MoCap) systems are widely used to provide ground truth (GT) trajectories for benchmarking SLAM algorithms. However, the accuracy of MoCap-based GT trajectories is mainly affected by two factors: spatiotemporal calibration errors between the MoCap system and the device under test (DUT), and inherent MoCap jitter. Consequently, existing benchmarks focus primarily on absolute translation error, as accurate assessment of rotation and inter-frame errors remains challenging, hindering thorough SLAM evaluation. This paper proposes MoCap2GT, a joint optimization approach that integrates MoCap data and inertial measurement unit (IMU) measurements from the DUT for generating high-precision GT trajectories. MoCap2GT includes a robust state initializer to ensure global convergence, introduces a higher-order B-spline pose parameterization on the SE(3) manifold with variable time offset to effectively model MoCap factors, and employs a degeneracy-aware measurement rejection strategy to enhance estimation accuracy. Experimental results demonstrate that MoCap2GT outperforms existing methods and significantly contributes to precise SLAM benchmarking. The source code is available at https://anonymous.4open.science/r/mocap2gt (temporarily hosted anonymously for double-blind review).
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
DVI-SLAM: A dual visual inertial SLAM network,
X. Peng, Z. Liu, W. Li, P. Tan, S. Y . Cho, and Q. Wang, “DVI-SLAM: A dual visual inertial SLAM network,” in Proc. IEEE Int. Conf. Robot. Autom., 2024, pp. 12 020–12 026
work page 2024
-
[2]
SchurVINS: Schur complement-based lightweight visual inertial navigation system,
Y . Fan, T. Zhao, and G. Wang, “SchurVINS: Schur complement-based lightweight visual inertial navigation system,” in Proc. IEEE Comput. Vis. Pattern Recognit., 2024, pp. 17 964–17 973
work page 2024
-
[3]
ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM,
C. Campos, R. Elvira, J. J. G. Rodr ´ıguez, J. M. Montiel, and J. D. Tard´os, “ORB-SLAM3: An accurate open-source library for visual, visual-inertial, and multimap SLAM,” IEEE Trans. Robot. , vol. 37, no. 6, pp. 1874–1890, May 2021
work page 2021
-
[4]
A benchmark for the evaluation of RGB-D SLAM systems,
J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of RGB-D SLAM systems,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. , 2012, pp. 573–580
work page 2012
-
[5]
Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality,
J. Li, B. Yang, D. Chen, N. Wang, G. Zhang, and H. Bao, “Survey and evaluation of monocular visual-inertial SLAM algorithms for augmented reality,” Virtual Real. & Intell. Hardware , vol. 1, no. 4, pp. 386–410, August 2019
work page 2019
-
[6]
D2SLAM: Decentralized and distributed collaborative visual-inertial SLAM system for aerial swarm,
H. Xu, P. Liu, X. Chen, and S. Shen, “ D2SLAM: Decentralized and distributed collaborative visual-inertial SLAM system for aerial swarm,” IEEE Trans. Robot. , vol. 40, pp. 3445–3464, July 2024
work page 2024
-
[7]
S. Guo and G. Gallego, “CMax-SLAM: Event-based rotational-motion bundle adjustment and SLAM system using contrast maximization,” IEEE Trans. Robot. , vol. 40, pp. 2442–2461, March 2024
work page 2024
-
[8]
NeRF-VINS: A real-time neural radiance field map- based visual-inertial navigation system,
S. Katragadda, W. Lee, Y . Peng, P. Geneva, C. Chen, C. Guo, M. Li, and G. Huang, “NeRF-VINS: A real-time neural radiance field map- based visual-inertial navigation system,” in Proc. IEEE Int. Conf. Robot. Autom., 2024, pp. 10 230–10 237
work page 2024
Show all 31 references
-
[9]
The EuRoC micro aerial vehicle datasets,
M. Burri, J. Nikolic, P. Gohl, T. Schneider, J. Rehder, S. Omari, M. W. Achtelik, and R. Siegwart, “The EuRoC micro aerial vehicle datasets,” Int. J. Rob. Res. , vol. 35, no. 10, pp. 1157–1163, January 2016
2016
-
[10]
The TUM VI benchmark for evaluating visual-inertial odometry,
D. Schubert, T. Goll, N. Demmel, V . Usenko, J. St ¨uckler, and D. Cremers, “The TUM VI benchmark for evaluating visual-inertial odometry,” in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. , 2018, pp. 1680–1687
2018
-
[11]
The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM,
E. Mueggler, H. Rebecq, G. Gallego, T. Delbruck, and D. Scaramuzza, “The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM,” Int. J. Rob. Res. , vol. 36, no. 2, pp. 142–149, February 2017
2017
-
[12]
A general approach to hand-eye calibration through the optimization of atomic transfor- mations,
E. Pedrosa, M. Oliveira, N. Lau, and V . Santos, “A general approach to hand-eye calibration through the optimization of atomic transfor- mations,” IEEE Trans. Robot. , vol. 37, no. 5, pp. 1619–1633, March 2021
2021
-
[13]
Evaluation of combined time-offset estimation and hand- eye calibration on robotic datasets,
F. Furrer, M. Fehr, T. Novkovic, H. Sommer, I. Gilitschenski, and R. Siegwart, “Evaluation of combined time-offset estimation and hand- eye calibration on robotic datasets,” in Field Serv. Robot. , 2018, pp. 145–159
2018
-
[14]
Vicon2GT: Derivations and analysis,
P. Geneva and G. Huang, “Vicon2GT: Derivations and analysis,” Uni- versity of Delaware, Tech. Rep. RPNG-2020-VICON2GT , December 2020
2020
-
[15]
AirSLAM: An efficient and illumination-robust point-line visual SLAM system,
K. Xu, Y . Hao, S. Yuan, C. Wang, and L. Xie, “AirSLAM: An efficient and illumination-robust point-line visual SLAM system,” IEEE Trans. Robot., vol. 41, February 2025
2025
-
[16]
VECtor: A versatile event-centric benchmark for multi- sensor SLAM,
L. Gao, Y . Liang, J. Yang, S. Wu, C. Wang, J. Chen, and L. Kneip, “VECtor: A versatile event-centric benchmark for multi- sensor SLAM,” IEEE Robot. Autom. Lett. , vol. 7, no. 3, pp. 8217– 8224, June 2022
2022
-
[17]
SEV AR: A stereo event camera dataset for virtual and augmented reality,
Y . Dong, Z. Chen, X. He, L. Li, Z. Shu, Y . Cao, J. Feng, S. Liu, C. Li, and J. Wang, “SEV AR: A stereo event camera dataset for virtual and augmented reality,” Front. Inf. Technol. Electron. Eng., vol. 25, no. 5, pp. 755–762, June 2024
2024
-
[18]
Correspondence matching and time delay estimation for hand-eye calibration,
J. Wu, M. Liu, C. Zhang, and Z. Zhou, “Correspondence matching and time delay estimation for hand-eye calibration,” IEEE Trans. Instrum. Meas., vol. 69, no. 10, pp. 8304–8313, April 2020
2020
-
[19]
Simultaneous hand-eye/robot-world/camera-IMU calibration,
J. Wu, M. Wang, Y . Jiang, B. Yi, R. Fan, and M. Liu, “Simultaneous hand-eye/robot-world/camera-IMU calibration,” IEEE/ASME Trans. Mechatron., vol. 27, no. 4, pp. 2278–2289, August 2021
2021
-
[20]
Continuous-time batch estimation using temporal basis functions,
P. Furgale, T. D. Barfoot, and G. Sibley, “Continuous-time batch estimation using temporal basis functions,” in Proc. IEEE Int. Conf. Robot. Autom., 2012, pp. 2088–2095
2012
-
[21]
Multi-visual-inertial system: Analysis, calibration, and estimation,
Y . Yang, P. Geneva, and G. Huang, “Multi-visual-inertial system: Analysis, calibration, and estimation,” Int. J. Rob. Res., vol. 43, no. 13, pp. 1995–2026, 2024
1995
-
[22]
A spatiotemporal hand- eye calibration for trajectory alignment in visual (-inertial) odometry evaluation,
Z. Shu, L. Li, R. Wang, and Z. Chen, “A spatiotemporal hand- eye calibration for trajectory alignment in visual (-inertial) odometry evaluation,” IEEE Robot. Autom. Lett. , vol. 9, no. 6, pp. 5134–5141, April 2024
2024
-
[23]
OpenVINS: A research platform for visual-inertial estimation,
P. Geneva, K. Eckenhoff, W. Lee, Y . Yang, and G. Huang, “OpenVINS: A research platform for visual-inertial estimation,” in Proc. IEEE Int. Conf. Robot. Autom. , 2020, pp. 4666–4672
2020
-
[24]
DIDO: Deep inertial quadrotor dynamical odometry,
K. Zhang, C. Jiang, J. Li, S. Yang, T. Ma, C. Xu, and F. Gao, “DIDO: Deep inertial quadrotor dynamical odometry,” IEEE Robot. Autom. Lett., vol. 7, no. 4, pp. 9083–9090, July 2022
2022
-
[25]
A general approach to spa- tiotemporal calibration in multisensor systems,
J. Rehder, R. Siegwart, and P. Furgale, “A general approach to spa- tiotemporal calibration in multisensor systems,” IEEE Trans. Robot. , vol. 32, no. 2, pp. 383–398, April 2016
2016
-
[26]
iKalibr: Unified targetless spatiotemporal calibration for resilient integrated inertial systems,
S. Chen, X. Li, S. Li, Y . Zhou, and X. Yang, “iKalibr: Unified targetless spatiotemporal calibration for resilient integrated inertial systems,” IEEE Trans. Robot. , vol. 41, January 2025
2025
-
[27]
Extending kalibr: Calibrating the extrinsics of multiple IMUs and of individual axes,
J. Rehder, J. Nikolic, T. Schneider, T. Hinzmann, and R. Siegwart, “Extending kalibr: Calibrating the extrinsics of multiple IMUs and of individual axes,” in Proc. IEEE Int. Conf. Robot. Autom. , 2016, pp. 4304–4311
2016
-
[28]
VINS-Mono: A robust and versatile monocular visual-inertial state estimator,
T. Qin, P. Li, and S. Shen, “VINS-Mono: A robust and versatile monocular visual-inertial state estimator,” IEEE Trans. Robot., vol. 34, no. 4, pp. 1004–1020, July 2018
2018
-
[29]
Chess–calibrating the hand-eye matrix with screw constraints and synchronization,
K. Pachtrachai, F. Vasconcelos, G. Dwyer, V . Pawar, S. Hailes, and D. Stoyanov, “Chess–calibrating the hand-eye matrix with screw constraints and synchronization,” IEEE Robot. Autom. Lett. , vol. 3, no. 3, pp. 2000–2007, January 2018
2000
-
[30]
Efficient derivative computation for cumulative B-splines on Lie groups,
C. Sommer, V . Usenko, D. Schubert, N. Demmel, and D. Cremers, “Efficient derivative computation for cumulative B-splines on Lie groups,” in Proc. IEEE Comput. Vis. Pattern Recognit. , 2020, pp. 11 148–11 156
2020
-
[31]
Ceres Solver,
S. Agarwal et al., “Ceres Solver,” Accessed: Jun. 1, 2024. [Online]. Available: http://ceres-solver.org/
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.