REVIEW 5 major objections 4 minor 50 references
Rolling Shutter Camera Self-Calibration
T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper presents the first self-calibration method for rolling-shutter cameras, jointly estimating camera intrinsics and the readout time ratio from image sequences without calibration targets.
desk verdict RSSC has a genuinely useful dual-projection formulation, but the 'first self-calibration' claim needs a sharper prior-art argument and the supplementary Jacobians contain an SE(3) error that has to be resolved before the results can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the dual-projection model over a shared cumulative cubic B-spline trajectory on $SE(3)$. The trajectory supplies a continuous pose $T(t)$, so the pose of a row at time $t(\gamma, v) = (i + \gamma v/H)T_f$ can be evaluated alongside the pose at a reference global-shutter time $t_i$. For each 3D point, the residual combines the row-dependent RS reprojection at $t(\gamma, v)$ with the rectified GS reprojection at $t_i$, where the rectified measurement comes from Lagrange or Hermite interpolation of the point's 2D trajectory across neighboring frames. That coupling forces $K$ and $\gamma$ to be consistent with both temporal interpretations of the same observation.
What would settle it
Run RSSC-DP on a synthetic video with known $K$ and $\gamma$ but a motion known to be critical for self-calibration (e.g., pure rotation about the camera center, or a planar scene with no parallax) while using the paper's standard initialization; if the recovered $\gamma$ stays at the initial value or the intrinsics drift, the general-video claim fails. A cheap screening test is to sweep $\gamma$ from near 0 to near 1 at fixed trajectory and noise level and record the recovery rate.
Extended reading notes
Core claim
The central claim is that rolling-shutter self-calibration is possible: a bundle adjustment that optimizes the intrinsics $K$, the readout time ratio $\gamma$, the control poses of a cumulative B-spline trajectory, and the 3D points, with each observation projected twice — once at the row-dependent timestamp and once at a reference global-shutter timestamp — recovers $K$ and $\gamma$ without markers or hardware. The component models are RSSC-TE (continuous-time trajectory estimation) and RSSC-CE (correction-field estimation with quadratic or Hermite interpolation), and their union RSSC-DP ties the two projections to the same trajectory. In real-data experiments the proposed variants consiste
Load-bearing premise
The method recovers $K$ and $\gamma$ from 'a general video' only if the motion is non-degenerate and the initial global-shutter reconstruction, started with $\gamma=0.5$, lies in the basin of convergence; the paper defers the degeneracy analysis that would say which motions satisfy this.
Editorial extensions
If this is right
- Because the method is target-free, any sufficiently textured, non-degenerate video from a rolling-shutter camera could replace checkerboards, LED panels, and inertial sensors as the calibration input.
- The recovered intrinsics and readout ratio can initialize structure-from-motion, odometry, and image rectification with correct rolling-shutter parameters instead of a global-shutter assumption.
- The ablation shows a practical trade-off: correction-field variants converge fastest, trajectory estimation is stable under smooth motion, and the dual-projection variant is the most accurate across conditions.
- The supplementary comparison with target-based rolling-shutter calibration indicates the target-free estimates are competitive, so the method could also be used in controlled settings where a pattern is inconvenient.
Reading between the lines
- The authors explicitly defer degeneracy analysis, so 'general video' should be read as 'non-degenerate motion'; a catalogue of motions that make the readout ratio unobservable is the immediate next test.
- Because the pipeline starts from a global-shutter initializer, it likely inherits that initializer's failure modes under strong rolling-shutter distortion; an alternating initialization-and-refinement loop is a testable extension.
- The dual-timestamp projection idea is not tied to B-splines specifically; it transfers to any continuous-time motion model and, in principle, to sensors with per-row or per-line temporal sampling.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents RSSC, a framework for rolling-shutter (RS) camera self-calibration from image sequences without calibration targets. It combines two complementary models: RSSC-TE, which represents the camera trajectory with cumulative cubic B-splines and evaluates row-dependent poses; and RSSC-CE, which rectifies RS observations to global-shutter (GS) timestamps via 2D interpolation. These are unified in RSSC-DP, a weighted bundle-adjustment objective that constrains each 3D point by both the original RS projection and the rectified GS projection. The method jointly estimates intrinsics K, readout time ratio γ, poses, and 3D points. Experiments on synthetic data and the WHU-RSVI/TUM-RSVI datasets report improved intrinsic and trajectory accuracy over GS baselines, and the supplementary compares with a target-based RS calibration method.
Significance. If the technical issues below are resolved, the paper would be a useful contribution: it is a systematic attempt at target-free RS self-calibration in a BA framework, the dual-projection idea is elegant, and the open-source project page plus complete supplementary Jacobians are good practice. The paper also provides falsifiable predictions in the form of numeric error tables on standard datasets. However, the central claims currently outrun the evidence: the analytic Jacobians are suspect and internally inconsistent, degeneracy analysis is explicitly deferred, and the real-data evaluation lacks uncertainty quantification and a true RS self-calibration baseline. With a corrected Jacobian derivation and a sharper evaluation, the work could be publishable.
major comments (5)
- [Supp. Sec. 2.1, Eq. (16)] The Jacobian derivation is internally inconsistent and, as written, likely incorrect. Supp. Eq. (16) states ∂a_k/∂ξ_{i+k-1} = -∂a_k/∂ξ_{i+k-2} for k=1,2,3, where a_k = B_k(s) log(T_{i+k-2}^{-1}T_{i+k-1}). The derivative with respect to the two endpoints involves different Lie derivatives: a right perturbation of T_{i+k-1} gives B_k J_r^{-1}(Ω), while a perturbation of T_{i+k-2} introduces an additional adjoint factor, so the two derivatives are not negatives except in the zero-relative-motion limit. Indeed, Eq. (17) uses J_r^{-1}(Ω_{i+1}) Ad_{T_i^{-1}} for the ξ_i block, which contradicts Eq. (16). Since Algorithm 1 (step 28) forms J^T J from these blocks, the correctness of the reported convergence and accuracy results depends on this discrepancy. Please provide a corrected derivation or verify all Jacobians against numerical/automatic differentiation.
- [Sec. 6 (and Sec. 5.1)] The paper claims target-free self-calibration from 'general video', but no degeneracy analysis is provided; Sec. 6 explicitly defers it to future work. The synthetic study varies trajectory smoothness, γ, and measurement noise, but does not vary motion type (e.g., pure rotation, constant velocity, planar motions) that are known to be critical for RS SfM and self-calibration (see Refs. [4,16]). A concrete test would be to perturb the trajectory to such critical sequences and report γ/K observability or the basin of convergence around the COLMAP initialization used in Sec. 5.2. Without this, the central claim is formally unsupported.
- [Sec. 5.2, Table 3 and Supp. Tables 3-4] The real-data claims are stronger than the evidence. Table 3 reports only medians over sequences, with no spread, confidence intervals, or repeated-run variation; the supplementary per-sequence tables show large variance (e.g., TUM-RSVI6 RSSC-CEQ has ATE 0.1348 while RSSC-TE is 0.0085, and RSSC-CEH is 0.3363). The main baselines are all global-shutter methods; the only RS-aware method, RSCC, appears only in the supplementary Table 2, requires a calibration pattern, and is evaluated on only two short sequences, where the proposed methods are generally worse on γ. Thus 'consistently outperform' and 'competitive with target-based RS calibration' are not established at this level of evidence.
- [Sec. 4.2, Eqs. (8)-(15)] The CE residual is not an independent measurement. The rectified observation GSm_j^i(t_i) is computed from the observed points and timestamps t(γ, v) via Eq. (8) or (10); the interpolation weights depend on γ (Eq. (9)/(12)-(13)). Minimizing the residual (15) can therefore shrink the interpolation error by moving γ rather than by improving geometric consistency. RSSC-DP inherits this effect through its CE term (Eq. (18)). The synthetic results show γ errors for RSSC-CE growing as γ decreases, which is consistent with this coupling. The paper should analyze this bias or justify that the BA does not exploit it.
- [Sec. 2 (Related Work) and References] The claimed novelty 'first self-calibration method for RS cameras' is not properly positioned. Reference [16] (Ito & Okatani, CVPR 2017) is cited in the bibliography but never discussed in Sec. 2. Its title explicitly concerns 'self-calibration-based approach to critical motion sequences of rolling-shutter structure from motion'. If that method estimates RS intrinsics/readout parameters from target-free sequences, then the 'first' claim in the Abstract and Conclusion is inaccurate; if not, the distinction should be stated explicitly. Please add a comparative discussion.
minor comments (4)
- [Fig. 6 and Table 3] Fig. 6 uses labels RSSC-ET and RSSC-ECQ, while the text/table uses RSSC-TE and RSSC-CEQ; please unify. Also, the caption should identify which RSSC variants are shown.
- [Algorithm 1, steps 20-24] Step 20 forms residual e = [TE; λ·CE], but it is unclear whether λ scales the residual or the normal equations; the Jacobian assembly should be specified consistently.
- [Supp. Table 2] The title has a typo ('Comparison with RS-aware approache'). Please also define the abbreviations in the table caption.
- [Sec. 3.1, Eq. (1)] The definition of γ = T_readout/T_f is given, but Eq. (1) does not make explicit whether exposure time is included in T_readout; please clarify the relationship between row timestamp and exposure.
Circularity Check
No load-bearing circularity; one mild self-reference in the RSSC-CE rectified-observation residual, whose target is constructed from the readout ratio being estimated.
-
self definitional
[Sec. 4.2, Eqs. (8), (9), (15); used in RSSC-CE and RSSC-DP]
"GSm_j^i(t_i) = L_0(t_i)m_{i-1}^j + L_1(t_i)m_i^j + L_2(t_i)m_{i+1}^j, ... where the Lagrange basis functions are: L0(ti)=... (with t(γ,v) from Eq. (1)). CEe_j^i = GSm_j^i(t_i) − π(K GST_i P^j)."
The rectified observation GSm used as the target in the CE residual is not a raw measurement; it is constructed from the raw points via interpolation weights that depend on the readout time ratio γ (through t(γ,v) in Eq. (9) and Eq. (1)). Thus γ appears on both sides of the residual: it determines the 'measured' corrected point and is then optimized to match the projection. This makes the CE term a self-consistency constraint rather than an independent observation of γ. The effect is partial—RSSC-TE and the TE term in RSSC-DP use raw observations and provide independent anchoring, and real-data comparisons against ground truth validate the result externally—but the CE residual is not an independent measurement by construction.
full rationale
The core self-calibration claim is not circular: RSSC-TE minimizes a reprojection error between raw RS observations and a continuous-time B-spline trajectory, with γ entering only through the row timestamp in the pose interpolation (Eq. (7)); this is a standard geometric fit, not a reduction to the target. The real-data experiments are benchmarked against external datasets (WHU-RSVI, TUM-RSVI) with ground-truth intrinsics and trajectories, providing independent anchoring. The self-citations to prior work (e.g., [31] for quadratic interpolation, [24] for RS BA) are technique citations, not load-bearing uniqueness theorems. The mild self-reference flagged above in RSSC-CE/-DP is real but does not sink the central claim, because the TE component and the raw observations remain independent. The paper's explicit deferral of degeneracy analysis (Sec. 6, 'Future work includes ... studying degeneracies in RS camera self-calibration') and the potential Jacobian issue in Supp. Eq. (16) are correctness/completeness concerns, not circularity. Overall, the derivation chain is self-contained against external benchmarks; score 2 reflects the one constructed-measurement dependency rather than any forced equivalence.
Assumptions & free parameters
free parameters (2)
- lambda (DP weight) =
0.5
- gamma initialization =
0.5
assumptions (4)
- domain assumption Row-linear readout model with a single readout ratio gamma (Eq. 1)
- domain assumption Cumulative cubic B-spline trajectory smoothness (Eq. 3)
- domain assumption Quadratic or cubic interpolation of 2D point trajectories (Eqs. 8, 10)
- ad hoc to paper Non-degenerate motion and basin of convergence around the COLMAP initialization (Sec. 5.2)
Cite this review
Pith. "Pith review of Rolling Shutter Camera Self-Calibration." pith.science (2026). https://pith.science/paper/5AOKH4PY
@misc{pith2026260801509,
author = {Pith},
title = {Pith review of: Rolling Shutter Camera Self-Calibration},
year = {2026},
howpublished = {\url{https://pith.science/paper/5AOKH4PY}},
note = {Machine review of arXiv:2608.01509}
}
read the original abstract
Rolling shutter (RS) cameras are widely used in consumer devices, but their row-wise exposure causes distortions under motion, making geometric 3D vision problems dependent on both camera intrinsics and readout time ratio. Existing RS calibration methods rely on calibration targets or specialised hardware, limiting their use in unconstrained settings. We present the first self-calibration method for RS cameras that directly estimates camera intrinsics and the readout time ratio from image sequences, without requiring calibration targets. The method is implemented as a self-calibrating bundle adjustment (BA), which critically depends on the RS imaging model. We combine two known complementary models. The first formulates RS imaging as continuous-time trajectory estimation under a row-wise pose representation. The second interprets RS images as temporally distorted global shutter (GS) images and requires to estimate correction fields. The combination is non-trivial and results in a unified dual-projection model, in which each 3D point is simultaneously constrained at both row-dependent and reference timestamps along a shared continuous trajectory, enforcing stronger geometric and temporal consistency. Extensive simulations analyse the applicability of several implementations under varying conditions, and real data experiments demonstrate the accuracy, robustness, and practical effectiveness of the proposed approach.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[16]
International journal of computer vision , volume=
Shape and motion from image streams under orthography: a factorization method , author=. International journal of computer vision , volume=. 1992 , publisher=
work page 1992
- [1]
-
[2]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Towards nonlinear-motion-aware and occlusion-robust rolling shutter correction , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[3]
Astronomical and space optical systems , volume=
Fundamental performance differences between CMOS and CCD imagers: part III , author=. Astronomical and space optical systems , volume=. 2009 , organization=
work page 2009
-
[4]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=
Fast rolling shutter correction in the wild , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2023 , publisher=
work page 2023
-
[5]
arXiv preprint cs/0503076 , year=
Geometric models of rolling-shutter cameras , author=. arXiv preprint cs/0503076 , year=
-
[6]
IEEE transactions on pattern analysis and machine intelligence , volume=
Rolling shutter camera absolute pose , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2019 , publisher=
work page 2019
-
[7]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Rolling shutter camera relative pose: Generalized epipolar geometry , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Show all 50 references
-
[8]
European Conference on Computer Vision , pages=
Degeneracies in rolling shutter sfm , author=. European Conference on Computer Vision , pages=. 2016 , organization=
2016
-
[9]
2012 IEEE Conference on Computer Vision and Pattern Recognition , pages=
Rolling shutter bundle adjustment , author=. 2012 IEEE Conference on Computer Vision and Pattern Recognition , pages=. 2012 , organization=
2012
-
[10]
Pattern Recognition Letters , volume=
Robustified structure from motion with rolling-shutter camera using straightness constraint , author=. Pattern Recognition Letters , volume=. 2018 , publisher=
2018
-
[11]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Revisiting rolling shutter bundle adjustment: Toward accurate and fast solution , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[12]
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=
Rolling shutter camera calibration , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=
-
[13]
IEEE Sensors Journal , volume=
Continuous-time spatiotemporal calibration of a rolling shutter camera-IMU system , author=. IEEE Sensors Journal , volume=. 2022 , publisher=
2022
-
[14]
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=
Self-calibration-based approach to critical motion sequences of rolling-shutter structure from motion , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=
-
[15]
2023 , publisher=
Automated rolling shutter calibration with an LED panel , journal=. 2023 , publisher=
2023
-
[17]
2003 , publisher=
Multiple view geometry in computer vision , author=. 2003 , publisher=
2003
-
[18]
Proceedings of the IEEE International Conference on Computer Vision , pages=
Rolling-shutter-aware differential sfm and image rectification , author=. Proceedings of the IEEE International Conference on Computer Vision , pages=
-
[19]
IEEE Transactions on Intelligent Vehicles , year=
Ego-motion Estimation for Vehicles with a Rolling Shutter Camera , author=. IEEE Transactions on Intelligent Vehicles , year=
-
[20]
IEEE transactions on image processing , volume=
Analysis and compensation of rolling shutter effect , author=. IEEE transactions on image processing , volume=. 2008 , publisher=
2008
-
[21]
Sensors , volume=
Calibration and noise identification of a rolling shutter camera and a low-cost inertial measurement unit , author=. Sensors , volume=. 2018 , publisher=
2018
-
[22]
European conference on computer vision , pages=
Autocalibration from planar scenes , author=. European conference on computer vision , pages=. 1998 , organization=
1998
-
[23]
International Journal of computer vision , volume=
Self-calibration of a moving camera from point correspondences and fundamental matrices , author=. International Journal of computer vision , volume=. 1997 , publisher=
1997
-
[24]
2015 IEEE International Conference on Image Processing (ICIP) , pages=
Deepfocal: A method for direct focal length estimation , author=. 2015 IEEE International Conference on Image Processing (ICIP) , pages=. 2015 , organization=
2015
-
[25]
Proceedings of the 15th ACM SIGGRAPH European Conference on Visual Media Production , pages=
DeepCalib: A deep learning approach for automatic intrinsic calibration of wide field-of-view cameras , author=. Proceedings of the 15th ACM SIGGRAPH European Conference on Visual Media Production , pages=
-
[26]
2022 International Conference on Robotics and Automation (ICRA) , pages=
Self-supervised camera self-calibration from video , author=. 2022 International Conference on Robotics and Automation (ICRA) , pages=. 2022 , organization=
2022
-
[27]
Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
Deep geometry-aware camera self-calibration from video , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=
-
[28]
European conference on computer vision , pages=
Camera self-calibration: Theory and experiments , author=. European conference on computer vision , pages=. 1992 , organization=
1992
-
[29]
International workshop on vision algorithms , pages=
Bundle adjustment—a modern synthesis , author=. International workshop on vision algorithms , pages=. 1999 , organization=
1999
-
[30]
Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
Structure-from-motion revisited , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=
-
[31]
European Conference on Computer Vision , pages=
Geocalib: Learning single-image calibration with geometric optimization , author=. European Conference on Computer Vision , pages=. 2024 , organization=
2024
-
[32]
Structure-from-Motion Revisited , booktitle=
Sch\". Structure-from-Motion Revisited , booktitle=
-
[33]
IEEE Access , volume=
The WHU rolling shutter visual-inertial dataset , author=. IEEE Access , volume=. 2020 , publisher=
2020
-
[34]
IROS , year=
Rolling-shutter modelling for direct visual-inertial odometry , author=. IROS , year=
-
[35]
International journal of computer vision , volume=
A theory of self-calibration of a moving camera , author=. International journal of computer vision , volume=. 1992 , publisher=
1992
-
[36]
Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003
A survey of camera self-calibration , author=. Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003. , pages=. 2003 , organization=
2003
-
[37]
Proceedings of the European conference on computer vision (ECCV) , pages=
Fisheyerecnet: A multi-context collaborative deep network for fisheye image rectification , author=. Proceedings of the European conference on computer vision (ECCV) , pages=
-
[38]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Learning to calibrate straight lines for fisheye image rectification , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[39]
Proceedings of the IEEE/CVF International conference on computer vision , pages=
Ctrl-c: Camera calibration transformer with line-classification , author=. Proceedings of the IEEE/CVF International conference on computer vision , pages=
-
[40]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Deep single image camera calibration with radial distortion , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
-
[41]
2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=
Degeneracy in self-calibration revisited and a deep learning solution for uncalibrated slam , author=. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=. 2019 , organization=
2019
-
[42]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Depth from videos in the wild: Unsupervised monocular depth learning from unknown cameras , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[43]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Self-supervised learning with geometric constraints in monocular video: Connecting flow, depth, and camera , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[44]
Proceedings of the IEEE/CVF international conference on computer vision , pages=
Self-calibrating neural radiance fields , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=
-
[45]
arXiv preprint arXiv:1806.04807 , year=
Ba-net: Dense bundle adjustment network , author=. arXiv preprint arXiv:1806.04807 , year=
-
[46]
Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
Vggsfm: Visual geometry grounded deep structure from motion , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=
-
[47]
2025 International Conference on 3D Vision (3DV) , pages=
Flowmap: High-quality camera poses, intrinsics, and depth via gradient descent , author=. 2025 International Conference on 3D Vision (3DV) , pages=. 2025 , organization=
2025
-
[48]
Proceedings of the European Conference on Computer Vision (ECCV) , pages=
Direct sparse odometry with rolling shutter , author=. Proceedings of the European Conference on Computer Vision (ECCV) , pages=
-
[49]
2016 IEEE International Conference on Robotics and Automation (ICRA) , pages=
Direct semi-dense SLAM for rolling shutter cameras , author=. 2016 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2016 , organization=
2016
-
[50]
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
From two rolling shutters to one global shutter , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.