{"id":"f0d299e2-caab-43e4-b074-37bfe6009916","arxiv_id":"2412.17235","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A LiDAR-visual SLAM system that detects degenerate LiDAR directions from the covariance of its state estimate and fuses visual measurements only along those directions.","lead":"This paper proposes a Selective Kalman Filter that uses visual camera data only when LiDAR positioning becomes degenerate, and only for the directions that are degenerate. Generalists should care because selective sensor fusion could make SLAM systems faster and more robust in feature-poor environments like walls and tunnels.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The information matrix J'_I in Eqs. (24)-(26) is not the information of the projected visual estimate; the filter overstates confidence whenever selected and unselected directions are correlated.","rationale":"The strongest claim of the paper is that the Selective Kalman Filter obtains accuracy comparable to full fusion while processing far less visual data. For that claim to hold, the selective update must be a valid Bayesian update: it must incorporate exactly the information contained in the visual measurements about the selected state directions, no more and no less. Section V attempts to construct a pseudo-measurement from the projected visual estimate, but the information matrix assigned to it is wrong. Since L = V S V^T is not guaranteed to commute with J_I, the paper's J'_I = L J_I L ignores the correlation between selected and unselected directions. The correct information about y = L x_hat is (L J_I^{-1} L)^+, which is strictly smaller in the sense of positive semidefinite order. Using J'_I therefore overstates the information, leading to underestimated posterior covariance and overconfident fusion. This directly undermines the theoretical justification of the method and the claim that it introduces fewer errors from visual measurements. The experimental section does not repair this: it reports single-run end-to-end errors without covariance consistency checks or statistical significance, and the code link is not verified. The degeneracy detection via covariance blocks (Section IV) is a sound and useful idea, but it is not enough to rescue the central derivation. Thus the reader's REJECT verdict is appropriate; our stress-test identifies the same load-bearing assumption and confirms it is not secure.","tokens_in":10804,"tokens_out":6056,"duration_ms":52621,"concrete_test":"Take a 2-DOF instance: J_I = [[a,b],[b,c]] with a = c = 1, b = 0.5, select only the first component (S = diag(1,0)). Compute the posterior covariance from Eq. (34) using the paper's J'_I = L J_I L and using the correct (L J_I^{-1} L)^+. With prior P = I, the paper's posterior variance in direction 1 is 1/(1+a) = 0.5, while the correct value is 1/(1+(a-b^2/c)) = 1/(1+0.75) ≈ 0.571. If the posterior variances differ, the selective update as stated is not a valid Bayesian update. Optionally, rerun SKF-Fusion on degenerate_seq_02 with the corrected information weight to see whether end-to-end error changes materially.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The selective update in Section V, Eqs. (24)-(26), treats J'_I = V S V^T J_I V S V^T as the information matrix of the projected visual estimate y = V S V^T J_I^{-1} b. This is incorrect. Since the visual estimate x_hat = J_I^{-1} b has covariance J_I^{-1}, the projected quantity y = L x_hat (with L = V S V^T) has covariance L J_I^{-1} L. Its information is the pseudo-inverse (L J_I^{-1} L)^+, not L J_I L. The two agree only when the selected and unselected directions are uncorrelated under J_I^{-1} (i.e., L commutes with J_I). In general, e.g., with L = diag(1,0) and J_I = [[a,b],[b,c]], L J_I L = a, while (L J_I^{-1} L)^+ = a - b^2/c < a. The paper's J'_I therefore overstates the information available from the visual measurement along the selected directions by ignoring the correlation with the discarded directions. Using J'_I in Eqs. (33)-(34) yields an underestimate of the posterior covariance and an overconfident update. This is a load-bearing error: the central claim that selective fusion 'introduces fewer errors from visual measurements' rests on the validity of this update, and the paper provides no proof that the cross terms vanish.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a LiDAR-visual-inertial SLAM fusion strategy that fuses visual measurements only when the LiDAR subsystem is deemed degenerate. Degeneracy is detected by eigendecomposing the marginal covariance blocks of Sigma = H_I^{-1}, which correctly accounts for rotation-translation coupling through the Schur complement, in contrast to methods that decompose the diagonal blocks of H_I separately. For fusion, the visual normal equations are projected onto the degenerate subspace with V S V^T and inserted into an information-form Kalman update. Experiments on R3LIVE datasets report accuracy comparable to R3LIVE with substantially reduced visual processing time.","tokens_in":11123,"tokens_out":13358,"duration_ms":126564,"significance":"The degeneracy detector in Section IV is mathematically sound and could be useful beyond this paper: the marginal covariances Sigma_rr and Sigma_tt are the correct rotation and translation uncertainties of the LiDAR least-squares estimate, and the paper explicitly shows why they differ from the inverse blocks of H_I. The paper also states that its code is openly available and reports a clear real-time benefit. However, the selective-update derivation in Section V is incorrect as written, so the central theoretical claim requires substantial revision.","major_comments":[{"comment":"The projected visual estimate tilde x = V S V^T x_hat has covariance V S V^T J_I^{-1} V S V^T, so its information matrix is the Moore-Penrose inverse of that covariance, not J'_I = V S V^T J_I V S V^T. These two expressions agree only when V S V^T and J_I commute, which the paper does not prove and which is false in general. For example, with L = diag(1,0) and J_I = [[a,b],[b,c]], one obtains L J_I L = a, whereas (L J_I^{-1} L)^+ = a - b^2/c < a. Consequently, Eq. (34) underestimates the posterior covariance whenever the selected and discarded visual directions are correlated. This is not a presentation issue: the claim that the Selective Kalman Filter introduces fewer errors from visual measurements depends on this update being a valid Kalman update. The authors should re-derive Eqs. (33)-(34) using the correct information matrix of the projected measurement, or prove that the cross terms vanish.","section":"V, Eqs. (24)-(26); VI, Eqs. (33)-(34)"},{"comment":"The accuracy comparison does not support the broad claim of fewer errors from visual measurements. In Table II, SKF-Fusion is sometimes worse than R3LIVE (hku park 00: 0.080 m vs 0.076 m) and only marginally better in most other sequences, and Section VIII notes that the datasets do not introduce visual disturbances such as drastic lighting changes. The well-supported benefit is the large reduction in visual subsystem computation time (Fig. 10, Table II); the accuracy and robustness claims should be stated more cautiously or tested under the visual failure modes named in the Introduction.","section":"VIII, Table II and accompanying text"}],"minor_comments":[{"comment":"The word 'implified' should be 'simplified', and the notation 'JIV SV^T' should be typeset as J_I V S V^T for readability.","section":"V, Eq. (25)"},{"comment":"The thresholds theta_r and theta_t are described only as set based on practical requirements and experience; given that the Conclusion admits they do not adapt across voxel resolutions, a sensitivity analysis or calibration procedure should be added.","section":"IV, thresholds"},{"comment":"The timing comparisons report only visual-subsystem cost, not total system runtime; the real-time claim should be scoped accordingly.","section":"VIII, Figs. 7 and 10"},{"comment":"The evaluation uses a single run per R3LIVE sequence and no statistical analysis; reporting multiple runs or at least error bars would strengthen the comparison.","section":"VIII, experimental protocol"},{"comment":"The abstract states that the code is openly available, but no repository URL is given in the manuscript.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The main mathematical error in Eqs. (24)-(26) and (33)-(34) appears fixable by using the correct information matrix of the projected visual measurement. However, because the reported experiments were run with the incorrect update, the revised filter may behave differently; the authors should be asked to re-run at least the degenerate-sequence experiments after correcting the update equations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the covariance-block degeneracy detector in Section IV is genuinely good: using Sigma = H^{-1} and the diagonal blocks of that gives a physically meaningful, rotation-translation-coupled measure, and the Schur complement argument against LION is correct. Second, the selective update in Section V has a real mathematical error that the authors have not noticed. The projected visual estimate y = L J_I^{-1} b does not have information matrix L J_I L; its information is the pseudo-inverse of L J_I^{-1} L. The paper's J'_I overstates the information whenever selected and unselected directions are correlated, which is generally the case. The 2x2 example in the stress-test note is enough to see it: with L=diag(1,0) and J_I = [[a,b],[b,c]], L J_I L = a while the correct information is a - b^2/c. So the posterior covariance in (34) is underestimated and the filter is overconfident. That is load-bearing for the 'introduces fewer errors' claim.\n\nWhat the paper does well: the degeneracy detector. The experiments show the covariance method predicts different degenerate directions than Hessian-based methods, and in the three degenerate sequences it gives better end-to-end errors and mapping than Zhang, LION, and X-ICP when embedded in the same SKF-Fusion framework. That is a meaningful empirical result. The code is released, though I did not run it.\n\nSoft spots beyond the math: the thresholds theta_r and theta_t are tuned by experience and the paper admits they do not transfer across voxel resolutions. The normal-dataset experiments are single-run and mostly show compute savings, which is fine but not decisive. The comparison to R3LIVE is on datasets without visual disturbances, so the 'fewer errors from visual measurements' claim is not really stress-tested.\n\nBottom line: the degeneracy detector is worth keeping, and the selective fusion idea is worth pursuing, but the update as written is incorrect. A revision that fixes the projected information matrix and re-derives the filter could be a solid paper. For now I would not trust the published equations, but I would send it to review because the detector alone, plus the reproducibility of the experiments, warrants referee time.","headline":"Covariance-based degeneracy detection is a real contribution, but the selective update misweights the projected visual information and is overconfident.","tokens_in":11661,"tokens_out":1960,"would_cite":true,"duration_ms":18120,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A selective Kalman filter that fuses visual data only when LiDAR degenerates matches full-fusion accuracy at a fraction of the cost.","keywords":["selective kalman filter","multi-sensor fusion","SLAM","degeneracy detection","LiDAR-inertial-visual odometry","covariance-based degeneracy","real-time performance"],"falsifier":"Construct a synthetic LiDAR-visual configuration where the visual Jacobian J has nonzero coupling between a degenerate and a non-degenerate direction, so that the information matrix J_I has off-diagonal blocks. Compute the true posterior covariance from the projected pseudo-measurement y = V S V^T $J_I^{{-1}}$ b and compare it with the covariance produced by Eq. (34). If the two disagree, the selective update is overconfident and the paper's uncertainty model is false.","tokens_in":10614,"feed_emoji":"🤖","tokens_out":4464,"duration_ms":38124,"temperature":0.7,"pith_summary":"This paper argues that multi-sensor fusion in SLAM does not need to use all data all the time. It proposes a Selective Kalman Filter that feeds visual measurements into a LiDAR-inertial system only when the LiDAR subsystem becomes degenerate, and only along the degenerate directions. The authors show that this selective strategy achieves accuracy comparable to the all-in fusion approach while drastically reducing the visual processing cost and avoiding errors from visual data in well-conditioned situations. The paper also introduces a covariance-based degeneracy detection method that accounts for the coupling between rotational and translational constraints, which it argues is more accurate than existing Hessian-based methods.","feed_headline":"Fuse visual data only when LiDAR degenerates: SLAM gets faster","feed_subtitle":"Covariance-based detection finds exactly which directions need help, so the visual subsystem runs a fraction of the time.","key_machinery":"The load-bearing object is the covariance matrix $\\Sigma = H_I^{-1}$ of the LiDAR point-to-plane measurement, split into rotation and translation blocks $\\Sigma_{rr}$ and $\\Sigma_{tt}$. Eigen-decomposition of these blocks gives the principal directions of uncertainty and their variances; a variance above a threshold marks a degenerate direction. A block-diagonal rotation matrix $V = \\mathrm{diag}(V_r, V_t)$ aligns the state coordinates with these principal directions, and a diagonal selective matrix $S$ retains only the degenerate diagonal entries. The filter then replaces the full visual information matrix $J_I = J^\\top Q^{-1} J$ and measurement vector $b$ with the projected quantities $J'_I = V S V^\\top J_I V S V^\\top$ and $b' = V S V^\\top J_I V S V^\\top J_I^{-1} b$, and runs the standard Kalman update with these replacements.","core_discovery":"The central claim is that the information contained in visual measurements is only needed in the directions where the LiDAR constraints are weak; fusing all visual data in all directions adds cost and can actively degrade accuracy. The paper formalizes this as a projection operation: after detecting degenerate directions from the eigenvalue decomposition of the covariance blocks, it defines a selective matrix S that zeroes out non-degenerate directions, and it derives a modified Kalman update (Eqs. 33–34) that uses this projected pseudo-measurement. The proposed degeneracy detector uses the inverse of the LiDAR information matrix, so that the diagonal blocks of the covariance reflect the actual variance of rotation and translation after accounting for the coupling between them, unlike Hessian-based methods that treat the blocks separately. Experiments on the R3LIVE datasets show that embedding this selective filter (SKF-Fusion) yields end-to-end errors as good as or better than R3LIVE while cutting per-frame visual processing time by roughly an order of magnitude.","pith_inferences":["If the assumption of block-diagonality (independence between selected and unselected directions) fails, the filter's posterior covariance understates uncertainty; a practitioner could recover the correct weight by using the pseudo-inverse of the projected covariance rather than $J'_I$.","The speed gain in the visual subsystem could allow the freed computation to be spent on higher-resolution visual processing, denser mapping, or more frequent LiDAR updates, potentially improving accuracy further.","The degeneracy detection principle — using the inverse information matrix rather than its blocks — could also improve observability analysis in other filtering-based fusion problems where state variables have different units.","The threshold selection is currently manual and tuned per voxel resolution; an adaptive or learned threshold could make the method work across different LiDAR configurations without retuning."],"forward_implications":["The computation time of the visual front end becomes proportional to the frequency of degeneracy rather than the full frame rate; in normal conditions the visual subsystem only maintains state without updating it.","The system becomes robust to visual failures (motion blur, lighting changes) because visual measurements are only trusted when LiDAR is degenerate, reducing the chance of injecting visual errors.","The method generalizes beyond LiDAR-visual fusion to any pair of sensors where one is more reliable but sometimes degenerate, such as wheel encoders or GPS, as the authors note.","The proposed degeneracy detector provides a physically meaningful, threshold-based test (variance in radians squared and meters squared) that can be used independently as a degeneracy monitoring tool."],"supporting_citations":[{"why":"Supplies the baseline LIVO framework (R3LIVE) on which SKF-Fusion is built and the datasets used for evaluation.","marker":"[2]"},{"why":"Provides the Kalman filter update formulas that the selective modification replaces.","marker":"[17]"},{"why":"Baseline degeneracy detection via Hessian condition number, which the paper argues lacks a dimensionless threshold.","marker":"[10]"},{"why":"Introduces the Hessian submatrix approach for rotation and translation separately, which the paper argues neglects coupling.","marker":"[12]"},{"why":"A related optimization-based method that fuses only in degenerate directions, used as a comparison point.","marker":"[14]"},{"why":"Uses eigenvalue thresholds for degeneracy detection, another baseline the paper compares against.","marker":"[11]"}],"fun_headline_variants":["SLAM fuses visuals only when LiDAR degenerates, boosting speed","Selective Kalman Filter: fuse visual data only in degenerate SLAM directions","Degeneracy-aware SLAM fusion: use visuals only where LiDAR is weak","When LiDAR fails, fuse visuals only in the needed direction: faster SLAM","Selective sensor fusion in SLAM: 10x less visual processing, same accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The filter's uncertainty after projecting visual data is correct only if the visual information in the directions we keep is uncorrelated with the information in the directions we discard; if that correlation is nonzero, the filter claims to know more than the visual measurement actually tells it.","fun_headline_variants_meta":{"raw":{"variants":["SLAM fuses visuals only when LiDAR degenerates, boosting speed","Selective Kalman Filter: fuse visual data only in degenerate SLAM directions","Degeneracy-aware SLAM fusion: use visuals only where LiDAR is weak","When LiDAR fails, fuse visuals only in the needed direction: faster SLAM","Selective sensor fusion in SLAM: 10x less visual processing, same accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000324,"raw_usage":{"total_tokens":1864,"prompt_tokens":1034,"completion_tokens":830,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":726}},"tokens_in":650,"tokens_out":830,"duration_ms":7214,"temperature":1.0,"reasoning_tokens":726,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:42:25.756998+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a synthetic LiDAR-visual configuration where the visual Jacobian J has nonzero coupling between a degenerate and a non-degenerate direction, so that the information matrix J_I has off-diagonal blocks. Compute the true posterior covariance from the projected pseudo-measurement y = V S V^T $J_I^{{-1}}$ b and compare it with the covariance produced by Eq. (34). If the two disagree, the selective update is overconfident and the paper's uncertainty model is false.","supporting_citations":[{"cited_title":"R 3 live: A robust, real-time, rgb-colored, lidar- inertial-visual tightly-coupled state estimation and mapping package,","cited_arxiv_id":null,"evidence_quote":"Supplies the baseline LIVO framework (R3LIVE) on which SKF-Fusion is built and the datasets used for evaluation."},{"cited_title":"On degeneracy of optimization-based state estimation problems,","cited_arxiv_id":null,"evidence_quote":"Baseline degeneracy detection via Hessian condition number, which the paper argues lacks a dimensionless threshold."},{"cited_title":"Lion: Lidar-inertial observability-aware navigator for vision-denied environments,","cited_arxiv_id":null,"evidence_quote":"Introduces the Hessian submatrix approach for rotation and translation separately, which the paper argues neglects coupling."},{"cited_title":"X-icp: Localizability-aware lidar registration for robust localization in extreme environments,","cited_arxiv_id":null,"evidence_quote":"A related optimization-based method that fuses only in degenerate directions, used as a comparison point."},{"cited_title":"Degeneracy-aware factors with applications to underwater slam,","cited_arxiv_id":null,"evidence_quote":"Uses eigenvalue thresholds for degeneracy detection, another baseline the paper compares against."}],"review_version":1}