{"id":"ee3947a0-5123-4744-beec-45e5da0619df","arxiv_id":"2506.15242","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"RA-NeRF reconstructs NeRF scenes from unposed image sequences, using optical-flow-derived relative poses and a learned pose filter to keep camera poses accurate on complex rotational trajectories.","lead":"RA-NeRF is a neural 3D reconstruction method that estimates camera poses from image sequences while building the scene, using pixel-motion cues and a learned pose filter. A generalist might read it because robust camera pose estimation is the bottleneck for turning casually captured video into usable 3D models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Flow-driven pose regulation presumes PDC-Net essential-matrix poses are reliable targets; this is unmeasured, and near-pure-rotation/small-baseline pairs in NeRFBuster could make Eq. 2 inject wrong supervision.","rationale":"I read the manuscript in full and focused on the strongest claim: RA-NeRF obtains high-quality camera poses and reconstructs scenes from unposed images. The ablation analysis shows that flow-driven pose regulation is essential to the reported success, so the reliability of the flow-derived relative pose supervision is the most load-bearing assumption. The reader's weakest_assumption identifies exactly this assumption, and I agree with it. My stress-test adds two concrete aggravations: first, NeRFBuster's rotation-dominated trajectories create small-baseline adjacent pairs where essential-matrix translation direction is ill-conditioned and Eq. 2's normalization amplifies noise; second, the paper changes lambda_f by two orders of magnitude between datasets, so the mechanism's robustness across trajectory types is not demonstrated. I also flag a potential notational or formulaic error in the essential-matrix conversion that should be clarified. These are correctness risks rather than established failures; the paper has strong ablations and large reported margins, but no code, error bars, or direct evaluation of the flow-pose supervision quality. Therefore the conditional verdict is appropriate, and my read does not require changing it. The proposed concrete test would settle whether the flow supervision is reliable enough to support the central claim.","tokens_in":15642,"tokens_out":9109,"duration_ms":97076,"concrete_test":"Re-implement the Section III-B flow-pose module (PDC-Net + eight-point + positive-depth selection) and run it on the exact NeRFBuster pairs used during initialization and localization (adjacent frames and the N_loc=2 neighbors). Compute rotation and translation-direction errors of (R_hat_ij, T_hat_ij) against COLMAP relative poses, stratified by baseline length and rotation magnitude. If more than 10% of pairs exceed 5 degrees rotation error or 10 degrees translation-direction error, or if errors concentrate at small baselines, then Eq. 2 injects incorrect supervision in the regime the paper claims to handle, and the robustness claim is not established. Also verify the essential-matrix conversion: using the manuscript's E=K^{-1}FK^{-1} instead of E=K^T F K would break the supervision, so the implemented formula must be checked.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the flow-driven pose regulation in Eq. 2 (Section III-B). Table III shows this term is indispensable: without it, mean rotation error jumps from 1.14 degrees to 78.04 degrees. The supervision target is the relative pose (R_hat_ij, T_hat_ij) recovered from PDC-Net optical flow via the eight-point method, essential-matrix SVD, and positive-depth disambiguation. The paper never measures how often that recovered pose is correct on the actual pairs used (adjacent frames plus N_loc=2 neighbors) in the two test datasets. This matters because NeRFBuster is rotation-heavy (Section IV-A); adjacent views on such trajectories often have small baselines, where the translation direction from E is ill-conditioned and the normalized T_ij term in Eq. 2 amplifies noise. The four-way cheirality disambiguation can also select the wrong rotation under noisy flow. Additionally, lambda_f is changed by two orders of magnitude between datasets (1e-1 vs 1e-3, Section IV-B), so the term's strength is not shown to be robust. If flow-derived poses are unreliable in these regimes, the regularizer actively pulls poses toward wrong targets, and the good reported numbers cannot be attributed to the proposed mechanism. Note also that the E = K^{-1} F K^{-1} formula in Section III-B is not the standard essential-matrix conversion, which should be K^T F K; as written, the described computation would not yield an essential matrix, so the flow supervision step needs either correction or a clear notational explanation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RA-NeRF, an incremental NeRF-based pipeline that estimates camera poses and reconstructs a scene jointly from unposed image sequences. Two technical components are introduced: a flow-driven pose regulation term that derives relative pose targets from PDC-Net optical flow via the eight-point method and essential-matrix decomposition, and an implicit pose filter, an MLP that maps global and local motion embeddings to per-image SE(3) updates. The method is evaluated on the NeRFBuster and Tanks&Temple datasets, where it reports substantially lower rotation and translation errors than CF-NeRF, CF-3DGS, Nope-NeRF, BARF, and related baselines, along with better PSNR and LPIPS. Ablations in Tables III, IV, and V isolate the contributions of the flow regulation term and the implicit pose filter.","tokens_in":15894,"tokens_out":2415,"duration_ms":26163,"significance":"If the reported results are reproducible, the paper makes a strong practical contribution: it demonstrates that an incremental, photometric-plus-flow approach can estimate camera poses from scratch on rotation-heavy trajectories, a regime where global bundle-adjusting NeRF methods fail badly. The central mechanism is not circular, since the flow-derived relative poses come from a pretrained external network (PDC-Net) and not from the optimized pose variables. The ablations are informative and show large performance changes when either proposed component is removed, and the pose errors reported for NeRFBuster are an order of magnitude below the closest incremental baseline. The main risks are the reliability of the flow-derived supervision targets and the dataset-specific tuning of the flow loss weight, both of which are load-bearing for the method's central claim.","major_comments":[{"comment":"The conversion from the fundamental matrix to the essential matrix is stated as E_ij = K^{-1}_j F_ij K^{-1}_i. The standard conversion is E = K^T F K (with the appropriate ordering for two cameras). As written, the computation does not yield an essential matrix, so the relative pose (R_hat, T_hat) obtained by SVD would not be geometrically valid. Because this target directly supervises the poses in Eq. (2), this is a load-bearing technical point that must be corrected or explained with a clear convention for the K matrices.","section":"Section III-B, Eq. (2)"},{"comment":"The paper never measures how often the flow-derived relative pose (R_hat_ij, T_hat_ij) is correct on the actual pairs used (adjacent frames plus N_loc=2 neighbors) in NeRFBuster and Tanks&Temple. The ablation in Table III shows that this term is indispensable, but if PDC-Net flow or the positive-depth disambiguation fails on small-baseline or near-pure-rotation pairs, the regulation in Eq. (2) actively pulls poses toward an incorrect target. The authors should report a success-rate or error histogram of the flow-derived poses against COLMAP ground truth on the evaluation pairs, and discuss how failures are handled.","section":"Section III-B, Eq. (2)"},{"comment":"The flow regulation weight lambda_f is set to 1e-1 on NeRFBuster and 1e-3 on Tanks&Temple, a change of two orders of magnitude between datasets, yet no sensitivity analysis or multiple-seed evaluation is reported. This weakens the robustness claim in the conclusion: it remains possible that the method's success depends on carefully selecting this weight per dataset. A sensitivity table (e.g., lambda_f in {1e-4, 1e-3, 1e-2, 1e-1}) and repeated runs with different seeds would establish the stability of the reported mean errors.","section":"Section IV-B, Tables III and IV"},{"comment":"The related work cites CT-NeRF and LocalRF as the closest flow-based and incremental baselines, but neither appears in the quantitative comparison. Since the paper claims state-of-the-art performance on both datasets, omitting these published baselines makes the comparison incomplete and overstates the SOTA claim. The authors should add these methods or justify their absence with a concrete explanation (e.g., code or setting incompatibility).","section":"Section IV-D, Tables I and II"}],"minor_comments":[{"comment":"The text says \"IF-NeRF proposes the implicit pose filter,\" but IF-NeRF is not introduced anywhere; this appears to be a typo for RA-NeRF and should be corrected.","section":"Section IV-E-b"},{"comment":"Reference [16] contains a typo in the title: \"imporved-nerfmm\" should be \"improved-nerfmm.\"","section":"References"},{"comment":"The phrase \"high-accurate\" should be \"high-accuracy,\" and the definition of lambda_i in Eq. (4) is ambiguous: Section III-D states lambda_i=1 during initialization/localization and lambda_i=0 during optimization, but the role of lambda_i versus lambda_f should be stated explicitly when the loss is first introduced.","section":"Introduction and Section III-D"},{"comment":"The table header uses \"se(3)\" and \"SE(3)\" with inconsistent capitalization; please unify the notation to SE(3) for the manifold and se(3) for the Lie algebra.","section":"Table V"}],"recommendation":"major_revision","confidential_remarks":"The paper builds directly on the authors' own CF-NeRF pipeline and uses a COLMAP-derived benchmark, which is a mild self-reference but not a correctness problem. The more substantive concern for the editor is the unvalidated reliability of the flow-derived pose targets: given the extremely large effect of the flow term in Table III, the paper's central claim depends on this external signal being accurate on the evaluated pairs. I would welcome a revision that adds a validation experiment for the flow targets and a hyperparameter sensitivity analysis; these are within the scope of the current manuscript and would materially strengthen the contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: RA-NeRF is an incremental NeRF pipeline that combines two concrete ideas -- regulating the estimated relative poses against relative poses computed from optical flow via the essential matrix, and an MLP \"implicit pose filter\" that maps a global-plus-per-image embedding to a pose update. The ablations show both components pull their weight: without flow regulation, rotation error jumps from ~1.1 to 78 degrees on NeRFBuster; the filter also helps. That is real evidence and I believe the components are doing something.\n\nThe paper is readable and the tables are thorough. But I have three concerns that keep me from accepting the results at face value.\n\nFirst, the essential-matrix step is described as E = K^{-1} F K^{-1}. That is not the standard conversion; the standard form is K^T F K (with appropriate indices). As written, the step would not recover an essential matrix. This is either a typo or a mathematical error, and it sits directly under the main loss.\n\nSecond, the flow-derived relative pose is the target for Eq. 2, but the paper never measures how often that target is correct on the actual pairs used. On rotation-heavy trajectories, adjacent frames often have small baselines, where the translation direction from E is ill-conditioned and normalizing T amplifies noise. The lambda_f weight differs by two orders of magnitude between datasets (1e-1 vs 1e-3), which suggests the method is sensitive to that term rather than robust.\n\nThird, the evaluation omits the closest incremental flow-based baselines, CT-NeRF and LocalRF, even though both are cited. On top of that, there are no error bars or multiple seeds, and the ground truth on NeRFBuster is from COLMAP, which is itself known to be fragile on rotation-heavy sequences.\n\nThese are not fatal. The proposed mechanism is sensible and the gains over the included baselines are large. But the missing baselines and the formula issue mean the SOTA claim is not yet established at the level the paper states. I'd send this out for review, with a request for the authors to fix the formula, add CT-NeRF and LocalRF, report variance, and ideally justify lambda_f or show robustness. If the code is released and the numbers reproduce, this would be a solid contribution to the pose-free NeRF literature.","headline":"A plausible incremental pose-free NeRF pipeline with strong ablations, but a suspect essential-matrix formula and missing closest baselines keep the SOTA claim from being fully established.","tokens_in":16537,"tokens_out":3102,"would_cite":false,"duration_ms":29441,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RA-NeRF recovers camera poses from raw image sequences with about one-degree mean rotation error, even under complex trajectories.","keywords":["neural radiance fields","camera pose estimation","optical flow","pose regularization","incremental learning","novel view synthesis","SE(3) pose updates","implicit pose filter"],"falsifier":"Run RA-NeRF on a trajectory with ground-truth poses and with a scene containing large textureless or reflective regions where learned optical flow is known to fail; then disable the flow-driven regulation and compare. If pose accuracy does not degrade, or if the regularization visibly increases error, the flow supervision is not the load-bearing component the paper claims it is. A more direct test: corrupt the optical flow during the initialization stage and check whether the final pose error mirrors the corruption.","tokens_in":15365,"feed_emoji":"🎥","tokens_out":11476,"duration_ms":99429,"temperature":0.7,"pith_summary":"RA-NeRF claims that camera poses accurate enough for high-quality 3D reconstruction can be recovered directly from raw image sequences, even when the camera follows large rotational trajectories rather than translating forward. The key claim is that a learned optical-flow network can supply reliable relative-pose supervision during incremental NeRF optimization, while an implicit pose filter denoises the pose updates. On a benchmark of rotational camera trajectories the method reports a mean rotation error of about 1.14 degrees and translation error of about 0.54, down from roughly 7 to 11 degrees for earlier pose-free pipelines, with state-of-the-art rendering quality. If the claims hold, NeRF and 3D Gaussian Splatting pipelines would no longer need a separate structure-from-motion stage to supply camera poses before reconstruction.","feed_headline":"Camera poses from unposed images drop to 1.1° mean rotation error","feed_subtitle":"Flow-driven pose regulation plus a learned pose filter lets NeRF handle large rotations without pose priors.","key_machinery":"Flow-driven pose regulation is the central object: for a pair of images, sampled correspondences from a learned optical-flow network go through the eight-point algorithm to give a fundamental matrix, which is converted to an essential matrix and decomposed by SVD into four candidate relative poses; the one with positive triangulated depth is chosen. The loss term then measures the distance between that flow-derived relative pose and the relative pose formed from the current camera estimates, with translation compared only in direction. The second mechanism is an implicit pose filter, an 8-layer MLP with a residual connection that takes a shared global motion embedding and a per-image local embedding and outputs a six-dimensional update per image in the Lie algebra se(3), converted to a rigid transformation by the exponential map. Camera poses are updated on the Lie group SE(3), the group of rigid camera motions, via the exponential map, which the ablations identify as more stable than adding a raw se(3) vector.","core_discovery":"RA-NeRF establishes that the relative pose between two images, computed directly from optical-flow correspondences by the eight-point algorithm and essential-matrix decomposition, can act as an external supervisory signal that keeps NeRF pose optimization from sinking into photometric local minima. The method compares these flow-derived relative rotations and translation directions with the relative poses implied by the current camera parameters and penalizes the difference, supervising only translation direction because the flow-derived translation has no absolute scale. It further replaces raw per-pose learnable updates with an implicit pose filter, a residual network that reads a global motion embedding and per-image local embeddings and outputs an se(3) update, then applies the update through the exponential map. The reported results on two real-world benchmarks support the claim that this combination yields state-of-the-art camera-pose accuracy and novel-view quality from unposed images.","pith_inferences":["The regularization's dependence on optical flow suggests the method should degrade on scenes with reflections, repetitive texture, or near-degenerate camera motion; the paper does not test these failure modes.","Because only translation direction is supervised, the absolute scale of the trajectory is pinned down by the NeRF photometric optimization; a testable prediction is that low-parallax videos will show larger translation drift than rotation error.","The same flow-regulation and implicit-filter components could in principle be attached to 3D Gaussian Splatting or a hash-grid NeRF backbone; the paper only demonstrates them with a plain MLP NeRF.","The global motion embedding may encode scene-specific motion patterns, so a filter trained on one scene may need re-initialization when applied to a different trajectory; adapting it across scenes without re-training remains an open question."],"forward_implications":["Pose-free NeRF reconstruction becomes practical for large-rotation trajectories, not only forward-looking scenes.","Incremental pipelines can use flow-derived relative poses instead of depth priors or known pose distributions to avoid local minima.","Supervising only the direction of translation sidesteps the scale ambiguity of monocular optical flow, keeping the regularization usable from scratch.","The implicit pose filter shows that learned denoising of pose gradients can be shared across frames through a global motion embedding, improving pose accuracy in complex motions.","The full pipeline offers an end-to-end route from unposed images to a reconstructed NeRF, removing a separate structure-from-motion preprocessing stage."],"supporting_citations":[{"why":"supplies the incremental reconstruction and localization pipeline that RA-NeRF extends with flow regulation and pose filtering.","marker":"[2]"},{"why":"provides the analogous incremental pipeline built on 3D Gaussian Splatting and depth priors, the strongest baseline to beat.","marker":"[3]"},{"why":"establishes the photometric-consistency training of NeRF with learnable poses and the no-positional-encoding choice adopted here.","marker":"[15]"},{"why":"supplies the learned optical-flow network used to generate image correspondences for the pose-regulation loss.","marker":"[26]"},{"why":"provides the eight-point algorithm that computes the fundamental matrix from flow matches.","marker":"[52]"},{"why":"gives the essential-matrix decomposition and multi-view geometry rules used to extract candidate relative poses.","marker":"[54]"},{"why":"identifies the four-solution ambiguity of essential-matrix decomposition that the direct relative-pose supervision is designed to avoid.","marker":"[19]"},{"why":"supplies a test set of casually captured scenes dominated by rotational camera motion, where pose-free methods usually fail.","marker":"[4]"},{"why":"supplies a standard outdoor reconstruction benchmark used for the second evaluation.","marker":"[35]"},{"why":"provides the rendering toolbox used to evaluate novel-view synthesis quality fairly across methods.","marker":"[57]"}],"fun_headline_variants":["Flow-based pose priors sharpen NeRF under complex trajectories","RA-NeRF: NeRF with flow-driven regulation and pose filter","Learning NeRF camera poses from optical-flow correspondences","Flow-guided pose supervision boosts NeRF camera tracking"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that optical-flow correspondences, after positive-depth disambiguation, give a trustworthy relative-pose signal even when the current camera estimates are far off; if the flow is wrong or the disambiguation picks the wrong solution, the regularizer actively drags poses away from the truth.","fun_headline_variants_meta":{"raw":{"variants":["Flow-based pose priors sharpen NeRF under complex trajectories","RA-NeRF: NeRF with flow-driven regulation and pose filter","Learning NeRF camera poses from optical-flow correspondences","Flow-guided pose supervision boosts NeRF camera tracking"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000164,"raw_usage":{"total_tokens":1240,"prompt_tokens":932,"completion_tokens":308,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":241}},"tokens_in":548,"tokens_out":308,"duration_ms":3621,"temperature":1.0,"reasoning_tokens":241,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:40:01.439513+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RA-NeRF on a trajectory with ground-truth poses and with a scene containing large textureless or reflective regions where learned optical flow is known to fail; then disable the flow-driven regulation and compare. If pose accuracy does not degrade, or if the regularization visibly increases error, the flow supervision is not the load-bearing component the paper claims it is. A more direct test: corrupt the optical flow during the initialization stage and check whether the final pose error mirrors the corruption.","supporting_citations":[{"cited_title":"Cf- nerf: Camera parameter free neural radiance fields with incremental learning,","cited_arxiv_id":null,"evidence_quote":"supplies the incremental reconstruction and localization pipeline that RA-NeRF extends with flow regulation and pose filtering."},{"cited_title":"Barf: Bundle- adjusting neural radiance fields,","cited_arxiv_id":null,"evidence_quote":"establishes the photometric-consistency training of NeRF with learnable poses and the no-positional-encoding choice adopted here."},{"cited_title":"Learning accu- rate dense correspondences and when to trust them,","cited_arxiv_id":null,"evidence_quote":"supplies the learned optical-flow network used to generate image correspondences for the pose-regulation loss."},{"cited_title":"In defense of the eight-point algorithm,","cited_arxiv_id":null,"evidence_quote":"provides the eight-point algorithm that computes the fundamental matrix from flow matches."},{"cited_title":"Lu-nerf: Scene and pose estimation by synchronizing local unposed nerfs,","cited_arxiv_id":null,"evidence_quote":"identifies the four-solution ambiguity of essential-matrix decomposition that the direct relative-pose supervision is designed to avoid."}],"review_version":2}