REVIEW 4 major objections 8 minor 15 references
Flow Distillation Sampling: Regularizing 3D Gaussians with Pre-trained Matching Priors
T0 review · 4 major / 8 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Frozen optical flow models can supervise 3D Gaussian geometry, restoring sparse indoor scenes.
desk verdict Solid within-subfield contribution showing optical flow priors regularize 3DGS geometry, but the load-bearing reliability claim rests on one qualitative figure. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the pair of flow fields between an input view and a deliberately sampled unobserved view. Radiance Flow is derived in closed form from the rendered depth and relative camera pose (equations 4-5); Prior Flow is the output of a pre-trained optical flow network $M_\theta(I_i, C_s)$ evaluated on the real input image and the rendered sampled view (equation 13). The FDS loss (equation 14) is the squared L2 distance between these two flows, with the Prior Flow detached so it acts as a stable teacher. The accompanying camera sampling scheme (equation 12) selects a pure-translation offset with radius $\epsilon_t = \sigma \bar{D}_i / f$, so that the expected optical flow magnitude is constant across views and datasets; this tunes the overlap between input and sampled views so the matching model sees enough parallax to constrain geometry without producing flows too large to estimate reliably.
What would settle it
Compare Prior Flow and Radiance Flow against ground-truth flow (computed from ground-truth depth and poses) for many sampled views during early training, when renders contain floaters; if Radiance Flow is on average as accurate as Prior Flow, the loss would be supervising good geometry with worse or equal signal. A second check: apply a fixed geometric distortion (e.g., a consistent scale change) to the Prior Flow teacher and show FDS still improves geometry; if it does, the gains are not attributable to the matching prior's correctness.
Extended reading notes
Core claim
On its own terms, FDS establishes that pairwise matching priors carry enough absolute-scale geometric signal to regularize a Gaussian radiance field in under-observed regions. For each training view, the method samples an unobserved view along a small depth-adaptive spiral translation, renders color and depth for that view, and forms the Radiance Flow by projecting pixels through rendered depth and the known relative pose. A pre-trained optical flow model then matches the real input image against the rendered image to produce the Prior Flow, which is detached and used as a fixed teacher in the squared-error loss $L_{\mathrm{fds}} = \|F_{\mathrm{prior}} - F_{\mathrm{radiance}}\|^2$. The claim is that even when the rendered image is blurry and filled with floaters, the Prior Flow is more robust and precise than the analytic Radiance Flow, so the loss guides Gaussian positions toward the true surface. Experiments on Mushroom, ScanNet, and Replica show FDS substantially improves depth rendering and mesh extraction over 3DGS and 2DGS baselines, and ablations indicate the matching prior outperforms monocular depth and normal priors alone while remaining complementary to them.
Load-bearing premise
The method assumes that the pre-trained optical flow model, when given one real photo and one blurry render full of floaters, produces a flow that is more accurate and more geometrically consistent than the flow computed from the current 3D model, throughout training.
Editorial extensions
If this is right
- Applied to either 3DGS or 2DGS, FDS cuts depth error (Abs Rel) by roughly half on Mushroom and ScanNet indoor scenes and improves F-score and normal consistency for extracted meshes.
- Because the supervision comes from pairwise matching rather than monocular cues, FDS provides absolute scale information, sidestepping the scale ambiguity of monocular depth and normal priors.
- Swapping the optical flow backbone from RAFT to Sea RAFT improves results, so the method's gains scale with the quality of the pre-trained matching model.
- Using the real input image rather than the rendered one for Prior Flow, and re-sampling the unobserved view each iteration, are both necessary: the ablation shows fixed or rendered inputs degrade geometry.
Reading between the lines
- If the pretrained flow model is systematically biased by rendering artifacts, the mutual-refinement loop could reinforce those artifacts instead of removing them; testing FDS with a deliberately corrupted flow teacher would reveal whether the loss survives imperfect teachers.
- The depth-adaptive sampling radius uses a single mean depth per image, so in scenes with strong depth variation the flow supervision is uneven across pixels; a per-pixel or depth-bucketed radius might extend robustness.
- The same recipe—freeze a matching network, render from a differentiable scene representation, match against a real image, and backprop through the renderer into the geometry—should transfer to other representations such as signed distance fields or meshes, not just Gaussians.
- The paper's stated limitations (lighting changes, reflective surfaces, motion blur) are inherited from the optical flow backbone, which suggests that improving or specializing the matching model could directly expand FDS's applicability to those conditions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Flow Distillation Sampling (FDS), a training-time regularizer for 3D Gaussian Splatting that uses a pretrained optical flow network to compute a 'Prior Flow' between an input view and a randomly sampled nearby unobserved view, then supervises the analytically computed 'Radiance Flow' from the current Gaussians with an L2 loss. The camera sampler uses a depth-adaptive translation radius to maintain a roughly constant expected flow magnitude. The method is evaluated on Mushroom, ScanNet, and Replica for depth rendering, mesh extraction, and novel view synthesis, showing consistent improvements over 3DGS and 2DGS baselines, as well as comparisons with other priors. The central claim is that the prior flow is more robust than the radiance flow during training, enabling a mutually reinforcing refinement of geometry.
Significance. If the central assumption holds, FDS offers a simple, representation-agnostic way to inject cross-view matching priors into radiance field optimization, with substantial gains in indoor geometry reconstruction. The method is clearly described and the camera sampling derivation is transparent. Strengths include the generality across 3DGS and 2DGS, the comparison against multiple priors (depth, normal, two-view depth), and the ablation of design choices. The main significance depends on whether the pretrained flow model can reliably supervise blurry rendered views; the current evidence is largely qualitative.
major comments (4)
- [Sec. 3.2.2, Eq. (14), Fig. 4] The entire FDS loss rests on the assertion that the Prior Flow F_prior is 'more robust and precise during training' than the Radiance Flow F_radiance, but the only evidence is the qualitative error map in Fig. 4, which uses Raft rather than the default SeaRaft and shows only a single training stage. Please provide a quantitative comparison of flow end-point error (against ground-truth depth-derived flow) for both flows across training iterations and across all test scenes, including the early phase around iteration 15k when C_s is blurry. This is necessary to rule out the possibility that Eq. (14) amplifies errors when M_theta is confused by rendering artifacts.
- [Sec. 3.2.2, Eqs. (13)-(14)] Because F_prior = M_theta(I_i, C_s) is computed from the rendered image C_s of the current Gaussians, detaching F_prior (as stated after Eq. 14) only prevents gradients from flowing into M_theta; the target itself remains a function of the current geometry. The paper claims a 'mutually reinforcing effect,' but it does not provide evidence that this loop converges to a better geometry rather than to a state that merely makes the two flows agree. Please report the evolution of the flow error and of the FDS loss over training, and compare against a variant where C_s is replaced by a less geometry-dependent input (e.g., a fixed reference rendering updated every N iterations).
- [Sec. 4.1.1, Tables 1-2] The authors remove the depth distortion loss from 2DGS for all indoor experiments, stating that it degrades results. This modifies the baseline against which FDS is compared. Since 2DGS with its original loss is the established method, please also report 2DGS and 2DGS+FDS with the original loss (or at least provide a supplementary experiment) to confirm that the observed gains are not an artifact of the weakened baseline. The same clarification is needed for whether the normal prior from Stable Normal is applied to all compared methods (GOF, PGSR, 3DGS, 2DGS) in Tables 1-2 or only to the 3DGS variants.
- [Table 3, last row] The LPIPS value 0.0403 in the last row is an order of magnitude smaller than the values in all other rows (approximately 0.26) and is inconsistent with the PSNR and SSIM in the same row. This appears to be a typographical error; please correct it and double-check all entries in the ablation tables.
minor comments (8)
- [Abstract and Introduction] The name 'Flow Distillation Sampling' may mislead readers into thinking the method samples flow; consider clarifying in the text that 'sampling' refers to the camera-view sampling scheme.
- [Eqs. (13)-(14)] The notation F^{i→s} is used for both the Prior Flow and the Radiance Flow; please use distinct symbols (e.g., F_p and F_r) to avoid ambiguity.
- [Fig. 4 caption] The meaning of RF* and the training-iteration numbers is not explained; please describe each panel and specify which flow model is used.
- [Sec. 3.2.1, Eq. (11)] The derivation uses per-pixel depth to set the translation radius, but then replaces it with the mean depth \bar{D}_i; the approximation error for pixels whose depth deviates from the mean is not discussed. Please add a sentence justifying this choice.
- [Sec. 4.2.1] The statement 'the render quality of RGB images shows a slight improvement, by 0.58 in 3DGS and 0.50 in 2DGS' should specify the metric (PSNR) and provide a per-scene breakdown.
- [Table 4] The table header 'Mθ(X,Cs) Loss Metric' is unclear; the checkmark configuration should be self-explanatory or the caption should explain each column.
- [General] No error bars or statistical significance tests are provided for the main metrics; given that each dataset uses only five scenes, please report standard deviations or per-scene values, especially for the ScanNet results where improvements are more moderate.
- [General] Typographical and notation issues: 'incoporated' in Related Work, 'Colmap' should be 'COLMAP', and Algorithm 1 line 7 mixes 2D and 3D coordinates; please proofread.
Circularity Check
The 'recover absolute scale' contribution is defined away by the depth-adaptive camera sampling (Eq. 12); the main benchmark geometry gains are external and non-circular.
-
self definitional
[Sec. 3.2.1, Eqs. (10)-(12); Sec. 1 contribution bullet; Sec. 3.2.2, Eq. (14)]
"Thus, the radius of translation in our camera sampling is defined as ϵt = σ Di(u1,v1)/f which helps maintain stable flow. ... Given that pixel depths vary within an image, we use the mean depth ¯Di of the image and set the radius of our translation ϵt = σ ¯Di/f. ... FDS leverages matching prior information to recover absolute scale."
Substituting Eq. (12) into Eq. (10) gives F^{i→s} = σ·(¯D_i/D_i)·(sin,cos), which is a scale-free depth ratio: the sampled-view translation is defined from the very current rendered depth whose absolute scale the method claims to recover. The Prior Flow M_θ(I_i,C_s) is also unchanged under a simultaneous rescaling of the scene depth and the sampled-camera translation, since C_s is rendered with that same scaled translation, and F_radiance is invariant by the same construction. Hence L_fds = ||F_prior − F_radiance||^2 in Eq. (14) cannot distinguish a scene at scale s from one at scale 1: the 'recover absolute scale' claim reduces, by the paper's own Eq. (12), to a scale-invariant ratio.
full rationale
The core FDS mechanism — computing Radiance Flow analytically from rendered depth and supervising it with Prior Flow from a fixed, independently pretrained optical flow model (Eqs. 13-14) — is not logically circular: no parameter is fitted to the model's own outputs, and the flow model is an external trained component. The reported geometry improvements on Mushroom, ScanNet, and Replica are measured against external depth, mesh, and rendering benchmarks, so the main empirical claim has independent content. I found no load-bearing self-citation and no imported uniqueness theorem. The one genuine self-definitional step is the secondary 'recover absolute scale' contribution: Eq. (12) sets the sampled-view translation from the current mean rendered depth, and Eq. (10) then makes the supervised F_radiance a scale-free ratio, so the loss cannot constrain absolute scale by construction. This is partial circularity in a supporting claim, not in the benchmarked central result. The additional concern that C_s depends on the current Gaussians is a soft self-training feedback risk, not a definitional circularity, because the flow model itself is external.
Assumptions & free parameters
free parameters (3)
- sigma (target flow magnitude) =
23
- lambda_fds =
0.015
- lambda_normal =
0.15
assumptions (3)
- domain assumption Pretrained optical flow models (Sea Raft, Raft) produce reliable metric flow between a real input image and a rendered sampled view, even when the rendered view is blurry or contains floaters.
- domain assumption A pure-translation camera movement with no rotation and t3=0 (with fx close to fy) is sufficient to obtain informative matching priors.
- domain assumption The mean depth of the image can be used to choose a camera translation radius that maintains stable flow magnitude globally.
Cite this review
Pith. "Pith review of Flow Distillation Sampling: Regularizing 3D Gaussians with Pre-trained Matching Priors." pith.science (2026). https://pith.science/paper/AEICWAKQ
@misc{pith2026250207615,
author = {Pith},
title = {Pith review of: Flow Distillation Sampling: Regularizing 3D Gaussians with Pre-trained Matching Priors},
year = {2026},
howpublished = {\url{https://pith.science/paper/AEICWAKQ}},
note = {Machine review of arXiv:2502.07615}
}
read the original abstract
3D Gaussian Splatting (3DGS) has achieved excellent rendering quality with fast training and rendering speed. However, its optimization process lacks explicit geometric constraints, leading to suboptimal geometric reconstruction in regions with sparse or no observational input views. In this work, we try to mitigate the issue by incorporating a pre-trained matching prior to the 3DGS optimization process. We introduce Flow Distillation Sampling (FDS), a technique that leverages pre-trained geometric knowledge to bolster the accuracy of the Gaussian radiance field. Our method employs a strategic sampling technique to target unobserved views adjacent to the input views, utilizing the optical flow calculated from the matching model (Prior Flow) to guide the flow analytically calculated from the 3DGS geometry (Radiance Flow). Comprehensive experiments in depth rendering, mesh reconstruction, and novel view synthesis showcase the significant advantages of FDS over state-of-the-art methods. Additionally, our interpretive experiments and analysis aim to shed light on the effects of FDS on geometric accuracy and rendering quality, potentially providing readers with insights into its performance. Project page: https://nju-3dv.github.io/projects/fds
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[3]
URL https://arxiv.org/abs/2312.00846. 2, 3 Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In IEEE Conf. Comput. Vis. Pattern Recog., pages 5828–5839,
-
[4]
Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray tracing
3 Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray tracing. arXiv preprint arXiv:2311.16043,
-
[5]
Mesh- based gaussian splatting for real-time large-scale deformation, 2024a
1 Lin Gao, Jie Yang, Bo-Tao Zhang, Jia-Mu Sun, Yu-Jie Yuan, Hongbo Fu, and Yu-Kun Lai. Mesh- based gaussian splatting for real-time large-scale deformation, 2024a. URL https://arxiv. org/abs/2402.04796. 2 Xiangjun Gao, Xiaoyu Li, Yiyu Zhuang, Qi Zhang, Wenbo Hu, Chaopeng Zhang, Yao Yao, Ying Shan, and Long Quan. Mani-gs: Gaussian splatting manipulation wi...
-
[6]
3 Antoine Guédon and Vincent Lepetit
URL https://arxiv.org/ abs/2311.12775. 3 Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In IEEE Conf. Comput. Vis. Pattern Recog.,
-
[7]
2d gaussian splatting for geometrically accurate radiance fields
3 Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In ACM SIGGRAPH Anal. Conf. , 2024a. doi: 10.1145/ 3641519.3657428. 2 Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields, 2024b. URL ht...
-
[9]
URL https://arxiv.org/abs/2404.00409. 2 Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106,
-
[10]
3 12 Published as a conference paper at ICLR 2025 Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gillingham, Elias Mueggler, Luis...
arXiv 2025
-
[11]
Dn-splatter: Depth and normal priors for gaussian splatting and meshing
3 Matias Turkulainen, Xuqian Ren, Iaroslav Melekhov, Otto Seiskari, Esa Rahtu, and Juho Kan- nala. Dn-splatter: Depth and normal priors for gaussian splatting and meshing. arXiv preprint arXiv:2403.17822, 2024a. 3 Matias Turkulainen, Xuqian Ren, Iaroslav Melekhov, Otto Seiskari, Esa Rahtu, and Juho Kannala. Dn-splatter: Depth and normal priors for gaussia...
Show all 15 references
-
[12]
org/abs/2402.01459
URL https://arxiv. org/abs/2402.01459. 2 Chaoyang Wang, Lachlan Ewen MacDonald, Laszlo A Jeni, and Simon Lucey. Flow supervision for deformable nerf. In IEEE Conf. Comput. Vis. Pattern Recog., pages 21128–21137, 2023a. 3 Guangcong Wang, Zhaoxi Chen, Chen Change Loy, and Ziwei ...
-
[13]
Sparsegs: Real-time 360 {\deg} sparse view synthesis using gaussian splatting
3 Haolin Xiong, Sairisheek Muttukuru, Rishi Upadhyay, Pradyumna Chari, and Achuta Kadambi. Sparsegs: Real-time 360 {\deg} sparse view synthesis using gaussian splatting. arXiv preprint arXiv:2312.00206,
-
[14]
Depth anything v2
9 Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. arXiv preprint arXiv:2406.09414,
-
[2021]
Midas v3.1 – a model zoo for robust monocular relative depth estimation
5 Reiner Birkl, Diana Wofk, and Matthias Müller. Midas v3.1 – a model zoo for robust monocular relative depth estimation. arXiv preprint arXiv:2307.14460,
-
[2022]
Mip-splatting: Alias-free 3d gaussian splatting
3 13 Published as a conference paper at ICLR 2025 Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splatting. In IEEE Conf. Comput. Vis. Pattern Recog., pages 19447–19456, 2024a. 1 Zehao Yu, Torsten Sattler, and And...
2025 arXiv
-
[2023]
Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction
9 Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang, Weijian Xie, Shangjin Zhai, Nan Wang, Haomin Liu, Hujun Bao, and Guofeng Zhang. Pgsr: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction. arXiv preprint arXiv:2406.06521,
-
[2024]
Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization
3 Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, and Lin Gu. Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization. arXiv preprint arXiv:2403.06912,
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.