InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis

Hao Yu; Jiawei Wang; Junbo Chen; Ruizhen Hu; Sida Peng; Tao Ni; Xiaowei Zhou; Xinyi Yang; Xin Zhan; Yongzhen Hu

InfiniSplat claims that single-image 3D view synthesis improves when Gaussians are anchored to depth-derived surface patches rather than fixed pixel centers, with a shared implicit decoder predicting their attributes.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 07:18 UTC pith:K2HAVUAX

load-bearing objection A solid empirical step for single-image feed-forward 3DGS, with the geometric mechanism less proven than the headlines suggest. the 3 major comments →

arxiv 2608.02437 v2 pith:K2HAVUAX submitted 2026-08-03 cs.CV

InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis

Jiawei Wang , Hao Yu , Yongzhen Hu , Xinyi Yang , Tao Ni , Xin Zhan , Junbo Chen , Xiaowei Zhou

show 2 more authors

Ruizhen Hu Sida Peng

This is my paper

classification cs.CV

keywords 3D Gaussian splattingsingle-image novel view synthesisfeed-forward 3D reconstructionsurface-aligned representationgeometry-guided samplingimplicit decodinglarge-baseline renderingdepth-guided Gaussian placement

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Single-image feed-forward 3D Gaussian Splatting methods currently generate Gaussians at fixed image-grid positions, which the paper identifies as a key bottleneck: those primitives behave like locally expanded splats and fall apart under large baseline changes. InfiniSplat replaces this pixel-aligned representation with a surface-aligned one. A frozen monocular depth model provides a geometric scaffold; surface-area-weighted sampling places Gaussian supports along depth-induced surface structure, and a shared query-conditioned implicit decoder predicts bounded attribute updates at these flexible supports. Across cross-dataset evaluations, the paper argues that this design produces more coherent surfaces, fewer cracks, and better large-baseline rendering than existing feed-forward baselines, and that the same framework also benefits from sparse depth/LiDAR conditioning. If correct, the work shows that geometric grounding of Gaussian placement is a viable path for single-image 3D scene generation without per-scene optimization.

Core claim

InfiniSplat claims that pixel-aligned Gaussian generation — primitives tied to fixed image-grid positions — is the key bottleneck in single-image view synthesis. It proposes a surface-aligned representation: geometry-guided sampling places Gaussian supports by area-weighting local back-projected depth triangles, and a shared implicit decoder predicts bounded attribute updates at those supports. Decoupling placement and prediction from the pixel grid lets Gaussians assemble into coherent surfaces, reducing cracks, holes, and drift under large viewpoint changes, in one feed-forward pass. The paper supports this with cross-dataset metrics, ablations, and normal-map visualizations.

What carries the argument

The mechanism is the pairing of geometry-guided support sampling with query-conditioned implicit decoding. Sampling back-projects a monocular depth map, discards triangles spanning depth discontinuities, and allocates support points proportional to 3D triangle area, so Gaussians concentrate on slanted or extended surfaces. The decoder is a shared function of support coordinates: it bilinearly samples semantic and texture features, fuses them with a gate, and predicts bounded residual updates to position, scale, rotation, color, and opacity on a base scaffold. Each stage requires the other: irregular supports demand a query-based decoder, and the decoder lacks surface awareness without the ge

Load-bearing premise

The paper's central premise is that the frozen monocular depth model's predicted depth is accurate enough that geometry-guided sampling places supports on true scene surfaces; the paper's own limitation section acknowledges that on reflective, transparent, thin, and textureless regions this scaffold fails, so the surface-aligned advantage degrades there.

What would settle it

Use a benchmark with ground-truth depth and compare the full model against the same model whose supports are sampled uniformly at random at equal density, keeping the decoder identical: if the geometry-guided supports do not beat random supports on large-baseline metrics, surface alignment is not the driver of the reported gains. A complementary check: corrupt the depth input (inverted or heavily blurred) at inference and measure whether large-baseline coherence degrades smoothly; the paper's claim predicts graceful decline on ordinary scenes but severe failure where depth is wrong.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

If the surface-aligned representation holds, single-image 3DGS can generate renderable scenes that remain stable at substantially larger camera baselines than nearby-view splatting.
Because the decoder is query-conditioned over arbitrary supports, the same trained framework can accept different support densities and arrangements, including sparse depth prompts, without retraining the core decoder.
Area-weighted sampling naturally allocates more Gaussians to slanted or geometrically demanding surface patches, showing that adaptive resource allocation is compatible with feed-forward prediction.
Cross-dataset zero-shot generalization from synthetic indoor scenes to real-world captures becomes achievable in a single forward pass, supporting real-time interactive browsing and AR/VR viewing.
LiDAR-conditioned variants show that more reliable geometric scaffolding further improves the representation, indicating the framework's gains are tied to the geometric scaffold rather than a specific depth source.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

A natural extension is uncertainty-aware support sampling: instead of treating monocular depth as a single scaffold, sample supports from a depth distribution, which could mitigate the claimed failure on reflective and textureless regions.
The separation of 'where to place primitives' from 'what attributes to give them' may generalize beyond 3DGS to other primitive-based scene representations such as point clouds or surfels, and could be combined with diffusion-based priors as a post-refinement step.
Because the decoder is a shared function of coordinates, the framework could be trained on varied camera intrinsics and resolutions by re-scaling query coordinates, potentially improving resolution robustness — a hypothesis the paper does not test.
If the surface-alignment claim is correct, the benefit should be measurable not only in image metrics but in geometric metrics such as normal consistency or chamfer distance on the predicted Gaussian centers; the paper's normal-map visualizations support this, but a quantitative geometry metric would make it testable.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A solid empirical step for single-image feed-forward 3DGS, with the geometric mechanism less proven than the headlines suggest.

read the letter

Worth a read if you work on feed-forward single-image 3DGS. The paper reports a clean empirical SOTA: replacing fixed pixel-center Gaussian prediction with depth-guided off-grid supports plus a query-conditioned implicit decoder gives consistent PSNR/SSIM/LPIPS gains over SHARP, Flash3D, and LagerNVS on four cross-dataset benchmarks. The controlled baselines (Flash3D-DepthPro, SHARP with DepthPro) and component ablations are done honestly; the qualitative normal-map comparisons are genuinely informative. The limitation section is upfront about depth-prior failures. That is a real contribution, though no code or data are released.

The main soft spot is exactly what the stress-test note says: the paper's signature “geometry-guided” mechanism is not isolated. The w/o Geometry-guided Sampling ablation swaps in pixel-aligned supports, which simultaneously removes continuous off-grid placement and removes depth-area allocation. So the 0.6–0.7 PSNR drop could come just from losing continuous support coordinates, with the implicit decoder doing the work. The support-budget ablation makes this concrete: going from 0.5M to 1.5M supports is worth about 3.5 PSNR on ScanNet++, far larger than removing geometry-guided sampling. A depth-agnostic continuous sampler (jittered grid or uniform random supports at matched budget) is the missing control. Without it, “surface-aligned” is a plausible story but not proven. This does not sink the empirical result—the full system wins—but it weakens the mechanistic claim.

Other soft spots: the “large-baseline” headline is not backed by per-baseline breakdowns; Table 1 averages over four baseline buckets, and the papers describes the buckets but does not split results. No error bars or significance tests, so top-line differences may partly be noise. The LiDAR variant uses InfiniDepth-Metric, a same-group prior, and the paper does not state whether that depth model was trained on any evaluation dataset; that comparison should be checked. These are fixable in review.

Overall this deserves a serious referee. The core empirical result is meaningful for the subfield; the mechanism needs one more ablation and preferably code release. I would bring it to reading group and cite it as a baseline, but I would not cite the geometric explanation as established.

Referee Report

3 major / 5 minor

Summary. The paper introduces InfiniSplat, a feed-forward single-image 3D Gaussian Splatting method. Instead of predicting Gaussians at fixed pixel-grid locations, InfiniSplat first samples 2D supports using depth-induced local surface areas (geometry-guided sampling), then predicts Gaussian attributes at these supports with a query-conditioned implicit decoder. The authors claim that this moves single-image 3DGS from pixel-aligned splat expansion to a surface-aligned representation, reducing cracks and improving large-baseline novel-view synthesis. They train on Hypersim and evaluate zero-shot on ETH3D, ScanNet++, Tanks-and-Temples, and DL3DV, reporting state-of-the-art PSNR/SSIM/LPIPS over feed-forward RGB-only baselines and a LiDAR-conditioned variant over ADGaussian. Ablations support the contribution of learned updates, dual-branch features, regularization, geometry-guided sampling, and implicit decoding, and a support-budget table shows diminishing returns beyond 1.5M supports.

Significance. If the empirical results hold, the paper makes a meaningful advance: shifting Gaussian placement away from the regular image lattice is a natural direction, and the cross-dataset zero-shot gains over SHARP and Flash3D are substantial (averaging +1.92 PSNR over SHARP, with larger gains on DL3DV). The paper has several strengths: the comparison controls the depth backend by including a Flash3D-DepthPro variant; the LiDAR-conditioned variant isolates whether gains depend on the depth source; the component ablations in Table 2 are systematic; Table 3 studies support budget; and Table 4 characterizes robustness to prompt-depth noise. The limitations section is candid about failures on reflective, transparent, thin, and textureless regions. The central mechanism, however, is not yet cleanly isolated from a simpler confound: continuous off-grid support placement alone may explain part or most of the benefit attributed to depth-guided surface alignment. The large-baseline claim also currently rests on a dataset-averaged table rather than per-baseline evidence. These issues are addressable and do not invalidate the empirical contribution, but they need to be resolved before the paper's sign

major comments (3)

[Section 4.1 / Table 1] The paper's headline claim is robustness under large-baseline viewpoint changes, and Section 4.1 states that the 512 pairs per dataset are constructed by targeting 128 pairs in each of four baseline ranges [0,0.5), [0.5,1), [1,2), and [2,∞) meters. Yet Table 1 reports only dataset-level averages. Averages over these bins can be dominated by easy nearby views or by the distribution of actual baselines within each bin. To support the 'large-baseline' claim, please report per-baseline-bin PSNR/SSIM/LPIPS for at least InfiniSplat-RGB and SHARP (and ideally all baselines). If the advantage concentrates in the [0,0.5) bin, the surface-alignment story would be weakened; if it persists or grows in the [1,2) and [2,∞) bins, it would directly support the abstract.
[Section 3.2 / Table 2 / Table 3] The 'w/o Geometry-guided Sampling' ablation in Table 2 replaces geometry-guided sampled supports with pixel-aligned supports while keeping the implicit decoder. This varies two factors at once: (i) support coordinates lose continuous, off-grid placement, and (ii) the depth-area allocation is removed. The observed drop (from 22.240 to 21.576 PSNR on ScanNet++) may therefore be caused entirely by the loss of coordinate flexibility rather than by geometric guidance. This is load-bearing for the central mechanistic claim that depth-induced area-weighted sampling creates surface-aligned Gaussians. The confound is compounded by Table 3: increasing the support budget from 1.0M to 1.5M yields +1.06 PSNR on ScanNet++, a larger gain than removing geometry-guided sampling. The authors should add a control with a depth-agnostic continuous sampler at the same support budget (e.g., stratified jittered
[Section 3.2 / Section 5] The geometry-guided sampling pipeline relies on the quality of a frozen monocular depth scaffold, and Section 5 acknowledges that the scaffold fails on reflective, transparent, thin, and textureless regions. This is a structural limitation, not a flaw in the experiments, and the authors are transparent about it. However, the paper does not quantify how often such failures occur in the evaluation datasets or how much the reported averages are affected. Since the central claim is that surface alignment improves large-baseline rendering, a short analysis of failure-mode frequency (e.g., counting cases where the source depth is clearly wrong on planar regions, or reporting a subset of pairs with strong depth discontinuities) would help readers calibrate the strength of the claim. I view this as a needed clarification rather than a reason to reject.

minor comments (5)

[Section 4.1] The evaluation protocol says pairs are selected with 'a fixed random seed' and 'a scene-level sampling cap', but the seed value and cap are not given. Please state these for reproducibility.
[Section 4.2 / Table 1] The LPIPS improvement on Tanks-and-Temples is only 0.004 relative to SHARP (0.310 vs 0.306), whereas PSNR/SSIM gains are clearer. The text acknowledges this, but a brief explanation of why (e.g., scene scale, texture distribution) would be useful.
[Table 3] The support-budget table reports inference and rendering times, but the paper does not state the hardware and software environment (GPU, PyTorch version, rasterizer). Please add this information so the runtimes are interpretable.
[Section 3.3] In the equation for the fused descriptor h_i, the gating notation is clear, but the projection P('f_cnn_i') is not defined. Please state the projection layer (e.g., 1x1 conv or linear) and its output dimension.
[Figures 3 and 4] The qualitative figures are compelling, but the color/normal color maps are not always accompanied by a legend or a description of the normalization. Adding a short caption note would make the normal visualizations easier to interpret.

Circularity Check

0 steps flagged

No significant circularity: the model is learned from rendered supervision on Hypersim and evaluated on held-out datasets, so the central claim is empirical rather than definitional.

full rationale

InfiniSplat's central claim is that geometry-guided support sampling plus query-conditioned implicit decoding produces surface-aligned Gaussians that improve large-baseline novel-view synthesis. This is not circular: Gaussian parameters are predicted by a trainable MLP, supervised by rendering losses on Hypersim target views, and evaluated zero-shot on ETH3D, ScanNet++, Tanks-and-Temples, and DL3DV. The geometry scaffold in Eq. (3), X(p) = Π^{-1}(p, D(p); K̂), is obtained from a frozen pretrained depth model (DepthPro for RGB, InfiniDepth-Metric for LiDAR), not from the evaluation target, and the decoder predicts bounded residual updates rather than reproducing a fitted quantity. The ablation 'w/o Geometry-guided Sampling' may conflate depth-guided placement with continuous off-grid coordinates, and the support-budget ablation shows coverage effects, but these are attribution or experimental-design concerns, not instances where a prediction reduces to its input by construction. The only notable self-citations (InfiniDepth for the implicit-decoder inspiration and the LiDAR variant's depth backbone) are not load-bearing for the RGB results, and no uniqueness theorem or prior ansatz is imported to force the method's choices. Section 5's acknowledged limitations about depth-prior failures weaken robustness but do not make the derivation tautological.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The method rests on learned/supervised modules and hand-chosen hyperparameters; no new physical entities are introduced. The core novelty is representational (where Gaussians are sampled) rather than a new theoretical entity.

free parameters (6)

λ_xy, λ_z = not reported
Position update ranges in Eq. (5); chosen by hand; control how far decoded Gaussians may move from the depth scaffold.
λ_scale, λ_smooth = 0.1, 0.1
Weights of Gaussian regularization losses (Section 3.4).
ℓ_min, ℓ_max = [-8, -3]
Log-scale bounds for scale regularization (Section 3.4).
γ = 10
Weight of Gram-matrix term in perceptual loss (Section 3.4).
support budget = 1.5M
Number of sampled Gaussian supports per image, chosen as a quality/speed trade-off (Section 4.4, Table 3).
training schedule = 100K steps, batch 8, LR 5e-5
Standard practice; no sensitivity analysis reported.

axioms (4)

domain assumption Predicted depth D and intrinsics K̂ from the frozen model are accurate enough to define surface structure
Geometry-guided sampling back-projects pixels using D and K̂ (Section 3.2); the paper's own limitations (Section 5) list failure cases.
domain assumption Local triangle areas from neighboring pixels approximate the local 3D surface area for sampling density
Area-weighted sampling uses A_t = 1/2 |(X(p_b)-X(p_a)) × (X(p_c)-X(p_a))| (Section 3.2, Eq. 4).
domain assumption Training on Hypersim and supervising target-view rendering transfers to the evaluation datasets
The empirical claim rests on zero-shot generalization; no theoretical guarantee.
domain assumption DINOv3 and CNN features provide sufficient conditioning for Gaussian attributes
Ablations show both matter; the assumption is that the feature set is sufficient.

pith-pipeline@v1.3.0-daily-deepseek · 18831 in / 14466 out tokens · 141678 ms · 2026-08-04T07:18:24.340539+00:00 · methodology

0 comments

Cite this review

Pith. "Pith review of InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis." pith.science (2026). https://pith.science/paper/K2HAVUAX

@misc{pith2026260802437,
  author       = {Pith},
  title        = {Pith review of: InfiniSplat: Implicit Gaussian Decoding for Large-Baseline Monocular View Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K2HAVUAX}},
  note         = {Machine review of arXiv:2608.02437}
}

Share X Bluesky LinkedIn Reddit HN

read the original abstract

Single-image feed-forward 3D Gaussian Splatting (3DGS) aims to directly generate a renderable 3D scene representation from one input image, avoiding the cost of multi-view capture and per-scene optimization. However, existing methods are often constrained by a pixel-aligned representation, where Gaussians are predicted from fixed image-grid locations. Such pixel-aligned primitives can produce promising nearby-view renderings, but they remain weakly coupled to underlying scene surfaces and struggle to preserve coherent structures under large viewpoint shifts. We present InfiniSplat, a feed-forward single-image 3DGS framework that moves from a pixel-aligned representation toward a surface-aligned representation. InfiniSplat constructs this representation by first using geometry-guided sampling to place 2D supports according to depth-induced local surface structure, and then applying a query-conditioned implicit decoder to predict Gaussian attributes from the image features queried at these supports. By grounding support locations in geometry while decoupling Gaussian prediction from fixed pixel centers, InfiniSplat produces Gaussian layouts that better follow scene surfaces and reduce scattered primitives caused by grid discretization. Across multiple cross-dataset NVS evaluations, InfiniSplat achieves state-of-the-art performance compared with single-image feed-forward baselines, and demonstrates zero-shot generalization from Hypersim indoor synthetic training to complex open-world scenes. Project page: https://zju3dv.github.io/InfiniSplat.

Figures

Figures reproduced from arXiv: 2608.02437 by Hao Yu, Jiawei Wang, Junbo Chen, Ruizhen Hu, Sida Peng, Tao Ni, Xiaowei Zhou, Xinyi Yang, Xin Zhan, Yongzhen Hu.