Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hybrid invertible distortion field lets uncalibrated fisheye images reconstruct scenes in as few as 10 captures.

desk verdict A solid, useful empirics-first paper on self-calibrating 3DGS for fisheye/wide-FOV, but the invertibility claim is oversold and the missing code/error bars keep it from being fully convincing. read the letter →

arxiv 2502.09563 v2 pith:EDNXQJV5 submitted 2025-02-13 cs.CV cs.GR

Youming Deng , Wenqi Xian , Guandao Yang , Leonidas Guibas , Gordon Wetzstein , Steve Marschner , Paul Debevec This is my paper
classification cs.CVcs.GR
keywords self-calibrationlensdistortionGaussiansplattingfisheyecamerawidefieldofviewinvertibleresidualnetworkcubemaprenderingnovelsynthesis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that 3D Gaussian splatting can be made self-calibrating for large-field-of-view cameras by modeling lens distortion with a hybrid neural field and rendering through a cubemap. The central claim is that this combination jointly optimizes distortion, intrinsics, extrinsics, and 3D Gaussians against raw pixels, so fisheye and wide-angle images can be used in full, including peripheral regions that conventional pipelines crop or stretch. If the claim holds, scene reconstruction no longer needs pre-calibration or perspective resampling, and a handful of wide-angle images can replace many narrow ones: the paper reports reconstructions from as few as 10 captures whose quality exceeds a 200-image perspective baseline on synthetic scenes. A sympathetic reader would take this as evidence that the limiting factor for wide-angle reconstruction is not the splatting representation but the lens model and projection surface.

What carries the argument

The engine of the method is the hybrid distortion field: an invertible residual network, a stack of residual blocks whose learned maps are constrained to be contractions so the whole map is bijective, predicts displacement vectors on a sparse grid of control points, and bilinear interpolation turns those vectors into a dense, smooth distortion field. Because the network runs only on the sparse grid, its cost does not grow with the number of Gaussians; the bilinear lookup is the per-Gaussian operation. The second piece is cubemap rendering, which projects the scene onto several 90-degree faces so peripheral pixels retain near-uniform sampling density instead of the tan-like stretching of a single perspective plane; Gaussians are ordered by distance from the camera center so the ordering stays consistent across faces.

What would settle it

Take a 180-degree fisheye lens with strong entrance-pupil shift or a sharp distortion change near the sensor edge, calibrate it densely with a checkerboard, run this pipeline on the same views, and compare the learned distortion field against the checkerboard field in the periphery: if peripheral reprojection errors exceed the center errors while the center fits well, the smooth-bijective sparse-grid assumption is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that the two conventional components of wide-angle reconstruction are both replaceable in a way that is compatible with fast Gaussian-splatting rasterization. Parametric distortion models, which fit real fisheye lenses poorly in the periphery, are replaced by a hybrid field: an invertible residual network predicts displacement vectors on a sparse grid of control points, and bilinear interpolation produces a smooth, dense, bijective distortion map whose optimization cost does not depend on the number of Gaussians. Single-plane perspective projection, which stretches pixels ever more severely as field of view approaches 180 degrees, is replaced by cubemap rendering that keeps pixel density approximately uniform across the image and applies the same distortion field during resampling. With these two pieces, the paper shows that distortion, intrinsics, extrinsics, and 3D Gaussians can be jointly optimized directly against raw pixels, yielding higher-quality reconstructions from fewer, wider captures than pipelines that first undistort and crop.

Load-bearing premise

The method assumes real lens distortion is a smooth, reversible (one-to-one) displacement that a coarse grid of corrected positions can represent; sharp or non-reversible distortions near the edge would break the peripheral accuracy that is the paper's main selling point.

Editorial extensions

If this is right

  • Uncalibrated large-FOV captures become usable directly: the method reports higher reconstruction quality than parametric-model baselines on both real fisheye scenes and synthetic 180-degree scenes.
  • Capture efficiency increases dramatically: as few as 10 wide-angle images can outperform 200 perspective images of the same scenes in the reported synthetic evaluation.
  • The full raw image is usable: peripheral regions that baseline pipelines crop or stretch are modeled without severe distortion, widening the reconstructed coverage.
  • Distortion, intrinsics, extrinsics, and scene geometry can be optimized jointly from noisy initial estimates, so pre-calibration with calibration targets is no longer required.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the authors leave implicit: the hybrid field is a general bijective-distortion prior, so the same code path should calibrate anamorphic or non-radially symmetric lenses without architectural changes; only the tested demonstrations are radial and tangential.
  • Because the network is evaluated on a sparse grid, the method's cost scales with calibration-grid resolution rather than scene size, suggesting a practical extension to online or video-rate self-calibration where a few frames would refine a running estimate of the lens.
  • The remaining artifacts the paper attributes to cubemap-face boundaries point to a concrete next step, projecting 3D covariance onto a sphere instead of per-face planes, which would likely remove the need for special sorting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents Self-Calibrating Gaussian Splatting, a 3D Gaussian Splatting extension for large field-of-view and fisheye imagery. It replaces parametric lens models with a hybrid distortion field: an invertible residual network evaluated on a sparse control-point grid, followed by bilinear interpolation to produce a continuous displacement field. Rendering is performed with cubemap projection to reduce peripheral stretching, and the system jointly optimizes extrinsics, intrinsics, distortion, and the 3D Gaussians against raw distorted pixels. Experiments on FisheyeNeRF, real 150- and 180-degree captures, and Mitsuba synthetic scenes report consistent improvement over 3DGS, Fisheye-GS, ADOP-GS, and CamP, with ablations for the hybrid field and cubemap. The central efficiency claim is that 10 to 100 fisheye images suffice where 200 perspective views are used for the baseline.

Significance. If the results hold, this is a practically useful contribution: it would allow 3DGS to be applied directly to raw wide-angle or fisheye footage without pre-calibration, improving peripheral fidelity relative to parametric distortion models and reducing capture counts. The paper has genuine strengths: evaluation on a public benchmark (FisheyeNeRF), comparisons against four independent baselines, ablations isolating the hybrid field and cubemap (Tables 3, 4, and 7), a frozen-initialization control demonstrating that the learned distortion field is not merely reproducing COLMAP's output, and a detailed supplement. However, the claimed invertibility of the hybrid distortion field is not established, and the headline few-capture comparison is confounded; both points need to be addressed before the significance claim is fully supported.

major comments (3)
  1. [Sec. 3.2, Eq. (7)] The hybrid distortion field is not guaranteed to be invertible, despite the paper's motivation. The iResNet R_theta is invertible, but D_theta(x) = x + interp(x, R_theta(P_c) - P_c) applies bilinear interpolation to displacements at sparse control points; when the grid spacing is coarse relative to the curvature of the displacement field, the interpolated field can have local Lipschitz constant greater than 1, producing folds where the Jacobian determinant is non-positive. Since the cubemap resampling in Sec. 3.3 and the physical interpretation of the distortion map depend on a one-to-one mapping between raw pixels and rays, a folded D_theta would let the photometric loss minimize a non-physical distortion model. The paper never reports Jacobian-determinant statistics, inverse-consistency errors, or a grid-resolution condition that guarantees bijectivity. Please add such diagnostics or modify the architecture to preserve invertibility, for example by constraining the Lipschitz constant of the interpolated displacement field or using an invertible interpolation scheme.
  2. [Sec. 4.3, Table 2] The 'few captures' comparison is confounded: the proposed method uses 10 to 100 fisheye views while the baseline uses 200 perspective views, so the two settings differ in total pixel count, resolution, and scene coverage rather than only in capture count. The claim that the method outperforms the baseline even with far fewer input views therefore conflates the advantage of a large FOV with reconstruction quality. The paper should report matched comparisons, such as a fisheye-capable baseline with the same pixel budget, per-scene coverage metrics, or effective per-pixel resolution. Supplementary Table 11 is a step in this direction, but it is not presented as a controlled test and still uses unmatched capture settings. Without this, the headline efficiency result is not fully substantiated.
  3. [Tables 1-12] All experiments appear to be single runs with no error bars or significance testing. Given the stochasticity of 3DGS training, the modest PSNR margins in some comparisons, and the sensitivity to control-grid resolution (Table 8 shows PSNR varying from 22.44 to 23.67 across grid resolutions) and per-scene learning rates (supplementary Sec. 6), the consistency of the reported gains cannot be assessed. Please report means and standard deviations over multiple seeds for the main FisheyeNeRF comparison and for the few-capture experiment.
minor comments (5)
  1. [Table 3 and Sec. 4.4] The text says that '-' indicates computationally infeasible cases, but the table itself uses '✗' and 'Out-of-Memory'; make the notation consistent.
  2. [Eq. (3)] The expression theta = arctan(r/1) is an unclear way to write the normalized fisheye angle; clarify the focal-length normalization used in the projection model.
  3. [Fig. 4 and Table 1] The baseline is referred to as both 'Adop-GS' and 'ADOP-GS' in different places; unify the naming.
  4. [Sec. 4.2] The paper says ADOP was re-implemented with an omnidirectional camera model, but the original ADOP already supports omnidirectional rendering; clarify what was changed in the re-implementation.
  5. [Reproducibility] The project page is referenced but no code release is indicated; providing the implementation would materially strengthen reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: hybrid distortion field is an optimized model, not a renamed input; results validated on external benchmarks.

full rationale

No circular step found. The central claim—that jointly optimizing poses, intrinsics, a hybrid distortion field, and cubemap rendering improves large-FOV 3D Gaussian reconstruction—is tested against external public datasets (FisheyeNeRF, NeRF-Synthetic) and independent baselines (3DGS, Fisheye-GS, ADOP-GS, CamP). The hybrid distortion field in Eq. 7 is a model choice fit by photometric loss; it is not defined in terms of the reconstruction quality metric, and held-out test views provide external falsification. The frozen-COLMAP ablation (Tab. 7) shows the optimized field improves over its own initialization, so the learned distortion is not merely reproducing the COLMAP parameters used to initialize it. The only relevant self-citation is NeuroLens [61] (co-author Xian), used for the iResNet architecture; the invertible-block construction is also attributed to external work [8] and implemented via FrEIA, and no uniqueness claim from prior work is invoked to force the design. The invertibility caveat raised by the skeptic concerns whether interpolating an iResNet displacement field on a sparse grid preserves bijectivity—a soundness/robustness question, not a reduction of outputs to inputs—and therefore does not count as circularity under the stated rules.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The free parameters are architectural hyperparameters; the central claim also relies on domain assumptions about lens distortion being a smooth bijection and about the sufficiency of photometric self-calibration.

free parameters (3)
  • Control point grid resolution = 265x149 for FisheyeNeRF; 132x74 and 66x37 in ablations
    Hand-chosen architectural hyperparameter; affects fidelity of distortion field at edges (Fig. 13), with PSNR dropping from 23.67 to 22.44 at lowest resolution (supplementary Table 8).
  • iResNet learning rate schedule = 1e-5 to 1e-7 for FisheyeNeRF; 1e-8 for real-world captures; 1e-7 for object-centric synthetic scenes
    Per-dataset hand-tuning of the distortion-network optimizer; not a scientific parameter but part of the method's recipe.
  • Number of iResNet blocks = L=5
    Fixed architecture choice; no sensitivity study reported.
assumptions (4)
  • domain assumption Lens distortion is a smooth, bijective mapping (a diffeomorphism) from ideal to observed pixel coordinates
    Stated in Sec. 3.2 as justification for iResNet with Lipschitz bound < 1; entrance pupil shift, excluded by the paper, is an example of an effect that can violate this assumption (Sec. 7).
  • domain assumption Photometric loss against raw distorted pixels is sufficient to recover extrinsics, intrinsics, and distortion jointly (self-calibration)
    Core premise of the whole pipeline; tested under perturbed initialization in supplementary Sec. 2.3, but no proof of global convergence.
  • domain assumption COLMAP provides a sufficiently accurate initialization for poses and distortion so that joint optimization is well-behaved
    Used in all real data experiments (Sec. 4.2, supplementary Sec. 3); the method is not tested from scratch or random initialization.
  • standard math Fixed-point iteration inverts the iResNet reliably when the Lipschitz constant is < 1
    Background from Behrmann et al. (ICML 2019), used in Sec. 3.2; standard but essential for the invertibility of the distortion field.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction." pith.science (2026). https://pith.science/paper/EDNXQJV5

@misc{pith2026250209563,
  author       = {Pith},
  title        = {Pith review of: Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EDNXQJV5}},
  note         = {Machine review of arXiv:2502.09563}
}
read the original abstract

In this paper, we present a self-calibrating framework that jointly optimizes camera parameters, lens distortion and 3D Gaussian representations, enabling accurate and efficient scene reconstruction. In particular, our technique enables high-quality scene reconstruction from Large field-of-view (FOV) imagery taken with wide-angle lenses, allowing the scene to be modeled from a smaller number of images. Our approach introduces a novel method for modeling complex lens distortions using a hybrid network that combines invertible residual networks with explicit grids. This design effectively regularizes the optimization process, achieving greater accuracy than conventional camera models. Additionally, we propose a cubemap-based resampling strategy to support large FOV images without sacrificing resolution or introducing distortion artifacts. Our method is compatible with the fast rasterization of Gaussian Splatting, adaptable to a wide variety of camera lens distortion, and demonstrates state-of-the-art performance on both synthetic and real-world datasets.

Figures

Figures reproduced from arXiv: 2502.09563 by the authors.

Figure 1
Figure 1. We introduce Self-Calibrating Gaussian Splatting, a differentiable rasterization pipeline with a hybrid lens distortion field that [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Conventional Paradigm vs. Our Method. (a) Con￾ventional approaches require reprojecting the image into perspec￾tive views compatible with 3DGS rasterization. As the field of view increases, pixel stretching becomes progressively severe, sig￾nificantly compromising the quality of the reconstruction. (b) In contrast, our cubemap resampling strategy maintains a consistent pixel density across the entire field of view. … view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (18 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparisons with Baselines on the FisheyeNeRF Dataset [36]. The images show comparisons across different scenes using two baselines (e.g., ADOP-GS [57] and Fisheye-GS [47]) and our method. PSNRs are computed for each patch. completely differentiable and cap…
Figure 5
Figure 5. Figure 5: Qualitative Comparisons with Fisheye-GS [47]. To validate our hybrid distortion modeling, we further compare our method with Fisheye-GS [47] on larger FOV scenes, including real-world captures using 150◦ cameras (a–c) and simulations using a 180◦ camera (d–f) in Mitsub…
Figure 6
Figure 6. Figure 6: Qualitative Results of Radial and Tangential Distor￾tion. We apply synthetic distortion to real-world images and (a) show the distorted rendering, comparing it with the reference im￾ages in (b). After training, we use the distortion field in (c) to reproject the image …
Figure 7
Figure 7. Figure 7: Qualitative Comparison of Reconstruction Coverage. We evaluate our method on real-world scenes where (a) 3DGS [38] relies on an SfM method like COLMAP [59], which crops the highly distorted periphery when the FOV is large. (b) Our method recovers a wider region with fe…
Figure 8
Figure 8. Figure 8: Camera Parameters Optimization. We initialize noisy cameras from COLMAP [59]. (a) Modeling fisheye distortion without optimizing camera parameters, while (b) jointly optimiz￾ing both in a self-calibration manner. Our full model can recover accurate lens distortion and …
Figure 9
Figure 9. Figure 9: Visual comparison of (a) the initial perturbed ( [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Qualitative Comparison on Perturbed FisheyeNeRF dataset [36]. We show the novel view rendering with perturbed camera poses. We disable and enable camera optimization to illustrate the capability of our pipeline on recovering inaccurate poses along with distortion mode…
Figure 11
Figure 11. Figure 11: We carry qualitative comparison with CamP at noise level 0.15. Each scene show CamP, our method, and the ground truth, from [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Comparison on Invertible ResNet Optimization. This version shows the comparison transposed, grouping by scenes instead of optimization stages. the estimated camera poses in the training views after ac￾counting for a global rigid transformation. Second, we mea￾sure the…
Figure 13
Figure 13. Figure 13: Resolution of Control Grid. When the resolution of the control grid is decreased, the central region retains decent quality due to minimal distortion. However, as highlighted by the red and blue boxes in the corners of the image, a sparse control grid for the hybrid f…
Figure 14
Figure 14. Figure 14: Single Planar Projection with Hybrid Field. Our hybrid field can be directly applied to a single plane during raster￾ization. However, the limitation of single planar projection is that it cannot cover the full FOV of the raw images, leading to partial loss of informa…
Figure 15
Figure 15. Figure 15: Qualitative Evaluation of Reconstruction with Varying Numbers of Large FOV Inputs. Our method achieves high-quality reconstruction even with a relatively small number of input images, thanks to our hybrid distortion representation and cubemap resampling. Room1 Kitchen…
Figure 16
Figure 16. Figure 16: Evaluation of Perspective Rendering. After recon￾struction, our method can render perspective views with arbitrary FOV. We compare the perspective renderings produced by our method with those rendered from small-FOV reconstructions us￾ing 3DGS [38]. Fisheye-GS [47], a…
Figure 17
Figure 17. Figure 17: Reconstruction from 180◦ FOV Fisheye Captures. Using a Canon fisheye camera, we capture the scene and reconstruct the office with our method. Both perspective and fisheye views are rendered to demonstrate the quality of our reconstruction. 5.6. Adaptability to Differe…
Figure 18
Figure 18. Figure 18: Large FOV Reconstruction from a Customized Fisheye Rig. We reconstruct a backyard from images captured using a fisheye rig. Our method achieves accurate geometric corrections, such as straightening the lines on the wall and the edges of the house. Method PSNR SSIM LPI…
Figure 19
Figure 19. Figure 19: Radial and Perspective Rendering. We evaluate our method on a synthetic radial distortion dataset. Our approach successfully recovers slight radial distortion during reconstruction and enables perspective rendering upon completion of training [PITH_FULL_IMAGE:figures…
Figure 20
Figure 20. Figure 20: Distortion Error Map. We visualize the error map between the predicted distortion and the ground truth distortion from Mitsuba synthetic scenes. the test set remain largely unchanged, applying the MCMC technique reduces visual floaters in novel viewpoints. For high-re…
Figure 21
Figure 21. Figure 21: Fisheye and Perspective Rendering. After optimization, our method allows rendering in either fisheye or perspective views. Perspective rendering can be achieved by simply removing the hybrid field. is negligible for distant scenes, it can cause splat misalign￾ments in…

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DirectFisheye-GS: Enabling Native Fisheye Input in Gaussian Splatting with Cross-View Joint Optimization

    cs.CV 2026-04 conditional novelty 5.5 of 10

    Native fisheye projection inside 3DGS plus feature-overlap cross-view joint optimization matches or beats prior fisheye and pinhole Gaussian methods on public datasets.

  2. Splat-Based Metal Artifact Reduction in Cone-Beam CT via Compact Attenuation Modeling

    cs.CV 2026-08 conditional novelty 5.0 of 10

    A Gaussian splatting CBCT method with a compact quadratic Bezier material model reduces metal artifacts about 3 to 5 times faster than neural field baselines while better preserving fine structure.

  3. Revisiting Pose Sensitivity in Splat-based Computed Tomography under Sparse-view Reconstruction

    cs.CV 2026-08 conditional novelty 5.0 of 10

    Splat-based CT artifacts under sparse views are traced to pose inaccuracy, and a joint pose-volume refinement substantially improves reconstruction quality.

Reference graph

Works this paper leans on

83 extracted references · 75 canonical work pages · cited by 3 Pith papers

  1. [1]

    https://lensfun.github.io/

    Lensfun. https://lensfun.github.io/. 6, 7, 19

  2. [2]

    Automatic lens distortion correction using one-parameter division models

    Miguel Alem ´an-Flores, Luis Alvarez, Luis Gomez, and Daniel Santana-Cedr´es. Automatic lens distortion correction using one-parameter division models. Image Processing On Line, 2014. 3

    2014

  3. [3]

    Framework for Easily Invertible Architectures (FrEIA), 2018-2022

    Lynton Ardizzone, Till Bungert, Felix Draxler, Ullrich K¨othe, Jakob Kruse, Robert Schmier, and Peter Sorren- son. Framework for Easily Invertible Architectures (FrEIA), 2018-2022. 22

    2018

  4. [4]

    360-gs: Layout-guided panoramic gaussian splatting for indoor roaming

    Jiayang Bai, Letian Huang, Jie Guo, Wen Gong, Yuanqi Li, and Yanwen Guo. 360-gs: Layout-guided panoramic gaussian splatting for indoor roaming. arXiv preprint arXiv:2402.00763, 2024. 3

    arXiv 2024

  5. [5]

    DSSIM: a structural similarity index for floating-point data

    Allison H Baker, Alexander Pinard, and Dorit M Hammer- ling. Dssim: a structural similarity index for floating-point data. arXiv preprint arXiv:2202.02616, 2022. 4

    work page Pith review arXiv 2022

  6. [6]

    Geometric properties of central catadioptric line images and their application in calibration

    Jo ˜ao Pedro Barreto and Helder Araujo. Geometric properties of central catadioptric line images and their application in calibration. IEEE TPAMI, 2005. 3

    work page 2005

  7. [7]

    Zip-nerf: Anti-aliased grid- based neural radiance fields

    Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti-aliased grid- based neural radiance fields. In ICCV, 2023. 16

    work page 2023

  8. [8]

    Invertible residual net- works

    Jens Behrmann, Will Grathwohl, Ricky TQ Chen, David Du- venaud, and J ¨orn-Henrik Jacobsen. Invertible residual net- works. In ICML, 2019. 2, 4

    work page 2019

Show all 83 references
  1. [9]

    Rendering resources, 2016

    Benedikt Bitterli. Rendering resources, 2016. https://benedikt-bitterli.me/resources/. 6, 19

    2016

  2. [10]

    G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000. 2, 6

    2000

  3. [11]

    Decentering distortion of lenses

    Duane Brown. Decentering distortion of lenses. Photogram- metric engineering, 1996. 3

    1996

  4. [12]

    New efficient solution to the absolute pose problem for camera with unknown focal length and radial distortion

    Martin Bujnak, Zuzana Kukelova, and Tomas Pajdla. New efficient solution to the absolute pose problem for camera with unknown focal length and radial distortion. In ACCV,

  5. [13]

    Non-parametric structure-based calibration of radially sym- metric cameras

    Federico Camposeco, Torsten Sattler, and Marc Pollefeys. Non-parametric structure-based calibration of radially sym- metric cameras. In ICCV, 2015. 3

    2015

  6. [14]

    Optimizing content-preserving projections for wide-angle images

    Robert Carroll, Maneesh Agrawala, and Aseem Agarwala. Optimizing content-preserving projections for wide-angle images. ACM TOG, 2009. 3

    2009

  7. [15]

    Autocalibration via rank-constrained estimation of the absolute quadric

    Manmohan Chandraker, Sameer Agarwal, Fredrik Kahl, David Nist ´er, and David Kriegman. Autocalibration via rank-constrained estimation of the absolute quadric. In CVPR, 2007. 2

    2007

  8. [16]

    Globally optimal algorithms for strati- fied autocalibration

    Manmohan Chandraker, Sameer Agarwal, David Kriegman, and Serge Belongie. Globally optimal algorithms for strati- fied autocalibration. IJCV, 2010. 2

    2010

  9. [17]

    Cohen and Donald P

    Michael F. Cohen and Donald P. Greenberg. The hemi- cube: a radiosity solution for complex environments. InSIG- GRAPH, 1985. 2, 5

    1985

  10. [18]

    Decentred lens-systems

    Alexander Eugen Conrady. Decentred lens-systems. Monthly notices of the royal astronomical society, 1919. 3

    1919

  11. [19]

    Real-time dense map- ping for self-driving vehicles using fisheye cameras

    Zhaopeng Cui, Lionel Heng, Ye Chuan Yeo, Andreas Geiger, Marc Pollefeys, and Torsten Sattler. Real-time dense map- ping for self-driving vehicles using fisheye cameras. In ICRA, 2019. 1

    2019

  12. [20]

    Robust fusion of lidar and wide-angle camera data for autonomous mobile robots

    Varuna De Silva, Jamie Roche, and Ahmet Kondoz. Robust fusion of lidar and wide-angle camera data for autonomous mobile robots. Sensors, 2018. 1

    2018

  13. [21]

    Physics-based indirect illumination for inverse ren- dering

    Youming Deng, Xueting Li, Sifei Liu, and Ming-Hsuan Yang. Physics-based indirect illumination for inverse ren- dering. In 3DV, 2024. 3

    2024

  14. [22]

    Straight lines have to be straight

    Frederic Devernay and Olivier Faugeras. Straight lines have to be straight. Machine vision and applications, 2001. 3

    2001

  15. [23]

    A photometrically calibrated benchmark for monocular visual odometry

    Jakob Engel, Vladyslav Usenko, and Daniel Cremers. A photometrically calibrated benchmark for monocular visual odometry. arXiv preprint arXiv:1607.02555, 2016. 3

    2016 arXiv

  16. [24]

    Self-supervised camera self-calibration from video

    Jiading Fang, Igor Vasiljevic, Vitor Guizilini, Rares Ambrus, Greg Shakhnarovich, Adrien Gaidon, and Matthew R Walter. Self-supervised camera self-calibration from video. InICRA,

  17. [25]

    Environment mapping and other applications of world projections

    Ned Greene. Environment mapping and other applications of world projections. IEEE computer graphics and Applica- tions, 1986. 5

    1986

  18. [26]

    A general imaging model and a method for finding its parameters

    Michael D Grossberg and Shree K Nayar. A general imaging model and a method for finding its parameters. In ICCV,

  19. [27]

    The raxel imaging model and ray-based calibration

    Michael D Grossberg and Shree K Nayar. The raxel imaging model and ray-based calibration. IJCV, 2005. 3

    2005

  20. [28]

    Omni-nerf: neural radiance field from 360 im- age captures

    Kai Gu, Thomas Maugey, Sebastian Knorr, and Christine Guillemot. Omni-nerf: neural radiance field from 360 im- age captures. In ICME, 2022. 3

    2022

  21. [29]

    High-quality depth from uncalibrated small motion clip

    Hyowon Ha, Sunghoon Im, Jaesik Park, Hae-Gon Jeon, and In So Kweon. High-quality depth from uncalibrated small motion clip. In CVPR, 2016. 3

    2016

  22. [30]

    Parameter-free ra- dial distortion correction with center of distortion estimation

    Richard Hartley and Sing Bing Kang. Parameter-free ra- dial distortion correction with center of distortion estimation. IEEE TPAMI, 2007. 3

    2007

  23. [31]

    Multiple view ge- ometry in computer vision

    Richard Hartley and Andrew Zisserman. Multiple view ge- ometry in computer vision . Cambridge university press,

  24. [32]

    360roam: Real-time indoor roaming using geometry- aware 360 radiance fields

    Huajian Huang, Yingshu Chen, Tianjian Zhang, and Sai-Kit Yeung. 360roam: Real-time indoor roaming using geometry- aware 360 radiance fields. SIGGRAPH Asia, 2022. 3

    2022

  25. [33]

    Sc-omnigs: Self-calibrating omnidirectional gaussian splatting

    Huajian Huang, Yingshu Chen, Longwei Li, Hui Cheng, Tristan Braud, Yajie Zhao, and Sai-Kit Yeung. Sc-omnigs: Self-calibrating omnidirectional gaussian splatting. arXiv preprint arXiv:2502.04734, 2025. 3

    2025 arXiv

  26. [34]

    Open source computer vision library

    Itseez. Open source computer vision library. https:// github.com/itseez/opencv, 2015. 3

    2015

  27. [35]

    Mitsuba renderer, 2010

    Wenzel Jakob, S ´ebastien Speierer, Nicolas Roussel, Merlin Nimier-David, Delio Vicini, Tizian Zeltner, Baptiste Nicolet, Miguel Crespo, Vincent Leroy, and Ziyi Zhang. Mitsuba renderer, 2010. 6, 7, 19

    2010

  28. [36]

    Self-calibrating neural radiance fields

    Yoonwoo Jeong, Seokjun Ahn, Christopher Choy, Anima Anandkumar, Minsu Cho, and Jaesik Park. Self-calibrating neural radiance fields. In ICCV, 2021. 2, 3, 6, 8, 12, 15, 17, 18, 22, 23

    2021

  29. [37]

    Cubemap-based perception-driven blind quality assessment for 360-degree images

    Hao Jiang, Gangyi Jiang, Mei Yu, Yun Zhang, You Yang, Zongju Peng, Fen Chen, and Qingbo Zhang. Cubemap-based perception-driven blind quality assessment for 360-degree images. IEEE TIP, 2021. 5 10

    2021

  30. [38]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023. 2, 3, 4, 5, 6, 7, 8, 12, 16, 17, 18, 19, 20, 22, 23

    2023

  31. [39]

    3d gaussian splatting as markov chain monte carlo

    Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Wei- wei Sun, Jeff Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splatting as markov chain monte carlo. In NeurIPS, 2024. 22

    2024

  32. [40]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR, 2015. 23

    2015

  33. [41]

    Wide- angle camera distortions and non-uniform illumination in mobile robot tracking

    Gregor Klan ˇcar, Matej Kristan, and Rihard Karba. Wide- angle camera distortions and non-uniform illumination in mobile robot tracking. Robotics and Autonomous Systems ,

  34. [42]

    Point-based neural rendering with per- view optimization

    Georgios Kopanas, Julien Philip, Thomas Leimk ¨uhler, and George Drettakis. Point-based neural rendering with per- view optimization. In Computer Graphics Forum, 2021. 3, 4

    2021

  35. [43]

    Efficient solution to the epipolar geometry for radially distorted cameras

    Zuzana Kukelova, Jan Heller, Martin Bujnak, Andrew Fitzgibbon, and Tomas Pajdla. Efficient solution to the epipolar geometry for radially distorted cameras. In ICCV,

  36. [44]

    360fu- sionnerf: Panoramic neural radiance fields with joint guid- ance

    Shreyas Kulkarni, Peng Yin, and Sebastian Scherer. 360fu- sionnerf: Panoramic neural radiance fields with joint guid- ance. In IROS, 2023. 3

    2023

  37. [45]

    Plane-based calibration and auto-calibration of a fish-eye camera

    Hongdong Li and Richard Hartley. Plane-based calibration and auto-calibration of a fish-eye camera. In ACCV, 2006. 3

    2006

  38. [46]

    Omnigs: Omnidirectional gaussian splatting for fast radiance field reconstruction using omnidirectional images

    Longwei Li, Huajian Huang, Sai-Kit Yeung, and Hui Cheng. Omnigs: Omnidirectional gaussian splatting for fast radiance field reconstruction using omnidirectional images. arXiv preprint arXiv:2404.03202, 2024. 3

    2024 arXiv

  39. [47]

    Fisheye-gs: Lightweight and extensible gaussian splatting module for fisheye cameras

    Zimu Liao, Siyan Chen, Rong Fu, Yi Wang, Zhongling Su, Hao Luo, Linning Xu, Bo Dai, Hengjie Li, Zhilin Pei, et al. Fisheye-gs: Lightweight and extensible gaussian splatting module for fisheye cameras. In ECCV Workshop, 2024. 1, 2, 3, 6, 7, 12, 18, 19, 20

    2024

  40. [48]

    3d reconstruction from full-view fisheye camera

    Chuiwen Ma, Liang Shi, Hanlu Huang, and Mengyuan Yan. 3d reconstruction from full-view fisheye camera. arXiv preprint arXiv:1506.06273, 2015. 1

    2015 arXiv

  41. [49]

    Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar

    Ben Mildenhall, Pratul P. Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view syn- thesis with prescriptive sampling guidelines. ACM TOG ,

  42. [50]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In ECCV, 2020. 1, 5, 12, 16

    2020

  43. [51]

    3d gaussian ray trac- ing: Fast tracing of particle scenes

    Nicolas Moenne-Loccoz, Ashkan Mirzaei, Or Perel, Ric- cardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, and Zan Gojcic. 3d gaussian ray trac- ing: Fast tracing of particle scenes. SIGGRAPH ASIA, 2024. 2, 3

    2024

  44. [52]

    Instant neural graphics primitives with a multires- olution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a multires- olution hash encoding. ACM TOG, 2022. 5

    2022

  45. [53]

    Camera pose estimation using implicit distortion models

    Linfei Pan, Marc Pollefeys, and Viktor Larsson. Camera pose estimation using implicit distortion models. In CVPR,

  46. [54]

    Camp: Camera preconditioning for neural radiance fields

    Keunhong Park, Philipp Henzler, Ben Mildenhall, Jonathan T Barron, and Ricardo Martin-Brualla. Camp: Camera preconditioning for neural radiance fields. ACM TOG, 2023. 16

    2023

  47. [55]

    Stratified self-calibration with the modulus constraint

    Marc Pollefeys and Luc Van Gool. Stratified self-calibration with the modulus constraint. TPAMI, 1999. 2

    1999

  48. [56]

    Self- calibration and metric reconstruction inspite of varying and unknown intrinsic camera parameters

    Marc Pollefeys, Reinhard Koch, and Luc Van Gool. Self- calibration and metric reconstruction inspite of varying and unknown intrinsic camera parameters. IJCV, 1999. 3

    1999

  49. [57]

    Adop: Approximate differentiable one-pixel point rendering

    Darius R ¨uckert, Linus Franke, and Marc Stamminger. Adop: Approximate differentiable one-pixel point rendering. ACM TOG, 2022. 3, 6, 7

    2022

  50. [58]

    A toolbox for easily calibrating omnidirectional cam- eras

    Davide Scaramuzza, Agostino Martinelli, and Roland Sieg- wart. A toolbox for easily calibrating omnidirectional cam- eras. In IROS, 2006. 3

    2006

  51. [59]

    Structure-from-motion revisited

    Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. In CVPR, 2016. 2, 3, 5, 6, 7, 8, 12, 14, 17, 20

    2016

  52. [60]

    Isocube: Exploiting the cubemap hardware

    Liang Wan, Tien-Tsin Wong, and Chi-Sing Leung. Isocube: Exploiting the cubemap hardware. TVCG, 2007. 5

    2007

  53. [61]

    Neural lens modeling

    Wenqi Xian, Alja ˇz Bo ˇziˇc, Noah Snavely, and Christoph Lassner. Neural lens modeling. In CVPR, 2023. 3, 4

    2023

  54. [62]

    Vr-nerf: High- fidelity virtualized walkable spaces

    Linning Xu, Vasu Agrawal, William Laney, Tony Garcia, Aayush Bansal, Changil Kim, Samuel Rota Bul `o, Lorenzo Porzi, Peter Kontschieder, Aljaˇz Boˇziˇc, et al. Vr-nerf: High- fidelity virtualized walkable spaces. In SIGGRAPH Asia ,

  55. [63]

    gsplat: An open-source library for Gaussian splatting

    Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, and Angjoo Kanazawa. gsplat: An open-source library for Gaussian splatting. arXiv preprint arXiv:2409.06765, 2024. 22

    2024 arXiv

  56. [64]

    Differentiable surface splatting for point-based geometry processing

    Wang Yifan, Felice Serena, Shihao Wu, Cengiz ¨Oztireli, and Olga Sorkine-Hornung. Differentiable surface splatting for point-based geometry processing. ACM TOG, 2019. 3, 4

    2019

  57. [65]

    Mip-splatting: Alias-free 3d gaussian splat- ting

    Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. Mip-splatting: Alias-free 3d gaussian splat- ting. In CVPR, 2024. 23

    2024

  58. [66]

    Camera self-calibration from video sequences: the Kruppa equations revisited

    Cyril Zeller and Olivier Faugeras. Camera self-calibration from video sequences: the Kruppa equations revisited. PhD thesis, INRIA, 1996. 2

    1996

  59. [67]

    Flexible camera calibration by viewing a plane from unknown orientations

    Zhengyou Zhang. Flexible camera calibration by viewing a plane from unknown orientations. In ICCV, 1999. 3

    1999

  60. [68]

    Ewa volume splatting

    Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. Ewa volume splatting. InVisualization, 2001. 4, 5 11 Self-Calibrating Gaussian Splatting for Large Field-of-View Reconstruction Supplementary Material Content

    2001

  61. [69]

    Supplementary Video 12

  62. [70]

    Optimization of Camera Parameters 12

  63. [71]

    Distortion Estimation from COLMAP 17

  64. [72]

    Computational Efficiency 18

  65. [73]

    Extra Experiments 18

  66. [74]

    Implementation Details 22

  67. [75]

    Failure Cases and Limitations 23

  68. [76]

    supp video.mp4,

    Supplementary Video We provide a video, “supp video.mp4,” to better compare our method with baselines. Our video is organized into three parts. The first part presents a comparison between our method and baselines on the FisheyeNeRF dataset [36] across three scenes. Vanilla 3D...

  69. [77]

    pose opt.mp4

    Optimization of Camera Parameters In this section, we first derive the gradients for all camera parameters during training in Sec. 2.1. We then demon- strate the effectiveness of the joint optimization of distor- tion alongside extrinsic and intrinsic parameters in Sec. 2.2. F...

  70. [78]

    However, these parameters are inaccurate when derived from highly distorted images

    Distortion Estimation from COLMAP In practice, the distortion estimated from the SfM [59] pipeline can be used as an initialization for our hybrid field, stabilizing training and accelerating convergence. However, these parameters are inaccurate when derived from highly distor...

  71. [79]

    To verify the hypothesis that our hy- brid method achieves a better balance between expressive- ness and efficiency, we perform an ablation study on the FisheyeNeRF dataset [36]

    Computational Efficiency Training Time. To verify the hypothesis that our hy- brid method achieves a better balance between expressive- ness and efficiency, we perform an ablation study on the FisheyeNeRF dataset [36]. Specifically, we analyze the grid resolution of Pc, which ...

  72. [80]

    fisheye-gs failure.mp4

    Extra Experiments 5.1. Quantitative Comparisons with Fisheye-GS In addition to Fig. 5 in the main paper, we also provide a quantitative evaluation of our method compared with the baseline Fisheye-GS [47] in Tab. 9 and Tab. 10. The per- formance degradation observed in the base...

  73. [81]

    We use the same loss function as 3DGS for training [38]

    Implementation Details Our implementation is based on the codebase from Gaus- sian Splatting [38] and gsplat [63]. We use the same loss function as 3DGS for training [38]. The invertible ResNet is constructed using FrEIA [3]. We follow Kerbl et al

  74. [82]

    We also adopt the implementation of MCMC densi- fication [39]

    to select hyperparameters for optimizing 3D Gaus- sians. We also adopt the implementation of MCMC densi- fication [39]. Compared with vanilla densification, MCMC helps remove floaters by using opacity thresholding to relo- cate dead Gaussians. While the final quantitative resu...

  75. [83]

    Recon- structing the sky poses challenges due to moving clouds and the large uniform regions of blue and white without tex- tures

    Failure Cases and Limitations Real-world outdoor captures often include the sky. Recon- structing the sky poses challenges due to moving clouds and the large uniform regions of blue and white without tex- tures. The 3DGS [38] method tends to assign large Gaus- sians to the sky...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.