REVIEW 3 major objections 6 minor 1 cited by
Spatiotemporal Gaussian Optimization for 4D Cone Beam CT Reconstruction from Sparse Projections
T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A deforming cloud of Gaussian blobs reconstructs 4D cone-beam CT from a 1-minute scan without prior CT images.
desk verdict A plausible, well-benchmarked combination of 4D Gaussian splatting and a deformation network for prior-free 4D-CBCT, but the headline accuracy claim rests on simulated equi-spaced projections, not real 1-min clustered sampling. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the spatiotemporal Gaussian cloud: a set of roughly 80,000 to 350,000 3D Gaussian kernels, each described by an 11-parameter set (3D position, three scaling factors, four rotation parameters, and one density), rendered into 2D X-ray projections by density integration along rays. A deformation network, using HexPlane multi-resolution planes (the six 2D planes xy, xz, yz, xt, yt, and zt) and four MLP heads, maps a phase index to per-Gaussian changes in position, scale, rotation, and density, letting one canonical cloud represent all respiratory phases. Joint optimization of the Gaussians and the network minimizes the L1 plus SSIM mismatch between rendered and measured projections, while an adaptive control module densifies and prunes Gaussians during training. A CUDA-based voxelizer converts the deformed Gaussians into the final 4D-CBCT volumes.
What would settle it
Reconstruct a digital phantom with known respiratory deformation from simulated sparse projections and compare the network's Gaussian displacements against the known ground-truth motion fields; if the projections match well but the displacement field does not reproduce the known motion, the geometric-accuracy result reflects artifact absorption rather than true motion recovery.
Extended reading notes
Core claim
The paper's central discovery is that a 4D dynamic scene can be encoded as an explicitly deformable set of 3D Gaussians, each with a position, covariance, rotation, and density, and that optimizing those parameters directly against raw projection images yields phase-resolved CBCT volumes. A deformation network built on HexPlane spatiotemporal feature encoding feeds four small multilayer perceptrons that predict per-phase changes in position, scale, rotation, and density; the whole model is trained end-to-end with only L1 and structural-similarity projection losses. The final 4D volume is obtained by voxelizing the deformed Gaussians. The authors report that this parameterization, roughly 350,000 Gaussians and fewer than 100,000 network parameters, achieves comparable RMSE and SSIM to methods that use a prior four-dimensional CT and the lowest 3D translation error of the planning target volume among the six methods tested.
Load-bearing premise
The load-bearing premise is that a fixed set of Gaussian blobs, deformed as a function of phase index, can represent true respiratory motion from projection consistency alone; the paper does not include an ablation or physics-based regularization that rules out the deformation network absorbing streaking artifacts and phase-sorting errors as blob displacements.
Editorial extensions
If this is right
- 4D-CBCT no longer depends on a pretreatment CT or an explicit motion model, removing the main bias source in prior-image-based methods.
- The compact parameterization (roughly 350k Gaussians plus under 100k network weights) replaces roughly 450 million voxel unknowns, cutting memory demands by two orders of magnitude and making high-resolution 4D reconstruction feasible on a single GPU.
- On the public benchmark's validation cohort, the method gives the lowest 3D target translation error (1.65 mm) among six methods, supporting its use for target alignment before beam delivery.
- A clinical 1-minute scan was reconstructed into both 10 and 50 respiratory phases, so temporal resolution is not capped by the acquisition time.
Reading between the lines
- Editorial extension: Conditioning the deformation network on continuous time rather than discrete phase indices could remove phase sorting entirely; the authors list phase-sorting-free reconstruction as future work.
- Editorial extension: The optimized Gaussian positions could act as an intrinsic tumor-motion surrogate, but the paper does not validate that individual Gaussians track the same tissue across phases, so this use would need independent verification.
- Editorial extension: Because the central objective is projection consistency alone, a physics-based deformation regularizer would be the most direct stress test of whether blob displacements correspond to real tissue motion; the paper acknowledges such regularization is missing.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a 4D-CBCT reconstruction framework based on an X-ray-adapted 3D Gaussian splatting representation with a HexPlane deformation network. Canonical Gaussians initialized from an average CBCT are optimized against phase-binned measured projections, jointly with a network that deforms position, scale, rotation, and density as a function of phase index; voxelization yields the 4D volume. The method is evaluated on 29 simulated AAPM SPARE validation scans against FDK and five prior-based methods, reporting comparable RMSE/SSIM and the lowest PTV translation error among the compared methods, plus a single clinical TrueBeam 1-minute scan assessed qualitatively. The authors claim no prior CT or explicit motion model is needed and that the representation reduces unknowns by more than 100x compared with voxel grids.
Significance. If the quantitative claims hold, this would be a practically important step for 1-minute 4D-CBCT in radiotherapy: it replaces prior-image and explicit motion-model dependencies with a self-contained differentiable optimization, and the parameter-efficiency argument is appealing. The paper's strengths are its use of the public SPARE benchmark with organizer-provided evaluation code, comparison against six established methods, and public release of code and results. The evaluation is reproducible in structure. However, the current evidence does not yet establish the central clinical claim under real 1-minute acquisition geometry, and the reported performance margins are not supported by statistical tests. The method is nonetheless well grounded in prior Gaussian splatting and X-ray rasterization work, and the reported improvements are plausible.
major comments (3)
- [Section III.C, Table II, Figures 3-6] The claim that the proposed method achieves the lowest PTV translation error and consistently better RMSE is not supported by any statistical significance analysis. The margins are small relative to the inter-patient variability visible in Figure 6 (e.g., 3D translational error 1.65 mm vs 1.79-1.90 mm for the closest competitors; rSI 0.93 deg vs 0.72 deg; AP 1.36 mm vs 0.99 mm). The paper reports RMS values over 29 cases only. Report paired differences with confidence intervals or p-values for the RMSE, SSIM, and PTV translation/rotation errors, and state which comparisons are meaningfully different.
- [Section III.A and III.E] The quantitative geometric accuracy of the method has not been demonstrated under the acquisition conditions that define the problem. The authors correctly note that the SPARE clinical datasets, generated with equi-spaced down-sampling, 'do not accurately replicate the challenges of a real 1-minute CBCT scan'; in a real 1-minute scan, projections within each respiratory bin cluster around specific angles and phase sorting is imperfect. The single real TrueBeam case is evaluated qualitatively only, without ground truth for PTV position. Therefore Table II does not establish the abstract's claim of reconstructing 4D-CBCT from a 1-minute scan with the lowest PTV translation error. Add a simulation study that combines realistic angular clustering (e.g., using the clinical scan's phase-bin angular distribution) with known ground truth, or a motion phantom with known target positions.
- [Section IV (Discussion) and Section II.C] Because the deformation network is not regularized by physics, and the authors state that each deformed Gaussian 'does not necessarily represent realistic soft tissue deformation,' it remains possible that the network absorbs streaking artifacts or phase-sorting errors into blob displacements rather than true respiratory motion. This is directly relevant to the claimed contribution of 'accurately capturing underlying motion.' The paper provides no ablation or validation of the learned deformations. I recommend an ablation removing the density/scale/rotation deformation heads, and/or a comparison of predicted Gaussian displacements against known motion in a simulated phantom, to show that the motion-capture mechanism is real.
minor comments (6)
- [Equation (13)] Equation (13): G{X', R', R', rho'} repeats R'; the second entry should be S' (the deformed scaling).
- [Equation (12)] Equation (12): The definition of f_h is difficult to parse; please clarify the index ranges for the resolution levels and the meaning of the product and union operations.
- [Section II.E and III.E] The text says timestamps are 10 values (0 to 0.9), but Section III.E reconstructs 50 phases; specify whether the network is retrained with 50 phase bins or timestamps are rescaled.
- [Figure 10 caption] The caption uses 'Phase 00' and 'Phase 50' while the clinical reconstruction is described as 10-phase; clarify the phase numbering.
- [Equations (7)-(8)] Use a distinct variable for the integration parameter (e.g., s) and give explicit integration limits to avoid the confusion of r(t) with the ray coordinate r.
- [Section III.C figure ordering] The figures in Section III.C appear to be ordered inconsistently: 'Fig. 6' is referenced before its caption and 'Fig. 5' appears in the caption sequence after that reference; renumber/reference the figures consistently.
Circularity Check
No circularity: the 4D Gaussian reconstruction is a projection-consistency fit benchmarked against independent 4DCT ground truth; self-citations are non-load-bearing.
full rationale
The reconstruction chain is self-contained and empirically benchmarked. The 4D Gaussian parameters and the Gaussian deformation network are optimized solely by minimizing the projection-domain loss in Eq. (14) against measured or simulated raw projections; no parameter is fitted to the evaluation metrics (RMSE, SSIM, PTV translation error) that are later reported. Geometric accuracy is assessed by registering the voxelized reconstructions to independent ground-truth 4DCT using the SPARE organizers' evaluation code, so the claims do not reduce to the optimization objective. The CGLS average-image initialization is standard and uses the same projection data, but this is an initialization, not a fitted prediction. Self-citations ([1], [25], [29]) are background clinical motivation, phase-sorting methodology, and a comparison motion-compensation method; none of them supplies an unverified premise that forces the paper's conclusions. The paper's own acknowledged limitations (3-hour runtime, no ablation, deformation network not physics-regularized) are correctness risks, not circular reasoning. No quoted reduction of an output to an input by construction was found.
Assumptions & free parameters
free parameters (5)
- SSIM loss weight lambda =
not specified
- Deformation ratio range for scaling, rotation, density =
[0.9, 1.1]
- Initial Gaussian sampling parameters =
1-of-8 grid; 20,000 edge points; density threshold; gradient threshold
- Adaptive control schedule =
densify/prune iterations 500 to 15000 at 100 intervals; 30000 total iterations
- Phase bin count =
10 (and 50 for temporal test)
assumptions (4)
- domain assumption X-ray attenuation is a line integral of a scalar density field, and the superposition of 3D Gaussians can represent that field.
- domain assumption Each phase bin can be assigned a timestamp and all motion is explained by deforming a canonical Gaussian set as a function of timestamp.
- standard math The projected 2D Gaussian remains Gaussian and the density scaling in Eq. 10 exactly corrects integration bias.
- domain assumption The AAPM SPARE simulated projections and phase labels are clinically representative enough for quantitative evaluation.
Cite this review
Pith. "Pith review of Spatiotemporal Gaussian Optimization for 4D Cone Beam CT Reconstruction from Sparse Projections." pith.science (2026). https://pith.science/paper/GAIVYSZG
@misc{pith2026250104140,
author = {Pith},
title = {Pith review of: Spatiotemporal Gaussian Optimization for 4D Cone Beam CT Reconstruction from Sparse Projections},
year = {2026},
howpublished = {\url{https://pith.science/paper/GAIVYSZG}},
note = {Machine review of arXiv:2501.04140}
}
read the original abstract
In image-guided radiotherapy (IGRT), four-dimensional cone-beam computed tomography (4D-CBCT) is critical for assessing tumor motion during a patients breathing cycle prior to beam delivery. However, generating 4D-CBCT images with sufficient quality requires significantly more projection images than a standard 3D-CBCT scan, leading to extended scanning times and increased imaging dose to the patient. To address these limitations, there is a strong demand for methods capable of reconstructing high-quality 4D-CBCT images from a 1-minute 3D-CBCT acquisition. The challenge lies in the sparse sampling of projections, which introduces severe streaking artifacts and compromises image quality. This paper introduces a novel framework leveraging spatiotemporal Gaussian representation for 4D-CBCT reconstruction from sparse projections, achieving a balance between streak artifact reduction, dynamic motion preservation, and fine detail restoration. Each Gaussian is characterized by its 3D position, covariance, rotation, and density. Two-dimensional X-ray projection images can be rendered from the Gaussian point cloud representation via X-ray rasterization. The properties of each Gaussian were optimized by minimizing the discrepancy between the measured projections and the rendered X-ray projections. A Gaussian deformation network is jointly optimized to deform these Gaussian properties to obtain a 4D Gaussian representation for dynamic CBCT scene modeling. The final 4D-CBCT images are reconstructed by voxelizing the 4D Gaussians, achieving a high-quality representation that preserves both motion dynamics and spatial detail. The code and reconstruction results can be found at https://github.com/fuyabo/4DGS_for_4DCBCT/tree/main
Figures
Forward citations
Cited by 1 Pith paper
-
DIGS: Dynamic CBCT Reconstruction using Deformation-Informed 4D Gaussian Splatting and a Low-Rank Free-Form Deformation Model
DIGS couples 4D Gaussian Splatting with a low-rank free-form deformation model to reconstruct motion-compensated CBCT images per projection, claiming better image quality and 6x faster training than HexPlane.
Reference graph
Works this paper leans on
-
[1]
4D cone- beam CT-based dose assessment for SBRT lung cancer treatment,
W. Cai, S. Dhou, F. Cifter, M. Myronakis, M. H. Hurwitz, C. L. Williams, R. I. Berbeco, J. Seco, and J. H. Lewis, "4D cone- beam CT-based dose assessment for SBRT lung cancer treatment," Phys. Med. Biol. , vol. 61, no. 2, pp. 554–568, 2016
work page 2016
-
[2]
SPARE: Sparse-view reconstruction challenge for 4D cone-beam CT from a 1 -min scan,
C. C. Shieh et al., “SPARE: Sparse-view reconstruction challenge for 4D cone-beam CT from a 1 -min scan,” Med Phys., vol. 46, no. 9, pp. 3799- 3811, Sep. 2019
work page 2019
-
[3]
X. Jia, Y. Lou, R. Li, W. Y. Song and S. B. Jiang, “GPU-based fast cone beam CT reconstruction from undersampled and noisy projection data via total variation,” Med Phys. Vol 37, no. 4, pp. 1757-60, Apr. 2010
work page 2010
-
[4]
G.-H. Chen, J. Tang, and S. Leng, "Prior image constrained compressed sensing (PICCS): A method to accurately reconstruct dynamic CT images from highly undersampled projection data sets," Med. Phys., vol. 35, no. 2, pp. 660–663, 2008
work page 2008
-
[5]
On -the-fly motion-compensated cone -beam CT using an a priori model of the respiratory motion,
S. Rit, J. W. H. Wolthaus, M. van Herk, and J.- J. Sonke, "On -the-fly motion-compensated cone -beam CT using an a priori model of the respiratory motion," Med. Phys., vol. 36, no. 6, pp. 2283– 2295, 2009. DOI: 10.1118/1.3115691
-
[6]
Y. Huang, K. Thielemans, G. Price, and J. R. McClelland, "Surrogate- driven respiratory motion model for projection -resolved motion estimation and motion-compensated cone-beam CT reconstruction from unsorted projection data," Phys. Med. Biol., vol. 69, pp. 025020, 2024
work page 2024
-
[7]
• M. Lauria, C. Miller, K. Singhrao, J. Lewis, W. Lin, D. O’Connell, L. Naumann, B. Stiehl, A. Santhanam, P. Boyle, A. C. Raldow, J. Goldin, I. Barjaktarevic, and D. A. Low, "Motion -compensated cone -beam CT reconstruction using an a priori motion model from CT simulation: A pilot study," Phys. Med. Biol., vol. 69, pp. 075022, 2024. DOI: 10.1088/1361- 65...
doi:10.1088/1361- 2024
-
[8]
M. J. Riblett, G. E. Christensen, E. Weiss, and G. D. Hugo, "Data-driven respiratory motion compensation for four -dimensional cone -beam computed tomography (4D -CBCT) using groupwise deformable registration," Med. Phys., vol. 45, no. 10 , pp. 4471– 4482, 2018. DOI: 10.1002/mp.13133
Show all 29 references
-
[9]
A novel digital t omosynthesis (DTS) reconstruction method using a deformation field map,
L. Ren, J. Zhang, D. Thongphiew, D. J. Godfrey, Q. J. Wu, S.- M. Zhou, and F. -F. Yin, "A novel digital t omosynthesis (DTS) reconstruction method using a deformation field map," Med. Phys., vol. 35 , no. 7 , pp. 3110–3122, 2008
2008
-
[10]
A patient -specific deep learning framework for 3D motion estimation and volumetric imaging during lung cancer radiotherapy,
N. Hindley, C.-C. Shieh, and P. Keall, "A patient -specific deep learning framework for 3D motion estimation and volumetric imaging during lung cancer radiotherapy," Phys. Med. Biol., vol. 68, pp. 14NT01, 2023
2023
-
[11]
DPI-MoCo: Deep prior image constrained motion compensation reconstruction fo r 4D CBCT,
D. Hu, C. Zhang, X. Fei, Y. Yao, Y. Xi, J. Liu, Y. Zhang, G. Coatrieux, J.-L. Coatrieux, and Y. Chen, "DPI-MoCo: Deep prior image constrained motion compensation reconstruction fo r 4D CBCT," IEEE Trans. Med. Imaging, 2024
2024
-
[12]
Self -contained deep learning-based boosting of 4D cone -beam CT reconstruction,
F. Madesta, T. Sentker, T. Gauer, and R. Werner, "Self -contained deep learning-based boosting of 4D cone -beam CT reconstruction," Med. Phys., vol. 47, no. 11, pp. 5619–5631, 2020
2020
-
[13]
Deep learning-based motion compensation for four -dimensional cone -beam computed tomography (4D-CBCT) reconstruction,
Z. Zhang, J. Liu, D. Yang, U. S . Kamilov, and G. D. Hugo, "Deep learning-based motion compensation for four -dimensional cone -beam computed tomography (4D-CBCT) reconstruction," Med. Phys., vol. 50, no. 1, pp. 808–820, 2023
2023
-
[14]
Enhancement of 4D cone -beam computed tomography using a dual - encoder convolutional neural network,
Z. Jiang, Z. Zhang, Y. Chang, Y. Ge, F.- F. Yin, and L. Ren, "Enhancement of 4D cone -beam computed tomography using a dual - encoder convolutional neural network," IEEE Trans. Radiat. Plasma Med. Sci., vol. 6, no. 2, pp. 222–230, 2022
2022
-
[15]
PNMC: Four -dimensional cone -beam CT reconstruction combining prior network and motion compensation,
Z. Ou, J. Xie, Z. Teng, X. Wang, P. Jin, J. Du, M. Ding, H. Li, Y. Chen, and T. Niu, "PNMC: Four -dimensional cone -beam CT reconstruction combining prior network and motion compensation," Comput. Biol. Med., vol. 171, pp. 108145, 2024
2024
-
[16]
U -net-based deformation vector field estimation for motion -compensated 4D -CBCT reconstruction,
X. Huang, Y. Zhang, L. Chen, and J. Wang, "U -net-based deformation vector field estimation for motion -compensated 4D -CBCT reconstruction," Med. Phys., vol. 47, no. 7, pp. 3000–3012, 2020
2020
-
[17]
Fast four - dimensional cone -beam computed tomography reconstruction using deformable convolutional networks,
Z. Jiang, Y. Chang, Z. Zhang, F.- F. Yin, and L. Ren, "Fast four - dimensional cone -beam computed tomography reconstruction using deformable convolutional networks," Med. Phys. , vol. 49, no. 12 , pp. 6461–6476, 2022
2022
-
[18]
Zhang, H.-C
Y. Zhang, H.-C. Shao, T. Mengke, and T. Pan, "Dynamic cone-beam CT reconstruction using spatial and temporal implicit neural representation Fig. 10. Reconstruction using 1-min scan for a clinical case. Phase 00: The end- inhalation. Phase 50: the end- exhalation of the reconst...
-
[19]
Dynamic CBCT imaging using prior model -free spatiotemporal implicit neural representation (PMF-STINR),
H.-C. Shao, T. Mengke, T. Pan, and Y. Zhang, "Dynamic CBCT imaging using prior model -free spatiotemporal implicit neural representation (PMF-STINR)," Phys. Med. Biol., vol. 69, pp. 115030, 2024
2024
-
[20]
3D Gaussian Splatting for Real-Time Radiance Field Rendering,
B. Kerbl, G. Kopanas, T. Leimkuehl er, and G. Drettakis, "3D Gaussian Splatting for Real-Time Radiance Field Rendering," ACM Trans. Graph., vol. 42, no. 1, pp. 1–14, 2023
2023
-
[21]
Radiative Gaussian Splatting for Efficient X -ray Novel View Synthesis,
Y. Cai, Y. Liang, J. Wang, A. Wang, Y. Zhang, X. Yang, Z. Zhou, and A. Yuille, "Radiative Gaussian Splatting for Efficient X -ray Novel View Synthesis," arXiv, 2024
2024
-
[22]
R2 -Gaussian: Rectifying radiative Gaussian splatting for tomographic reconstruction,
R. Zha, T. J. Lin, Y. Cai, J. Cao, Y. Zhang, and H. Li, "R2 -Gaussian: Rectifying radiative Gaussian splatting for tomographic reconstruction," NeurIPS, 2024
2024
-
[23]
4D Gaussian Splatting for real -time dynamic scene rendering,
G. Wu, T. Yi, J. Fang, L. Xie, X. Zhang, W. Wei, W. Liu, Q. Tian, and X. Wang, "4D Gaussian Splatting for real -time dynamic scene rendering," arXiv, 2023
2023
-
[24]
HexPlane: A Fast Representation for Dynamic Scenes,
A. Cao and J. Johnson, "HexPlane: A Fast Representation for Dynamic Scenes," in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 130–141
2023
-
[25]
Robust breathing signal extraction from cone beam CT projections based on adaptive and global optimization techniques,
M. Chao, J. Wei, T. Li, Y. Yuan, K. E. Rosenzweig, and Y.- C. Lo, "Robust breathing signal extraction from cone beam CT projections based on adaptive and global optimization techniques," Phys. Med. Biol., vol. 61, no. 8, pp. 3109–3126, 2016
2016
-
[26]
Motion-aware temporal regularization for improved 4D cone -beam computed tomography,
C. Mory, G. Janssens, and S. Rit, "Motion-aware temporal regularization for improved 4D cone -beam computed tomography," Phys. Med. Biol. , vol. 61, no. 18, pp. 6856–6877, 2016. DOI: 10.1088/0031 - 9155/61/18/6856
2016 doi
-
[27]
4D -CBCT reconstruction from a one -minute scan,
C.-C. Shieh, V. Caillet, J. Booth, N. Hardcastle, C. Haddad, T. Eade, and P. Keall, "4D -CBCT reconstruction from a one -minute scan," in Engineering and Physical Sciences in Medicine Conference , Hobart, Australia, 2017
2017
-
[28]
A review on 4D cone- beam CT (4D -CBCT) in radiation therapy: Technical advances and clinical applications,
Y. Zhang, Z. Jiang, Y. Zhang, and L. Ren, "A review on 4D cone- beam CT (4D -CBCT) in radiation therapy: Technical advances and clinical applications," Med. Phys., vol. 51, no. 8, pp. 5164–5180, Aug. 2024. DOI: 10.1002/mp.17269. Epub Jun. 23, 2024
2024 doi
-
[29]
Enhanced 4D cone -beam CT with inter- phase motion model,
T. Li, A. Koong, and L. Xing, "Enhanced 4D cone -beam CT with inter- phase motion model," Med. Phys., vol. 34, no. 9, pp. 3688–3695, Sep. 2007
2007
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.