REVIEW 3 major objections 5 minor 24 references
HuSc3D: Human Sculpture dataset for 3D object reconstruction
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read HuSc3D is a dataset of six white sculptures captured under realistic, messy conditions, and the paper claims it separates 3D reconstruction methods more sharply than conventional benchmarks.
desk verdict A genuinely useful small dataset for 3D reconstruction under amateur capture conditions, but the benchmark numbers for the two video scenes are compromised by a random rather than temporal test split. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the dataset itself, defined by six scenes of white sculptures and three acquisition styles: sparse DSLR photo sets (Kacper, Paulina), high-resolution phone photos (Jarosław, Hanna), and frames extracted from phone video (Tosia, Wiktor). The argument is carried by the contrast between these designed challenge dimensions and the fixed evaluation protocol: the same six methods, each trained for 30,000 iterations, with 12.5% of each scene's images reserved as a test set. The dataset is meant to force methods to cope simultaneously with dynamic background, white-balance inconsistency, low-texture intricate geometry, and sparse or oversized inputs.
What would settle it
Re-run the Tosia and Wiktor scenes with a temporal holdout, training on one block of frames and testing on a later block, and compare PSNR and LPIPS to the paper's random-split numbers; if the metrics drop sharply or the method ranking changes, the claimed separation is an artifact of near-duplicate test frames rather than a robustness difference.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that a deliberately unpolished, small dataset is a better discriminator of reconstruction methods than the polished datasets currently in use. HuSc3D combines several real-world failure modes at once: low-texture white surfaces that give feature matching and color estimation little to work with, fine perforated geometry that demands high-frequency detail, photometric inconsistency from automatic white balance, moving people in the background, and some scenes with only 66 or 104 registered views. The reported experiments show Splatfacto-MCMC leading all metrics (average PSNR 32.45, SSIM 0.953, LPIPS 0.109), Nerfacto trailing (average PSNR 21.98), and Instant-NGP running out of memory on two of the six scenes. The authors read this spread as evidence that HuSc3D differentiates model performance along axes that conventional benchmarks mask.
Load-bearing premise
The evaluation assumes that a random 12.5% test split measures how well a method generalizes, but in the two video-derived scenes the test frames are extracted from the same continuous footage as the training frames, so they are near-duplicates and the reported metrics are likely inflated.
Editorial extensions
If this is right
- A benchmark that includes casual acquisition conditions can reveal robustness differences that clean datasets hide; methods that look similar on synthetic or curated scenes separate sharply on HuSc3D.
- Gaussian-splatting methods, especially Splatfacto-MCMC, are more reliable than NeRF-based methods on low-texture white objects with sparse views and photometric inconsistency; however, they pay for this with longer training and lower frame rates.
- High-resolution captures (Jarosław, Hanna) force 8x downsampling, and even the best method loses visible fine detail; methods that can process large images more efficiently would directly improve reconstruction fidelity on such scenes.
- In the video-derived scenes, NeRF-based methods kept a more stable white balance while Gaussian-splatting methods better captured sudden white-balance changes, so method choice changes how the same photometric ambiguity is resolved.
Reading between the lines
- A stricter evaluation of the two video scenes would use a temporal split, training on one contiguous block of frames and testing on a later block, rather than a random 12.5% split, because frames extracted every 20th frame from continuous footage are near-duplicates; without such a holdout, the reported video-scene scores likely overstate generalization to genuinely new viewpoints.
- The dataset is well suited as a controlled testbed for preprocessing: one could quantify how much of the method gap disappears when automatic white balance is post-corrected or when dynamic background regions are masked out before training.
- The low-texture, perforated white surface is close to an adversarial input for feature-based structure-from-motion; extending the dataset with more artists or materials could turn it into a broader robustness benchmark rather than a single-style stress test.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces HuSc3D, a new dataset of six real-world scenes showing white, intricately perforated sculptures by Paweł Althamer, captured under conditions that mimic casual acquisition by a non-expert user: sparse COLMAP-registered views (Kacper, Paulina), high-resolution phone stills requiring downsampling (Jarosław, Hanna), and phone-video frames with automatic white-balance shifts (Tosia, Wiktor). Six popular reconstruction methods—2DGS, Mip-Splatting, Nerfacto, Splatfacto, Splatfacto-MCMC, and Instant-NGP—are benchmarked with PSNR, SSIM, LPIPS, training time, and FPS. The central claim is that HuSc3D's combination of dynamic background, photometric inconsistency, sparse views, and low-texture objects 'effectively differentiat[es] model performance,' exposing limitations masked by conventional datasets.
Significance. If the quantitative claims are made secure, HuSc3D would be a useful benchmark: it addresses a real gap, uses real public sculptures, combines several realistic acquisition challenges that existing datasets treat only partially, and evaluates a reasonable set of open-source baselines under a stated protocol. The qualitative results support the rough ordering (Splatfacto-MCMC best, Nerfacto worst) and the dataset artifact itself, with code and data publicly announced, is a concrete community resource. However, the central 'distinctiveness' claim is currently supported mainly by a single quantitative table on HuSc3D alone, without statistical error bars, without a temporal holdout for video-derived scenes, and without same-protocol comparisons on existing datasets; these issues must be addressed before the benchmark's differentiating power is established.
major comments (3)
- [Section 4; Section 3 (Tosia and Wiktor)] The test split is not a valid generalization split for the video-derived scenes. Section 4 states that 'Following the protocol of 3DGS 12.5% of images from each scene are reserved as a test set,' while Section 3 reports that Tosia and Wiktor were obtained by extracting every 20th frame from continuous phone video. A random 12.5% split places each test frame within one or two extracted-frame steps of a training frame in capture time, so test views are near-duplicates of training views in viewpoint, lighting, and background. This likely inflates PSNR, SSIM, and LPIPS for those scenes and can change the method ranking reported in Table 2. Please rerun the benchmark with a temporal holdout (e.g., a contiguous block of frames or frames spaced by a sufficient capture-time interval) and report both sets of numbers, or justify why temporal leakage would not affect the conclusions.
- [Table 2; Section 4 (Quantitative results)] All quantitative results are single-run point estimates with no variance. The paper's central claim that HuSc3D 'effectively differentiat[es] model performance' is a statistical statement about rankings and margins, yet Table 2 reports no multiple seeds, confidence intervals, or significance tests. For example, the PSNR gap between Splatfacto and Mip-Splatting is about 0.7 dB on average, and per-scene differences vary in sign; without error bars it is unclear whether the ordering is stable. Please report results over at least three seeds (with different random initializations and test splits) and include standard deviations or confidence intervals, especially for the averaged metrics.
- [Section 4 (Comparative Analysis); Table 3] The 'distinctiveness' of HuSc3D is asserted from a qualitative feature-by-feature comparison in Table 3, not from a direct measurement. To support the statement that HuSc3D 'effectively differentiat[es] model performance' better than conventional datasets, the same set of methods should be evaluated under the same protocol on at least one or two representative existing datasets (e.g., LLFF or Mip-NeRF 360 subset), and the spread or ranking separability of metrics should be compared. Without such a head-to-head comparison, the claim that limitations are 'often masked by more conventional datasets' remains unsupported by the experiments presented.
minor comments (5)
- [Section 2] The paragraph describing Mip-NeRF 360 is duplicated verbatim; please remove one copy.
- [Throughout] There are several typos and grammatical slips, for example 'NICON D7000' should be 'Nikon D7000', 'are are particurarly' should be 'are particularly', 'sensivities' should be 'sensitivities', and 'more taggle' should be 'more tackle'; a thorough proofread is needed.
- [Table 2] Several time entries have missing separators, e.g., '7:328:20' for 2DGS on Tosia and '6:2110:45' for Splatfacto on Wiktor; these likely need spaces or a consistent format.
- [Table 1] The DeepBlending row uses a '‡' symbol with no accompanying footnote explaining the symbol, and the header text 'Min no of Max no of images per scene' is ambiguous; please clean up the table formatting.
- [Section 3 (Tosia and Wiktor scenes)] The sentence 'Tosia scene has 13.2% of scenes with misleading white balance and Wiktor 28.4%' should say 'frames' rather than 'scenes', and it would be helpful to state how this percentage was computed.
Circularity Check
No circularity: HuSc3D is a new artifact benchmarked with external methods; self-citations are only related-work examples.
full rationale
HuSc3D is a dataset contribution, not a derived theoretical claim. The central claim that the dataset separates reconstruction methods rests on measuring six external baseline implementations (2DGS, Mip-Splatting, Nerfacto, Splatfacto, Splatfacto-MCMC, Instant-NGP) with standard PSNR, SSIM, LPIPS, training time, and FPS metrics on held-out images. These methods come from public codebases with default hyperparameters and are not fit to the dataset's distinctiveness claim, so the observed performance spread is independent evidence rather than an input to the paper's reasoning. The only author self-citations are [12] and [14] in a related-work survey sentence; neither is load-bearing for the dataset's design, the evaluation protocol, or the conclusions. No equation, fitted parameter, or uniqueness theorem is invoked, and no prediction is defined in terms of the data it claims to explain. The random 12.5% test split for the video-derived Tosia and Wiktor scenes is a plausible temporal-leakage validity concern for the benchmark numbers, but that is an evaluation-design issue, not circularity of the kind defined in this analysis. Accordingly, the derivation chain is self-contained and the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- AWB misclassification threshold =
13.2% (Tosia), 28.4% (Wiktor)
- Input downsampling factors =
4x (most scenes), 8x (Hanna, Jaroslaw)
assumptions (3)
- domain assumption COLMAP-registered subsets of 66 and 104 images represent a realistic sparse-view challenge for reconstruction
- domain assumption White, low-texture, perforated surfaces are a representative hard case for contemporary 3D reconstruction
- domain assumption Image-based metrics (PSNR/SSIM/LPIPS) capture the reconstruction quality relevant to fine geometry and color ambiguity
Cite this review
Pith. "Pith review of HuSc3D: Human Sculpture dataset for 3D object reconstruction." pith.science (2026). https://pith.science/paper/AEK5OIMD
@misc{pith2026250607628,
author = {Pith},
title = {Pith review of: HuSc3D: Human Sculpture dataset for 3D object reconstruction},
year = {2026},
howpublished = {\url{https://pith.science/paper/AEK5OIMD}},
note = {Machine review of arXiv:2506.07628}
}
read the original abstract
3D scene reconstruction from 2D images is one of the most important tasks in computer graphics. Unfortunately, existing datasets and benchmarks concentrate on idealized synthetic or meticulously captured realistic data. Such benchmarks fail to convey the inherent complexities encountered in newly acquired real-world scenes. In such scenes especially those acquired outside, the background is often dynamic, and by popular usage of cell phone cameras, there might be discrepancies in, e.g., white balance. To address this gap, we present HuSc3D, a novel dataset specifically designed for rigorous benchmarking of 3D reconstruction models under realistic acquisition challenges. Our dataset uniquely features six highly detailed, fully white sculptures characterized by intricate perforations and minimal textural and color variation. Furthermore, the number of images per scene varies significantly, introducing the additional challenge of limited training data for some instances alongside scenes with a standard number of views. By evaluating popular 3D reconstruction methods on this diverse dataset, we demonstrate the distinctiveness of HuSc3D in effectively differentiating model performance, particularly highlighting the sensitivity of methods to fine geometric details, color ambiguity, and varying data availability--limitations often masked by more conventional datasets.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Srinivasan, Matthew Tancik, Jonathan T
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. InECCV, 2020
2020
-
[2]
3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July 2023
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering.ACM Transactions on Graphics, 42(4), July 2023
2023
-
[3]
Pixelwise view selection for unstructured multi-view stereo
Johannes Lutz Schönberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for unstructured multi-view stereo. InEuropean Conference on Computer Vision (ECCV), 2016
2016
-
[4]
Structure-from-motion revisited
Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016
2016
-
[5]
Srinivasan, Peter Hedman, Ricardo Martin-Brualla, and Jonathan T
Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, Peter Hedman, Ricardo Martin-Brualla, and Jonathan T. Barron. MultiNeRF: A Code Release for Mip-NeRF 360, Ref-NeRF, and RawNeRF, 2022
2022
-
[6]
Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T. Barron, and Pratul P. Srinivasan. Ref-NeRF: Structured view-dependent appearance for neural radiance fields.CVPR, 2022
work page 2022
-
[7]
D-NeRF: Neural Radiance Fields for Dynamic Scenes
Albert Pumarola, Enric Corona, Gerard Pons-Moll, and Francesc Moreno-Noguer. D-NeRF: Neural Radiance Fields for Dynamic Scenes. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020
work page 2020
-
[8]
Headnerf: A real-time nerf-based parametric head model
Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, and Juyong Zhang. Headnerf: A real-time nerf-based parametric head model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20374–20384, June 2022
work page 2022
Show all 24 references
-
[9]
Depth-supervised nerf: Fewer views and faster training for free
Kangle Deng, Andrew Liu, Jun-Yan Zhu, and Deva Ramanan. Depth-supervised nerf: Fewer views and faster training for free. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12882–12891, June 2022
2022
-
[10]
4d gaussian splatting for real-time dynamic scene rendering
Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Wang Xinggang. 4d gaussian splatting for real-time dynamic scene rendering. arXiv preprint arXiv:2310.08528, 2023
2023 arXiv
-
[11]
Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes.arXiv preprint arXiv:2312.14937, 2023
Yi-Hua Huang, Yang-Tian Sun, Ziyi Yang, Xiaoyang Lyu, Yan-Pei Cao, and Xiaojuan Qi. Sc-gs: Sparse-controlled gaussian splatting for editable dynamic scenes.arXiv preprint arXiv:2312.14937, 2023
2023 arXiv
-
[12]
D- miso: Editing dynamic 3d scenes using multi-gaussians soup.arXiv preprint arXiv:2405.14276, 2024
Joanna Waczy´nska, Piotr Borycki, Joanna Kaleta, Sławomir Tadeja, and Przemysław Spurek. D- miso: Editing dynamic 3d scenes using multi-gaussians soup.arXiv preprint arXiv:2405.14276, 2024
2024 arXiv
-
[13]
2d gaussian splatting for geometrically accurate radiance fields
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. InSIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024
2024
-
[14]
Gaussian splatting with nerf-based color and opacity.Computer Vision and Image Understanding, 251:104273, 2025
Dawid Malarz, Weronika Smolak-Dy˙zewska, Jacek Tabor, Sławomir Tadeja, and Przemysław Spurek. Gaussian splatting with nerf-based color and opacity.Computer Vision and Image Understanding, 251:104273, 2025
2025
-
[15]
Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018
Blender Online Community.Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018
2018
-
[16]
A photogrammetry-based framework to facilitate image-based modeling and automatic camera tracking.International Conference on Computer Graphics Theory and Applications, 2021
Sebastian Bullinger, Christoph Bodensteiner, and Michael Arens. A photogrammetry-based framework to facilitate image-based modeling and automatic camera tracking.International Conference on Computer Graphics Theory and Applications, 2021
2021
-
[17]
3dgs render blender addon by kiri engine
Kiri-Innovation. 3dgs render blender addon by kiri engine. https://github.com/ Kiri-Innovation/3dgs-render-blender-addon, 2025
2025
-
[18]
Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar
Ben Mildenhall, Pratul P. Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines.ACM Transactions on Graphics (TOG), 2019. 10
2019
-
[19]
Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction
Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. InInternational Conference on Computer Vision, 2021
2021
-
[20]
Deep blending for free-viewpoint image-based rendering.ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings), 37(6), November 2018
Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. Deep blending for free-viewpoint image-based rendering.ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings), 37(6), November 2018
2018
-
[21]
Gaussian opacity fields: Efficient high-quality compact surface reconstruction in unbounded scenes.arXiv:2404.10772, 2024
Zehao Yu, Torsten Sattler, and Andreas Geiger. Gaussian opacity fields: Efficient high-quality compact surface reconstruction in unbounded scenes.arXiv:2404.10772, 2024
2024 arXiv
-
[22]
Nerfstudio: A modular framework for neural radiance field development
Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, and Angjoo Kanazawa. Nerfstudio: A modular framework for neural radiance field development. In ACM SIGG...
2023
-
[23]
3d gaussian splatting as markov chain monte carlo
Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Yang-Che Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 3d gaussian splatting as markov chain monte carlo. InAdvances in Neural Information Processing Systems (NeurIPS), 2024. Spotlight Pre...
2024
-
[24]
Instant neural graphics primitives with a multiresolution hash encoding.ACM Trans
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding.ACM Trans. Graph., 41(4):102:1–102:15, July 2022. 11
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.