From images to properties: a NeRF-driven framework for granular material parameter inversion

Cheng-Hsi Hsiao; Krishna Kumar

REVIEW 4 major objections 4 minor 7 references

From images to properties: a NeRF-driven framework for granular material parameter inversion

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A NeRF-MPM loop recovers the friction angle of sand from images alone, within roughly 2 degrees in synthetic tests.

desk verdict Clean synthetic proof-of-concept for decoupled NeRF-MPM-Bayesian inversion; the 2-degree claim is only self-consistency until real data is tested. read the letter →

arxiv 2507.09005 v1 pith:EQII74LY submitted 2025-07-11 cs.CV physics.geo-ph

Cheng-Hsi Hsiao , Krishna Kumar This is my paper

classification cs.CVphysics.geo-ph

keywords frictionanglegranularmaterialsNeuralRadianceFieldsMaterialPointMethodBayesianoptimizationinverseproblemsplow-soilinteractionvision-basedgeotechnicalcharacterization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a single soil property—the internal friction angle—can be inferred from visual observations alone, without touching the soil. Its answer, in synthetic experiments, is yes: a pipeline that reconstructs the starting sand surface with NeRF, re-simulates a plow dragging through the sand with the Material Point Method, and optimises candidate angles by comparing rendered images against two fixed-camera videos recovers the true angle with mean absolute error of 0.64 to 1.38 degrees. The error stays within 2 degrees across all tested ground truths (25, 30, 35, and 45 degrees). The motivation is planetary exploration, where direct shear or triaxial tests are impractical and only rover cameras are available.

What carries the argument

The load-bearing mechanism is the closed loop among three components. First, a NeRF volume trained on 40 multi-view images of the undisturbed sand provides the initial positions of the material points. Second, an MPM simulation replays a four-stage plow motion (lower, push forward, pause, pull back) with a candidate friction angle and renders frames at times 50, 100, 150, and 199 from the same two fixed cameras used for the observation. Third, Bayesian optimization fits a Gaussian Process to the summed image losses, selects the next candidate angle by Expected Improvement, and iterates up to ten evaluations. The crucial relationship is that at the correct friction angle the rendered deformation pattern—the trench, the piled crown, the collapse after pull-back, and the backward accumulation—aligns with the observed images, making the image loss a proxy for physical agreement.

What would settle it

Run the pipeline on a physical plow-in-sand experiment whose friction angle is measured independently with direct shear or triaxial tests: if the recovered angle is off by more than 2 degrees, or if the image-loss minimum drifts away from the true angle when the plow trajectory or camera poses are perturbed, the central claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that image loss—the mean squared RGB difference between the rendered simulation and the recorded footage—is a reliable objective for inverse estimation of the friction angle. Even though the NeRF-reconstructed geometry used to seed the simulation is not identical to the geometry that generated the observations, the pixel differences grow systematically as the candidate angle moves away from the ground truth, especially near the soil crown and behind the plow. Bayesian optimization with a Gaussian Process surrogate and Expected Improvement acquisition finds the true angle in roughly ten simulation evaluations, with mean absolute errors of 0.64, 1.07, 1.24, and 1.38 degrees for ground truths of 25, 30, 35, and 45 degrees. This establishes that a decoupled, gradient-free pipeline—NeRF for geometry, MPM for forward simulation, image MSE as cost, Bayesian optimization for search—can perform one-parameter material inversion from purely visual data.

Load-bearing premise

The inversion only works if the MPM simulator with a single unknown friction angle exactly reproduces the observed motion and the plow trajectory is known; the paper never tests what happens when real soil behavior departs from the simulator.

Editorial extensions

If this is right

Friction angle can be estimated from images alone with an error below 2 degrees, covering the lunar soil range of 30–50 degrees that motivated the work.
Because the pipeline is decoupled and requires no simulator gradients, it can be applied to non-differentiable or closed-source forward simulation codes.
Error and variance systematically increase with the ground truth angle, implying that high-friction soils produce subtler deformation differences that demand more evaluations or better observations.
The image loss is most sensitive near the plow and in the collapsing pile behind it, so future camera placements should target those regions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: the same loop should be testable in a laboratory sand tank with a known friction angle; a successful transfer would turn this into a tool for field geotechnics, not just simulation.
Inference: if two or more parameters (for example cohesion and dilation angle) are unknown, the Gaussian Process surrogate may need many more evaluations and the image loss may develop multiple minima, so the single-parameter success should not be assumed to generalize without further study.
Inference: replacing RGB loss with depth maps or silhouette overlap could remove lighting and texture sensitivity while preserving geometry sensitivity; this is a natural variant the paper does not explore.
Inference: the reported variance (up to ±1.36 degrees at 45 degrees) suggests the NeRF geometry mismatch dominates the residual loss; higher-fidelity reconstructions such as Mip-NeRF or 3D Gaussian Splatting should tighten the minimum and improve precision.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a clean, well-scoped proof-of-concept for the decoupled NeRF-MPM-Bayesian optimization pipeline, and the inverse-crime limitation is real but not disqualifying at this stage.

The genuinely useful contribution is the decoupling. By using Bayesian optimization instead of a differentiable simulator, the framework works with any forward model, including non-differentiable or closed-source ones. That is a practical point worth having on the record, and the paper makes it clearly. The synthetic experiments are also executed honestly: observations are generated by MPM, rendered, reconstructed with NeRF, and then inverted against. The image-loss maps in Figure 3 are sensible, and the reported MAEs (0.64–1.38°) are consistent with a working closed loop. The writing is straightforward and the authors do not overclaim beyond the synthetic setting.

The soft spot is exactly the one the stress-test flags: the observation generator and the candidate simulator are the same MPM model, so this is an inverse-crime test. The only mismatch is NeRF's geometric reconstruction, which the paper themselves call background noise—that does not exercise constitutive-model error, rendering mismatch, or camera calibration error. So the 2-degree accuracy is a self-consistency result, not a prediction about real soil. The authors implicitly concede this in the conclusion when they put real-world video in future work. That is not a fatal flaw for a proof-of-concept, but it should be stated as a limitation in the abstract and results, not just in the conclusion.

Practical gaps: no per-trial error table, no code or data release, and no details on BO acquisition, GP priors, MPM discretization, or NeRF training budget. All fixable, but required for independent verification. The citation pattern is fine and correctly distinguishes the work from PAC-NeRF and PIE-NeRF.

Who is this for: people building vision-based soil parameter estimation for rovers or field robots. I would send it to review and ask the authors to add validation that is not self-referential—either real footage, or a different simulator as the observation generator, or at least a clear analysis of what changes when model mismatch is present. I would cite it for the decoupled BO idea, not for the accuracy claim.

Referee Report

4 major / 4 minor

Summary. The paper presents a decoupled pipeline for estimating the internal friction angle of granular material from visual observations. The pipeline uses NeRF to reconstruct the initial 3D geometry of a sand surface from multi-view images, initializes MPM simulations with that geometry, runs simulations under candidate friction angles with a known plow trajectory, renders the simulated deformations from fixed cameras, and compares them with observed frames via an RGB image loss. Bayesian optimization with a Gaussian process surrogate and expected improvement is used to minimize the loss. The method is tested on synthetic observations generated by MPM simulations for ground-truth friction angles of 25, 30, 35, and 45 degrees. The reported mean absolute errors range from 0.64 to 1.38 degrees across ten independent trials per angle.

Significance. If the reported accuracy were shown to persist under realistic model mismatch, the proposed decoupled NeRF-MPM-BO framework would be a useful step toward vision-based geotechnical characterization for planetary exploration. The design is attractive because it decouples the neural reconstruction from the simulation, allowing non-differentiable or closed-source simulators, and Bayesian optimization is a sensible choice for expensive black-box forward models. However, the central evidence is a synthetic inverse-crime test: observations and candidate simulations come from the same MPM generator. The paper therefore demonstrates self-consistency of the pipeline rather than the ability to infer properties from real experimental images. The significance is accordingly conditional on future validation against independent physical data or at least a mismatched forward model.

major comments (4)

[Section 4, Results] The validation is an inverse-crime test. The observation data are stated to be 'generated from MPM simulations' and candidate runs use 'the same plow trajectory as in the observation case.' Thus the forward model used for inversion is exactly the generator of the observations, with only the friction angle varied. This setup excludes constitutive-model error, plow-trajectory uncertainty, camera-calibration error, and rendering mismatch. As a result, the abstract's claim that the approach is 'effective ... through purely visual observations' is not established for real imagery. The authors should either add an experiment where the observations come from a different forward model (e.g., a different constitutive model or a simulator with added noise) and demonstrate that the image-loss minimum still tracks the true parameter, or substantially weaken the abstract and conclusion claims to explicit self-consistency validation.
[Section 3.2 / Section 4, Bayesian Optimization] The Bayesian optimization description lacks reproducibility-critical details. The paper does not state the number of initial random evaluations, the search bounds beyond the '25 to 55' example for the 45-degree case, the GP kernel and hyperparameter settings, or whether the 10-evaluation budget includes the initial random sample. These details are needed to assess whether the reported MAEs depend on favorable initialization or tuning. Please provide a precise algorithm specification and, ideally, a sensitivity analysis of the BO hyperparameters.
[Section 4, NeRF reconstruction and Table 1] The influence of NeRF geometry error on the inversion is not quantified. The paper notes that material points are sampled from the NeRF-reconstructed geometry, which introduces 'background noise' in the image comparisons, but no metric is given for the geometry error (e.g., Chamfer distance or intersection-over-union against the true surface) and no experiment isolates this source of error. A comparison between inversions initialized from perfect geometry and inversions initialized from NeRF-reconstructed geometry would clarify whether NeRF accuracy is a limiting factor and would strengthen the claim that the framework is robust to reconstruction noise.
[Table 1, Results] The 'within 2 degrees' claim is supported only by the mean absolute error, not by the worst-case or distributional behavior. For the 45-degree case the standard deviation is 1.36 degrees, so if errors are roughly normal a non-negligible fraction of trials will exceed 2 degrees. Please report the maximum error across the ten trials, the number of trials within 2 degrees, and ideally the full distribution of recovered angles for each ground-truth value. Without this, the robustness statement is stronger than the data show.

minor comments (4)

[Abstract and Section 1] The text contains the typo 'an plow'; it should read 'a plow.'
[Table 1] The table header reads 'Ground T ruth' with a space inside 'Truth'; please correct it to 'Ground Truth (°).'
[References] The Mildenhall reference is duplicated in the citation format ('Issue: arXiv:2003.08934 arXiv: 2003.08934 [cs]'), and the Barron reference lists an extraneous 'In' before the conference name. Please clean up the bibliography formatting.
[Figure 4] The caption states 'The triangle marker represents the initial observation,' but the figure as described in the text appears to show only dots and a dashed line; please ensure the triangle marker is clearly visible or update the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the inversion is validated on same-generator synthetic data, which is a generalization limitation rather than a circular reduction.

full rationale

The paper's derivation chain is self-contained. The central inversion loop—reconstruct initial geometry with NeRF, run MPM simulations for candidate friction angles, render the simulations from the observation camera poses, and minimize RGB image loss via Bayesian optimization—does not define the unknown friction angle in terms of the data or in terms of the paper's own prior results. The ground-truth angle is the value used to generate the observed synthetic frames; it is not inserted into the loss or into the optimizer, and no equation in the paper equates the recovered angle to a fitted parameter by construction. The only substantive concern is that the observations are produced by the same MPM simulator used during inversion (Section 4: 'the observation data is generated from MPM simulations with sand friction angles of 25, 30, 35, and 45' and candidate runs use 'the same plow trajectory as in the observation case'). This is an inverse-crime setup: it validates the optimizer inside the model's own assumption set and does not test robustness to constitutive-model mismatch, rendering discrepancies, or camera calibration error. But that is a limitation for real-world generalization, not a circular derivation. The paper explicitly defers real-world testing to future work ('Future work will focus on testing real-world video'), and the NeRF-reconstructed initial geometry introduces a nontrivial, independently reported error source ('This discrepancy introduces background noise in all comparisons'). There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled in via citation. The reported mean absolute errors of 0.64–1.38 degrees are therefore evidence about the behavior of the proposed pipeline under its own forward model, not a claim that the answer is recoverable by definition. Score 0.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The ledger contains one fitted target parameter, the friction angle, and five domain assumptions about the forward model, trajectory, NeRF initialization, material parameters, and camera calibration. No new physical entities are introduced. The central claim rests on the assumed fidelity of the MPM model as a generator of real observations, which is the main unvalidated premise.

free parameters (1)

friction_angle_phi = estimated values around 25, 30, 35, 45 with MAE 0.64 to 1.38 degrees
The target parameter inferred by minimizing image loss via Bayesian optimization. It is fitted to synthetic observations rather than measured independently, so it functions as a fitted value in the inverse problem.

assumptions (5)

domain assumption The MPM forward model, with a single unknown friction angle and fixed other material parameters, is an exact generator of the observed soil motion.
Section 4 generates observations with MPM and Section 3.2/4 uses the same MPM for candidate simulations. This is an inverse-crime assumption and is not checked against physical experiments.
domain assumption The plow trajectory and all boundary conditions are known exactly and are replayed identically in the candidate simulations.
Section 4 states 'applying the same plow trajectory as in the observation case'. In real rover scenarios, the exact soil-plow interaction forces and trajectory would likely be uncertain.
domain assumption The NeRF-reconstructed geometry is accurate enough to initialize MPM without biasing the friction angle estimate.
The paper acknowledges in Section 4 that NeRF geometry differs from the original simulation and introduces background noise; the inversion assumes this noise does not overwhelm the friction-angle signal.
domain assumption MPM material parameters other than friction angle, such as density, stiffness, and cohesion, are known or fixed to values that reproduce the observations.
These parameters are not stated in the paper, yet they must be held fixed while friction angle varies; otherwise the inversion would not be identifiable.
domain assumption Camera poses and rendering photometry are known and calibrated for both NeRF reconstruction and simulation rendering.
NeRF requires known camera poses and the renderer assumes the same camera setup. Real rover images would need calibration that is not discussed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From images to properties: a NeRF-driven framework for granular material parameter inversion." pith.science (2026). https://pith.science/paper/EQII74LY

@misc{pith2026250709005,
  author       = {Pith},
  title        = {Pith review of: From images to properties: a NeRF-driven framework for granular material parameter inversion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EQII74LY}},
  note         = {Machine review of arXiv:2507.09005}
}

read the original abstract

We introduce a novel framework that integrates Neural Radiance Fields (NeRF) with Material Point Method (MPM) simulation to infer granular material properties from visual observations. Our approach begins by generating synthetic experimental data, simulating an plow interacting with sand. The experiment is rendered into realistic images as the photographic observations. These observations include multi-view images of the experiment's initial state and time-sequenced images from two fixed cameras. Using NeRF, we reconstruct the 3D geometry from the initial multi-view images, leveraging its capability to synthesize novel viewpoints and capture intricate surface details. The reconstructed geometry is then used to initialize material point positions for the MPM simulation, where the friction angle remains unknown. We render images of the simulation under the same camera setup and compare them to the observed images. By employing Bayesian optimization, we minimize the image loss to estimate the best-fitting friction angle. Our results demonstrate that friction angle can be estimated with an error within 2 degrees, highlighting the effectiveness of inverse analysis through purely visual observations. This approach offers a promising solution for characterizing granular materials in real-world scenarios where direct measurement is impractical or impossible.

Figures

Figures reproduced from arXiv: 2507.09005 by the authors.

**Figure 2.** Overview of Neural Radiance Field 3.2 Bayesian Optimization Bayesian Optimization (BO) is used to estimate the friction angle by minimizing the image loss between simulated and observed scenes. Since each simulation is computationally expensive, BO builds a surrogate model to guide the search efficiently. The process begins by evaluating one or a few randomly selected friction angles and recording their image losses… view at source ↗

**Figure 3.** Image comparison between observed and simulated scenes at frames 100 and 199 from [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Bayesian optimization result for ϕtrue = 45◦ . friction angle estimation with purely visual input. In our synthetic experiments, estimated angles remained within 2° of ground truth across multiple test cases. This method has strong implications for field applications w…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

7 extracted references · 6 canonical work pages

[1]

T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., and Srinivasan, P

Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., and Srinivasan, P. P. (2021). Mip- NeRF : A Multiscale Representation for Anti - Aliasing Neural Radiance Fields . In 2021 IEEE / CVF International Conference on Computer Vision ( ICCV ) , pages 5835--5844, Montreal, QC, Canada. IEEE

work page 2021
[2]

Feng, Y., Shang, Y., Li, X., Shao, T., Jiang, C., and Yang, Y. (2023). PIE - NeRF : Physics -based Interactive Elastodynamics with NeRF . arXiv preprint arXiv:2311.13099

work page Pith review arXiv 2023
[3]

Kerbl, B., Kopanas, G., Leimkuehler, T., and Drettakis, G. (2023). 3D Gaussian Splatting for Real - Time Radiance Field Rendering . ACM Transactions on Graphics , 42(4):1--14

work page 2023
[4]

Y., Jatavallabhula, K

Li, X., Qiao, Y.-L., Chen, P. Y., Jatavallabhula, K. M., Lin, M., Jiang, C., and Gan, C. (2023). PAC - NeRF : Physics Augmented Continuum Neural Radiance Fields for Geometry - Agnostic System Identification . In International Conference on Learning Representations

work page 2023
[5]

P., Tancik, M., Barron, J

Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. (2020). NeRF : Representing Scenes as Neural Radiance Fields for View Synthesis . Issue: arXiv:2003.08934 arXiv: 2003.08934 [cs]

arXiv 2020
[6]

K., Houston, W

Mitchell, J. K., Houston, W. N., Scott, R. F., Costes, N. C., Carrier, III, W. D., and Bromwell, L. G. (1972). Mechanical properties of lunar soil: Density , porosity, cohesion and angle of internal friction. Lunar and Planetary Science Conference Proceedings , 3:3235

work page 1972
[7]

Müller, T., Evans, A., Schied, C., and Keller, A. (2022). Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics , 41(4):1--15

work page 2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., and Srinivasan, P

Barron, J. T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., and Srinivasan, P. P. (2021). Mip- NeRF : A Multiscale Representation for Anti - Aliasing Neural Radiance Fields . In 2021 IEEE / CVF International Conference on Computer Vision ( ICCV ) , pages 5835--5844, Montreal, QC, Canada. IEEE

work page 2021

[2] [2]

Feng, Y., Shang, Y., Li, X., Shao, T., Jiang, C., and Yang, Y. (2023). PIE - NeRF : Physics -based Interactive Elastodynamics with NeRF . arXiv preprint arXiv:2311.13099

work page Pith review arXiv 2023

[3] [3]

Kerbl, B., Kopanas, G., Leimkuehler, T., and Drettakis, G. (2023). 3D Gaussian Splatting for Real - Time Radiance Field Rendering . ACM Transactions on Graphics , 42(4):1--14

work page 2023

[4] [4]

Y., Jatavallabhula, K

Li, X., Qiao, Y.-L., Chen, P. Y., Jatavallabhula, K. M., Lin, M., Jiang, C., and Gan, C. (2023). PAC - NeRF : Physics Augmented Continuum Neural Radiance Fields for Geometry - Agnostic System Identification . In International Conference on Learning Representations

work page 2023

[5] [5]

P., Tancik, M., Barron, J

Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. (2020). NeRF : Representing Scenes as Neural Radiance Fields for View Synthesis . Issue: arXiv:2003.08934 arXiv: 2003.08934 [cs]

arXiv 2020

[6] [6]

K., Houston, W

Mitchell, J. K., Houston, W. N., Scott, R. F., Costes, N. C., Carrier, III, W. D., and Bromwell, L. G. (1972). Mechanical properties of lunar soil: Density , porosity, cohesion and angle of internal friction. Lunar and Planetary Science Conference Proceedings , 3:3235

work page 1972

[7] [7]

Müller, T., Evans, A., Schied, C., and Keller, A. (2022). Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics , 41(4):1--15

work page 2022