Pith. sign in

REVIEW 3 major objections 6 minor 48 references

Objects as Audio-Visual Modal Sound Fields

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that an object's full impact-sound field can be reconstructed from multi-view images and a few impact recordings, and that the resulting representation enables novel-location synthesis, contact localization, and material…

desk verdict A novel few-shot representation for object impact sound fields, but the SOTA claim is undercut by per-object model selection on a reported metric. read the letter →

arxiv 2608.05145 v2 pith:UEN7XH6Q submitted 2026-08-05 cs.CV cs.MMcs.SD

Zisen Shao , Zihao Wei , Derong Jin , Ruohan Gao This is my paper · ORCID
classification cs.CVcs.MMcs.SD
keywords impactsoundsynthesismodalanalysisaudio-visualrepresentation3DGaussianSplattingfew-shotlearningcontactlocalizationeditingmultisensoryobjects
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that an object's impact sound at any contact point can be reconstructed from multi-view images and only a handful of impact recordings. The proposed representation, the Audio-Visual Modal Sound Field (AV-MSF), couples 3D Gaussian Splatting and dense visual features with physically meaningful modal parameters, so the model can generalize across the object's surface. The authors report better novel-position impact sound rendering than both physics-based inverse rendering and data-driven baselines on two real-world datasets, and they use the same representation to localize an impact from its sound and to edit an object's material sound from text. A sympathetic reader would care because it points toward practical few-shot multisensory object capture: a few taps with a hammer plus ordinary photos could digitize not just how objects look but how they sound.

What carries the argument

The central object is the Audio-Visual Modal Sound Field, written as $F=(\{f_i,d_i\}_{i=1}^N,\{m_i\}_{i=1}^F,G_\theta(x))$, where $f_i$ and $d_i$ are object-wide modal frequencies and dampings, $m_i$ are residual noise magnitudes, and $G_\theta(x)$ is a neural gain field returning each mode's excitation gain at a surface point $x$. The gain field is conditioned on region-level DINOv2 descriptors through dot-product attention, the fine-grained feature of the nearest Gaussian, and positional offsets, so visual similarity across the object carries the generalization from recorded to unrecorded contact points. This machinery makes the full sound synthesis differentiable, allowing two-stage optimization: a warm-up on extracted gains followed by end-to-end multi-scale STFT loss minimization over only a few contacts.

What would settle it

Take an object whose surface looks uniform but whose interior is not, such as a hollow box with one solid partition painted identically, and tap two visually identical points over the hollow and solid parts. If the mode gains AV-MSF predicts from appearance are nearly equal while the recorded gains differ well beyond the model's training residual, then the visual-prior assumption fails and the gain field will not generalize to that kind of novel location.

Watch

Extended reading notes

Core claim

The central claim is that an object's impact-sound field factorizes into global, position-invariant modal frequencies and dampings plus a position-dependent excitation gain field, and that this gain field can be predicted from local visual appearance and geometry. The paper reconstructs a 3D Gaussian Splatting model from RGB views, lifts DINOv2 visual features into a dense 3D feature field, refines those features with symmetry-aware alignment, and uses attention over region descriptors together with the nearest Gaussian's local feature to predict modal gains at any query point. The sparse impact recordings supply the global modal parameters, a warm-up stage anchors the gain field, and a filtered-noise residual absorbs environmental effects. On ObjectFolder Real and RealImpact the method outperforms DiffSound and SonicGauss on both datasets, with the largest gains at low shot counts and on non-symmetric objects, and it enables contact localization and text-driven material sound editing.

Load-bearing premise

The load-bearing premise is that similar-looking and similarly curved regions of an object vibrate in similar ways, so visual features predict which vibration modes a tap will excite at any point.

Editorial extensions

If this is right

  • With roughly 4 to 8 impact recordings per object, AV-MSF renders novel-location impact sounds with lower L1, log-spectral, envelope, and perceptual error than the physics-based and data-driven baselines tested.
  • On non-symmetric objects, where nearest-neighbor interpolation is weakest, the visual-conditioned gain field gives consistent improvements, indicating the method learns spatial acoustic variation rather than simply copying nearby recordings.
  • Contact localization from a single novel impact recording becomes possible by matching mode-specific gains to the learned gain field, with lower localization error than the physics-based baseline on the non-symmetric evaluation subset.
  • Text-driven material edits, such as changing ceramic to wood, can be applied by updating modal frequencies, dampings, and per-mode gain scales while leaving gain ratios across locations invariant, so the edited sound field remains physically consistent.
  • Training cost is modest and inference is real-time, so the representation is practical for object-level digitization pipelines rather than offline simulation only.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, if visual similarity truly predicts excitation gains, then a gain field trained on one object could plausibly initialize another object of similar shape, potentially cutting the required recordings to one or two taps.
  • The paper explicitly defers objects with non-uniform materials; a natural extension is an object like a ceramic mug with a wooden handle, where the visual prior predicts different gains in different regions and the residual noise model may need to become spatially varying.
  • The invariant gain-ratio property used for editing implies a stronger statement the paper leaves implicit: global material edits should preserve contact-localization accuracy, because cosine-distance matching uses relative gains rather than absolute magnitudes.
  • A practical testable consequence is that AV-MSF should fail predictably on visually identical but structurally different regions, which means the representation's error map could be used to flag exactly where internal structure is not visible from the surface.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Audio-Visual Modal Sound Field (AV-MSF), an object-level representation for impact sound rendering. Given multi-view RGB images and a small number of impact recordings, the method builds a 3D Gaussian Splatting scene, lifts DINOv2 features into a 3D feature field, and predicts position-dependent modal gains through an attention-based neural field, while modal frequencies, dampings, and a residual noise component are extracted from the recordings. The resulting representation is evaluated on ObjectFolder Real and RealImpact for novel-position impact sound synthesis, and is also used for contact localization and text-guided object sound editing. The central claims are that AV-MSF outperforms physics-based and data-driven baselines on two real-world datasets, and that the physics-based parameterization enables the two downstream applications.

Significance. If the empirical claims are established, this is a useful contribution to multisensory object modeling: it combines a physically interpretable modal-sound parameterization with a learned visual prior, and it addresses a few-shot regime in which data-driven baselines struggle. The task and representation are clearly motivated, and the ablation study in Table 2 provides evidence for the contributions of the visual features, initialization, symmetry alignment, and residual noise. I agree with the reader's assessment that the novel-position evaluation is not circular: gains are predicted at held-out positions, and the global modal parameters are extracted from training recordings. The main weakness is evidential rather than conceptual. The headline comparison is weakened by the model-selection protocol described in Supplement B, and several stated improvements are not uniformly supported by Table 1. The core idea is sufficiently promising that these issues may be fixable within a revision.

major comments (3)
  1. [Supplementary B (Implementation Details)] The paper states: 'For each object, we run both the single-damping and spatial-damping variants, and choose the best model based on the ENV metric.' Since ENV is one of the four metrics reported in Table 1 and no validation split is described for this selection, nor for the cluster count, learning rate, epochs, or cutoff-weight choices, the reported ENV values are the outcome of selection on the evaluation set rather than an estimate of generalization performance, and the other metrics in Table 1 come from the same selected checkpoints. This is load-bearing because the abstract's state-of-the-art claim rests on Table 1. Please either introduce a validation split disjoint from the test contacts and select all hyperparameters on it, or fix a single protocol and report means and standard deviations over multiple runs. With the current protocol, the comparison against KNN, DiffSound, and SonicGauss does not establish that AV-MSF is systematically better.
  2. [Table 1 and Sec. 4.3] On ObjectFolder Real, the KNN baseline achieves a better L1 Log value (0.930 vs. 0.951) and an equal ENV value (0.014 vs. 0.014) compared with AV-MSF; on RealImpact, AV-MSF is better, but the margins are small (e.g., L1 Log 0.996 vs. 1.036). The text in Sec. 4.3 says AV-MSF 'achieves better results on most metrics' and the Introduction claims a '2× improvement over both prior physics-based methods and data-driven baselines.' These statements are not uniformly supported: the 2× improvement holds only for L1 on ObjectFolder Real (0.013 vs. 0.031/0.033), not for the other metrics or for RealImpact. Please qualify the claims to the specific metrics and datasets where they hold, or provide error bars over objects to show that the differences are significant.
  3. [Sec. 4.2 (DiffSound baseline)] The sentence 'We further add a learned noise residual from our method for fairness' modifies the DiffSound baseline, but the modification is not specified: it is unclear whether the residual is learned per object, whether it is applied to all contact locations, whether it is drawn at inference or fitted once, and whether a similar residual is added to the other baselines. Because DiffSound is one of the two main competitors, please describe the exact protocol or remove the modification entirely.
minor comments (6)
  1. [Sec. 3.2] The text refers to 'Following Eq. (20)', but Eq. (20) first appears in Supplement A.2; the main-text synthesis equation is Eq. (3). Please fix the cross-reference.
  2. [Sec. 3.3] The symmetry detection step is described only as 'evaluating the geometric alignment error of candidate transformations,' with no objective, threshold, or optimization procedure; Supplement A.1 refers to 'off-the-shelf methods' without naming one. Please specify the exact detection method or provide a reference.
  3. [Sec. 3.4 / Eq. (6)] Because fresh white noise is drawn at synthesis time, the rendered waveform is stochastic; please state how many noise draws are used when computing the metrics in Table 1 and whether the baselines are evaluated with the same degree of stochasticity.
  4. [Supplement D.2 / Table 5] The editing comparison uses 128 modes for AV-MSF versus 2048 modes for Audio-SDS, which the supplement states is due to computational constraints; please clarify whether the reported improvement comes from the hierarchical frequency parameterization or from the different mode count.
  5. [Table 4 and Fig. 4/7] The contact localization comparison reports a single RMED number per method; given the high variance visible in the qualitative heatmaps, please provide per-object results or error bars.
  6. [Fig. 2] The caption contains 'Project qk,v', which should presumably read 'Project q,k,v'; please proofread the figure and supplements for minor typos (e.g., 'these Gaussian centers is further considered' in Supplement A.1).

Circularity Check

1 steps flagged · score 4.0 of 10

Headline comparison is compromised by per-object model selection on the reported ENV metric; the core few-shot sound-field derivation is not circular.

  1. fitted input called prediction [Supplementary Sec. B (Training Setup Details); affects Table 1]
    "For each object, we run both the single-damping and spatial-damping variants, and choose the best model based on the ENV metric."

    ENV is also one of the four metrics reported in Table 1 for novel-position rendering. By choosing, per object, the variant with the best ENV on the evaluation set and then reporting that ENV as the method's performance, the reported ENV is by construction the minimum of the two variants' ENV values, not an unbiased estimate for a fixed method. The L1, L1-log, and CDPAM numbers are reported for those same ENV-selected checkpoints and inherit the selection. No separate validation split, error bars, or fixed selection protocol is described, so the Table 1 superiority claim is partly an artifact of optimizing the reported metric rather than an independent prediction.

full rationale

The central AV-MSF derivation is self-contained: modal parameters are extracted from training recordings, the gain field is optimized with a differentiable synthesizer against held-out-evaluated contacts, and novel-position synthesis is tested on contact points not used for training, so the few-shot prediction claim does not reduce to its inputs. Self-citations (DiffImpact for residual noise, DiffSound for material scaling observations, SonicGauss for the 3DGS reconstruction pipeline) are used for standard components and are not load-bearing for the main result; the audio-visual gain-field mechanism is introduced here. The contact-localization and sound-editing evaluations share the paper's own modal-parameter extractor, but this is a common preprocessing choice rather than a definitional equivalence: the editing loss is driven by Audio-SDS/CLAP and the localization metric is measured against ground-truth 3D coordinates. However, the headline SOTA comparison is weakened by the Supp. B protocol of selecting the best of two model variants on the very ENV metric that Table 1 reports; that makes one of the headline metrics an optimized selection statistic. This is a partial circularity in the evidence for the SOTA claim, while the core representation and few-shot synthesis pipeline remain independent.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard modal analysis plus several domain assumptions about how visual appearance relates to acoustic response. The free parameters are hyperparameters and thresholds that are tuned per object or left unspecified, making the exact configuration of the final model under-determined from the paper alone.

free parameters (5)
  • Number of modes N = not reported
    The modal sum in Eq. (4) sums over N modes, but the paper never states how N is chosen per object; it is a central capacity parameter for the representation.
  • Number of clusters K = selected from {128, 256, 512}
    Used to form region-level descriptors in Sec. 3.5; selected per object to balance efficiency and performance.
  • Learning hyperparameters = learning rate from {1e-2, 5e-3, 1e-3}; epochs from {5000, 10000, 20000}; cutoff loss weight from {0.5, 1, 2}
    Chosen per object in Supp. B; affects optimization and final rendering quality.
  • Noise filtering thresholds = not specified
    Sec. 3.4 discards peaks that 'occur beyond a time threshold or are not significantly damped', but the thresholds are never given, making the modal extraction pipeline under-specified.
  • Damping variant choice = single-damping or spatial-damping per object
    Supp. B: 'we run both the single-damping and spatial-damping variants, and choose the best model based on the ENV metric.' This is a per-object model selection.
assumptions (6)
  • domain assumption Impact sound is a sum of exponentially damped sinusoids (linear modal analysis).
    Eqs. (3) and (4) model s(t) as the sum of N modes; this is the standard modal synthesis model, but it is an assumption that real impact sounds, including non-modal noise, can be captured this way.
  • domain assumption Modal frequencies and dampings are position-invariant object properties.
    Sec. 3.1 states this assumption and the paper later relaxes damping to be spatially varying for some objects. The central formulation in Eq. (4) relies on this.
  • domain assumption Visually and geometrically similar object regions produce similar modal excitation gains.
    The Introduction asserts 'impact sounds vary predictably across an object and are strongly correlated with local visual and geometric cues'; this motivates the use of DINOv2 features and geometric offsets for gain prediction in Sec. 3.5.
  • domain assumption Objects exhibit detectable rotational or mirror symmetries that can be used to refine lifted features.
    Sec. 3.3 and Supp. A.1 automatically detect symmetries and pool features across symmetric counterparts; the benefit depends on the validity of this assumption for the tested objects.
  • domain assumption Non-modal residual can be modeled as static filtered noise.
    Sec. 3.4 models residual as white noise shaped by learnable per-band magnitudes; this is a simplifying assumption that may not hold for all recording conditions.
  • standard math Under material scaling, modal mode shapes remain unchanged and gains rescale inversely with frequency.
    Supp. A.2 derives this from the generalized eigenproblem with M proportional to density and K to Young's modulus; it is standard modal analysis, used to justify the editing gain rescaling in Eq. (21).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Objects as Audio-Visual Modal Sound Fields." pith.science (2026). https://pith.science/paper/UEN7XH6Q

@misc{pith2026260805145,
  author       = {Pith},
  title        = {Pith review of: Objects as Audio-Visual Modal Sound Fields},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UEN7XH6Q}},
  note         = {Machine review of arXiv:2608.05145}
}
read the original abstract

While modern 3D reconstruction excels at modeling object geometry and appearance, it largely ignores the rich acoustic cues revealed through physical interaction. Object impact sounds convey material, stiffness, and structural properties that complement vision, yet existing impact sound modeling approaches either rely on expensive physics-based simulation or require large datasets to generalize in a purely data-driven manner. We introduce Audio-Visual Modal Sound Field (AV-MSF), a novel object-level acoustic representation reconstructed from multi-view images and only a few impact sound recordings. AV-MSF builds on 3D Gaussian Splatting integrated with dense 3D visual feature to provide a strong geometry-aware prior, and represents the impact sound field using compact, physically meaningful modal parameters, enabling robust few-shot reconstruction. Experiments on two real-world datasets show that AV-MSF achieves state-of-the-art impact sound rendering, outperforming both physics-based and data-driven baselines. Furthermore, we demonstrate downstream applications enabled by our representation, including contact localization and object sound editing.

Figures

Figures reproduced from arXiv: 2608.05145 by the authors.

Figure 1
Figure 1. Left: We reconstruct the Audio-Visual Modal Sound Field (AV-MSF) from multi-view RGB observations and only a few impact recordings, leveraging the insight that visually similar object regions exhibit similar vibration patterns. Right: AV-MSF enables diverse applications, including novel-position impact sound synthesis, contact localization, and object sound editing. appearance from images, enabling downstream applic… view at source ↗
Figure 2
Figure 2. Overview of AV-MSF. Given multi-view image observations and few-shot impact recordings, our framework reconstructs an audio-visual modal sound field. (1) Visual Processing: We extract dense features with a pre-trained vision encoder and lift them into a 3D Gaussian Splatting (3DGS) representation to form a geometry-aware visual feature field. (2) Acoustic Processing: We extract modal parameters from a few reference … view at source ↗
Figure 3
Figure 3. Qualitative Spectrogram Comparisons. We visualize the rendered impact sounds from DiffSound [19], SonicGauss [42], and our method, alongside the ground truth. All spectrograms are generated on the settings of 20–20k Hz, the first second, scaled to (−1, 1). See Supp. for more examples [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Examples for Downstream Applications. Left [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Illustration of Objects in the ObjectFolder Real Dataset [12]. Moreover, for two impact locations x and y under the same direction d, the gain ratio ak(x) ak(y) = ϕk(x) ⊤dx ϕk(y)⊤dy is invariant to such parameter editing, since the frequency-dependent factor cancels ou…
Figure 6
Figure 6. Figure 6: Spectrogram Comparisons. We provide additional spectrogram compar￾isons between the baselines and our method. All spectrograms are generated using the same settings as in the main paper. In most cases, our method yields more accurate and sharper reconstructions. Two re…
Figure 7
Figure 7. Figure 7: Contact Localization Heatmaps. Predicted regions are shown with red indicating high likelihood and blue indicating low likelihood. Objects 7, 8, 9, and 10 show failure cases. and larger objects (Objects 2 and 3), impact sounds exhibit more diverse patterns that better …
Figure 8
Figure 8. Figure 8: CLAP Score Progression. Semantic alignment consistently improves during editing. – CLAP Score: Measures the semantic alignment between the edited impact sound and the target text prompt (higher is better). – UMAP Distance: Quantifies how accurately the edited frequenci…
Figure 9
Figure 9. Figure 9: Sound Editing in UMAP Space. Modal frequencies consistently shift to￾ward target material clusters post-editing [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 1 linked inside Pith

  1. [1]

    In: ICCV (2021)

    Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., Srini- vasan, P.P.: Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In: ICCV (2021)

    2021

  2. [2]

    In: CVPR (2023)

    Clarke, S., Gao, R., Wang, M., Rau, M., Xu, J., Wang, J.H., James, D.L., Wu, J.: Realimpact: A dataset of impact sound fields for real objects. In: CVPR (2023)

    2023

  3. [3]

    In: CoRL (2021)

    Clarke, S., Heravi, N., Rau, M., Gao, R., Wu, J., James, D., Bohg, J.: Diffimpact: Differentiable rendering and identification of impact sounds. In: CoRL (2021)

    2021

  4. [4]

    In: ICCV (2025)

    Clarke, S., Wistreich, S., Ze, Y., Wu, J.: X-capture: An open-source portable device for multi-sensory learning. In: ICCV (2025)

    2025

  5. [5]

    Presence (2007)

    Corbett, R., van den Doel, K., Lloyd, J.E., Heidrich, W.: Timbrefields: 3d interac- tive sound models for real-time audio. Presence (2007)

    2007

  6. [6]

    SIGGRAPH (1999)

    Curless, B.: From range scans to 3d models. SIGGRAPH (1999)

    1999

  7. [7]

    In: SIGGRAPH (2001)

    van den Doel, K., Kry, P.G., Pai, D.K.: Foleyautomatic: physically-based sound effects for interactive simulation and animation. In: SIGGRAPH (2001)

    2001

  8. [8]

    In: CVPR (2025)

    Dou, Y., Oh, W., Luo, Y., Loquercio, A., Owens, A.: Hearing hands: Generating sounds from physical interactions in 3d scenes. In: CVPR (2025)

    2025

Show all 48 references
  1. [9]

    In: CVPR (2024)

    Dou, Y., Yang, F., Liu, Y., Loquercio, A., Owens, A.: Tactile-augmented radiance fields. In: CVPR (2024)

    2024

  2. [10]

    CVPR (2025)

    Fan, J., Luan, F., Yang, J., Hasan, M., Wang, B.: Rng: Relightable neural gaus- sians. CVPR (2025)

    2025

  3. [11]

    In: CoRL (2021)

    Gao, R., Chang, Y.Y., Mall, S., Fei-Fei, L., Wu, J.: Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations. In: CoRL (2021)

    2021

  4. [12]

    In: CVPR (2023)

    Gao, R., Dou, Y., Li, H., Agarwal, T., Bohg, J., Li, Y., Fei-Fei, L., Wu, J.: The objectfolder benchmark: Multisensory learning with neural and real objects. In: CVPR (2023)

    2023

  5. [13]

    In: CVPR (2022)

    Gao, R., Si, Z., Chang, Y.Y., Clarke, S., Bohg, J., Fei-Fei, L., Yuan, W., Wu, J.: Objectfolder 2.0: A multisensory object dataset for sim2real transfer. In: CVPR (2022)

    2022

  6. [14]

    In: ICCV (2007)

    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: ICCV (2007)

    2007

  7. [15]

    In: CVPR (2024)

    Guédon, A., Lepetit, V.: Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In: CVPR (2024)

    2024

  8. [16]

    In: CoRL (2025)

    Higuera, C., Sharma, A., Fan, T., Bodduluri, C.K., Boots, B., Kaess, M., Lambeta, M., Wu, T., Liu, Z., Hogan, F.R., et al.: Tactile beyond pixels: Multisensory touch representations for robot manipulation. In: CoRL (2025)

    2025

  9. [17]

    In: SIGGRAPH (2024)

    Huang, B., Yu, Z., Chen, A., Geiger, A., Gao, S.: 2d gaussian splatting for geo- metrically accurate radiance fields. In: SIGGRAPH (2024)

    2024

  10. [18]

    In: ACM SIGGRAPH 2016 Courses (2016)

    James, D.L., Langlois, T.R., Mehra, R., Zheng, C.: Physically based sound for computer animation and virtual environments. In: ACM SIGGRAPH 2016 Courses (2016)

    2016

  11. [19]

    In: SIGGRAPH (2024)

    Jin, X., Xu, C., Gao, R., Wu, J., Wang, G., Li, S.: Diffsound: Differentiable modal sound rendering and inverse rendering for diverse inference tasks. In: SIGGRAPH (2024)

    2024

  12. [20]

    TOG (2023)

    Kerbl, B., Kopanas, G., Leimkuehler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. TOG (2023)

    2023

  13. [21]

    RAL (2025) Objects as Audio-Visual Modal Sound Fields 17

    Lee, M., Yoo, U., Oh, J., Ichnowski, J., Kantor, G., Kroemer, O.: Sonicboom: Contact localization using array of microphones. RAL (2025) Objects as Audio-Visual Modal Sound Fields 17

    2025

  14. [22]

    In: SIGGRAPH (2000)

    Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginz- ton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital michelangelo project: 3d scanning of large statues. In: SIGGRAPH (2000)

    2000

  15. [23]

    In: CoRL (2022)

    Li, H., Zhang, Y., Zhu, J., Wang, S., Lee, A.M., Xu, H., Adelson, E., Li, F.F., Gao, R., Wu, J.: See, hear, and feel: Smart sensory fusion for robotic manipulation. In: CoRL (2022)

    2022

  16. [24]

    In: ICML (2025)

    Li, T., Huang, B., Zhuang, X., Jia, D., Chen, J., Wang, Y., Chen, Z., Anu- manchipalli, G., Wang, Y.: Sounding that object: Interactive object-aware image to audio generation. In: ICML (2025)

    2025

  17. [25]

    Li, Y., Kim, H., Zhan, F., Qiu, R.Z., Ji, M., Shan, X., Zou, X., Liang, P., Pfister, H., Wang, X.: Visual acoustic fields (2025)

    2025

  18. [26]

    In: ICRA (2015)

    Luo, S., Mou, W., Althoefer, K., Liu, H.: Localizing the object contact through matching tactile features with visual map. In: ICRA (2015)

    2015

  19. [27]

    In: ICASSP (2021)

    Manocha, P., Jin, Z., Zhang, R., Finkelstein, A.: Cdpam: Contrastive learning for perceptual audio similarity. In: ICASSP (2021)

    2021

  20. [28]

    In: ECCV (2020)

    Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. In: ECCV (2020)

    2020

  21. [29]

    In: ACM SIGGRAPH/Eurographics Symposium on Computer Ani- mation (2002)

    O’Brien, J.F., Shen, C., Gatchalian, C.M.: Synthesizing sounds from rigid-body simulations. In: ACM SIGGRAPH/Eurographics Symposium on Computer Ani- mation (2002)

    2002

  22. [30]

    TMLR (2024)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H.V., Szafraniec, M., Khalidov, V., Fernandez, P., HAZIZA, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.Y., Li, S.W., Misra, I., Rabbat, M., Sharma, V., Synnaeve, G., Xu, H., Jegou, H., Mairal, ...

    2024

  23. [31]

    In: CVPR (2016)

    Owens, A., Isola, P., McDermott, J., Torralba, A., Adelson, E.H., Freeman, W.T.: Visually indicated sounds. In: CVPR (2016)

    2016

  24. [32]

    In: SIGGRAPH (2001)

    Pai, D.K., Doel, K.v.d., James, D.L., Lang, J., Lloyd, J.E., Richmond, J.L., Yau, S.H.: Scanning physical interaction behavior of 3d objects. In: SIGGRAPH (2001)

    2001

  25. [33]

    Pang, B., Xu, C., Ren, J., Wang, G., Li, S.: Vibraverse: A large-scale geometry- acoustics alignment dataset for physically-consistent multimodal learning (2025)

    2025

  26. [34]

    Qi, Z., Yang, Y., Zhang, M., Xing, L., Wu, X., Wu, T., Lin, D., Liu, X., Wang, J., Zhao, H.: Tailor3d: Customized 3d assets editing and generation with dual-side images (2024)

    2024

  27. [35]

    TOG (2013)

    Ren, Z., Yeh, H., Lin, M.C.: Example-guided physically based modal sound syn- thesis. TOG (2013)

    2013

  28. [36]

    arXiv preprint arXiv:2505.04621 (2025)

    Richter-Powell, J., Torralba, A., Lorraine, J.: Score distillation sampling for audio: Source separation, synthesis, and beyond. arXiv preprint arXiv:2505.04621 (2025)

    2025 arXiv

  29. [37]

    In: CVPR (2016)

    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: CVPR (2016)

    2016

  30. [38]

    In: ICCV (2023)

    Sella, E., Fiebelman, G., Hedman, P., Averbuch-Elor, H.: Vox-e: Text-guided voxel editing of 3d objects. In: ICCV (2023)

    2023

  31. [39]

    In: NeurIPS (2020)

    Smith, E.J., Calandra, R., Romero, A., Gkioxari, G., Meger, D., Malik, J., Drozdzal, M.: 3D shape reconstruction from vision and touch. In: NeurIPS (2020)

    2020

  32. [40]

    In: CVPR (2023)

    Su, K., Qian, K., Shlizerman, E., Torralba, A., Gan, C.: Physics-driven diffusion models for impact sound synthesis from videos. In: CVPR (2023)

    2023

  33. [41]

    In: ICRA (2022)

    Suresh, S., Si, Z., Mangelson, J.G., Yuan, W., Kaess, M.: ShapeMap 3-D: Efficient shape mapping through dense touch and vision. In: ICRA (2022)

    2022

  34. [42]

    In: ACMMM (2025) 18 Z

    Wang, C., Li, H., Luo, Y.: Sonicgauss: Position-aware physical sound synthesis for 3d gaussian representations. In: ACMMM (2025) 18 Z. Shao et al

    2025

  35. [43]

    In: NeurIPS (2021)

    Wang, P., Liu, L., Liu, Y., Theobalt, C., Komura, T., Wang, W.: Neus: learning neural implicit surfaces by volume rendering for multi-view reconstruction. In: NeurIPS (2021)

    2021

  36. [44]

    In: ICLR (2026)

    Ye, J., Xie, S., Zhao, R., Wang, Z., Yan, H., Zu, W., Ma, L., Zhu, J.: NANO3d: A training-free approach for efficient 3d editing without masks. In: ICLR (2026)

    2026

  37. [45]

    TMLR (2023)

    Yu, H.X., Guo, M., Fathi, A., Chang, Y.Y., Chan, E.R., Gao, R., Funkhouser, T., Wu, J.: Learning object-centric neural scattering functions for free-viewpoint relighting and scene composition. TMLR (2023)

    2023

  38. [46]

    TOG (2021)

    Zhang, X., Srinivasan, P.P., Deng, B., Debevec, P., Freeman, W.T., Barron, J.T.: Nerfactor: neural factorization of shape and reflectance under an unknown illumi- nation. TOG (2021)

    2021

  39. [47]

    In: NeurIPS (2017)

    Zhang, Z., Li, Q., Huang, Z., Wu, J., Tenenbaum, J.B., Freeman, W.T.: Shape and material from sound. In: NeurIPS (2017)

    2017

  40. [48]

    wood to ceramic

    Zhao, X., Srinivasan, P.P., Verbin, D., Park, K., Martin-Brualla, R., Henzler, P.: Illuminerf: 3d relighting without inverse rendering. In: NeurIPS (2024) Objects as Audio-Visual Modal Sound Fields 19 In this supplementary material, we first expand on our method formulations i...

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.