{"id":"7954ea94-7958-4b7d-8ff1-0b5760487391","arxiv_id":"2502.08297","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A pipeline that adds physically-based surface materials to dynamic 4D Gaussians, producing relightable volumetric video from multi-view RGB footage.","lead":"BEAM converts multi-view RGB video of a moving person into a relightable 4D Gaussian model with surface materials: roughness, ambient occlusion, and base color. The resulting asset works in real-time game engines and offline ray tracing, which matters for virtual production, VR, and telepresence.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation 5's base-color solve treats every non-specular contribution as direct environment lighting; interreflection omitted from Eqs. 6-8 is absorbed into base color and AO, undermining the claim that the extracted materials are intrinsic and reliably relightable.","rationale":"The reader's weakest assumption already identifies the omission of indirect illumination in Eqs. 5-8, and I agree that this is the most load-bearing concern. The central claim is that BEAM decouples intrinsic PBR properties from multi-view RGB; the base-color solve in Eq. 5 is the step that converts assumed direct-only outgoing radiance into an intrinsic albedo. Any radiance that reaches the surface after one or more bounces is necessarily misattributed, and the paper's own Limitations section acknowledges the approximation without quantifying it. I also weighed two other candidates: the unvalidated CLAY-generated roughness and the internal inconsistency in the user-study percentages. Both are legitimate reporting issues, but they are less central because they concern auxiliary components or evidence quality rather than the correctness of the core decomposition identity. The paper's math is internally consistent and mostly standard rendering-equation manipulation; I found no destructive algebraic error. If the proposed synthetic test shows large material errors under full transport, the conditional verdict is clearly correct and should perhaps be strengthened toward rejection of the 'accurate decoupling' claim until the method is shown to handle interreflection or explicitly scope it away. If the test shows small errors in the capture configuration, the concern lands only weakly and the existing conditional verdict remains appropriate on reproducibility grounds.","tokens_in":15679,"tokens_out":9344,"duration_ms":112984,"concrete_test":"Render a synthetic human from RenderPeople with known albedo, AO, and roughness maps and a known HDR environment in Cycles twice: once with direct illumination only (zero bounces) and once with full path tracing. Run the paper's material-decomposition pipeline (Eqs. 4-8) on both renders using the same known environment map and the same estimated geometry and roughness. Compare recovered base color and AO maps against ground truth, separately for concave and occluded regions. If the full path-traced variant shifts mean base-color error by more than about 10% or AO error by more than 0.1 in those regions, the interreflection approximation materially threatens the central claim; if the shifts stay within denoiser noise, the concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In the central material decomposition, base color is solved from Eq. 5 as rho = (L_o - L_S_o)/L_D_o after Monte Carlo evaluation of Eqs. 6 and 7. Those integrals only account for direct illumination: L(x,omega_i) = V_env(x,omega_i)L_env(omega_i). Real multi-view captures of a human in a dome contain significant interreflection, especially in concave regions such as armpits, clothing folds, and under the chin. All multiply-scattered radiance arriving at x contributes to the measured numerator L_o but is absent from the computed denominator L_D_o, so the solver bakes that light into rho, and through the coupled AO optimization, into A. The paper explicitly concedes in Sec. 6 that the rendering equation is approximated and that material results contain errors, but it does not quantify the effect. This matters directly for the headline claim: if decoupled materials encode the capture environment's indirect transport rather than intrinsic albedo and occlusion, re-rendering under a novel HDRI will reproduce the wrong diffuse response, weakening the 'CG-friendly relightable asset' claim. This is not a derivation error; the rendering-equation manipulation is standard. It is an unquantified failure mode of the central decomposition equation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents BEAM, a pipeline for producing relightable 4D Gaussian sequences from multi-view RGB input with known environment lighting. Geometry is obtained by combining DualGS-based performance tracking with RaDe-GS-style geometry-aware rasterization; roughness is generated by CLAY's material diffusion module; ambient occlusion and base color are estimated per input view by evaluating visibility with a Gaussian ray tracer (Eqs. 4-8) and solving a simplified Disney BRDF rendering equation (Eq. 5). The resulting Gaussians support deferred real-time rendering in Unity and offline ray tracing. Claims are supported by qualitative results on six captured performances, quantitative synthetic comparisons against Relightable-3DGS, GS-IR, and MeshAvatar (Table 1), a camera-count ablation (Table 2), and a user study.","tokens_in":15980,"tokens_out":4829,"duration_ms":53669,"significance":"If the material decomposition were validated, this would be a practically significant contribution: it would provide a CG-friendly, relightable volumetric-video workflow with real-time 1080p rendering at 100 FPS and compatibility with standard engines. The core equations are standard inverse rendering, and the pipeline choices are reasonable and clearly described. However, the central decomposition rests on assumptions that the paper itself concedes in Section 6: direct-only environment illumination and the use of a generative model for roughness. The quantitative validation is thin, and the reported user-study percentages are internally inconsistent. These gaps must be addressed before the intrinsic-material and relightability claims can be considered established.","major_comments":[{"comment":"The base-color solve in Eq. (5) uses L_D_o and L_S_o computed with L(x,omega_i)=V_env(x,omega_i)L_env(omega_i), which accounts only for direct environment light. In a real dome capture of a human, interreflections in concave regions such as armpits, clothing folds, and under the chin add multiply-scattered radiance to the measured numerator L_o while being absent from the computed denominator L_D_o. The solver therefore bakes that indirect radiance into rho and, through the coupled AO optimization, into A. This is an unquantified failure mode of the central decomposition; Section 6 explicitly admits the approximation but no experiment measures its magnitude. Please add a synthetic experiment with known interreflection (or otherwise quantify the effect) and show that the recovered base color and AO, as well as the relit output, remain correct in concave regions.","section":"Sec. 3.2, Eqs. (5)-(8); Sec. 6 Limitations"},{"comment":"Roughness is generated by the CLAY material diffusion module and assigned to each Gaussian by nearest-pixel UV projection. It is not derived from the captured multi-view images under the known environment map, nor is it validated against any measured reflectance. Since r enters the specular integral L_S_o in Eq. (7), and since Eq. (5) obtains base color by subtracting L_S_o from measured L_o, errors in the hallucinated roughness directly contaminate the recovered base color and AO. Please either validate the generated roughness against synthetic ground truth or provide a sensitivity analysis showing that plausible roughness errors have a small effect on the final relighting result.","section":"Sec. 3.2, 'Roughness'; Eq. (7)"},{"comment":"The quantitative evidence is thin and partly inconsistent. Table 1 reports metrics over only two synthetic sequences without standard deviations or per-sequence breakdowns, and metrics are computed only inside the human bounding box. The user study states that 30 users participated but reports 95.65% and 87% preference; with 30 users, 95.65% is not attainable, so either the participant count or the percentages need correction. Additionally, Table 2 reports an 'Ours' relighting PSNR of 26.21 while Table 1 reports an 'Ours' relighting PSNR of 26.57 for what appears to be the same 50-camera synthetic setup; the relationship between the two tables should be clarified. These issues undercut the claim of consistent superiority and should be fixed with proper statistics.","section":"Sec. 5.1 and 5.3, Tables 1-2 and user study"}],"minor_comments":[{"comment":"The text says 'E_norm is introduced' but the energy term defined above is E_normal; please unify the notation.","section":"Sec. 3.1, after Eq. (2)"},{"comment":"In the introduction, citations such as 'Li [43]' and 'High-quality FVV [9]' should use the same style as the rest of the text, adding 'et al.' where there are multiple authors.","section":"Sec. 1, related work citations"},{"comment":"The sentence 'initialize the AO attributes with zeros and the base color attributes with RGB values' is vague; please specify the initialization (for example, from the input-view SH color, from a per-view bake, or from a constant).","section":"Sec. 3.2, AO/base-color optimization"},{"comment":"The ablations for the ray-origin offset and the sampling strategy are qualitative only; adding quantitative metrics, even for a few frames, would make the reported choices easier to evaluate.","section":"Sec. 5.2, Figs. 7 and 8"}],"recommendation":"major_revision","confidential_remarks":"The paper is a competent systems integration with a strong practical story, but the novelty is incremental over DualGS, RaDe-GS, 3DGRT, and CLAY, and the validation is not yet commensurate with the central claim of decoupling intrinsic materials. The interreflection issue is the main technical risk; a synthetic test with known indirect illumination would either validate the approximation or reveal its scope. The user-study percentage inconsistency and the absence of error bars are straightforward to fix and should be before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"BEAM is a pragmatic, well-scoped systems paper: it stitches together DualGS tracking, RaDe-GS rasterization, a CLAY diffusion roughness prior, and 3DGRT ray tracing into a pipeline that outputs 4D Gaussians with baked PBR materials, drops the view-dependent SH colors, and integrates with Unity for real-time deferred shading and offline path tracing. That integration is the actual novelty, and it is a real contribution for virtual production and volumetric-video use cases. The qualitative results on six captures look convincing, and the pipeline description is clear enough to implement.\n\nThe math in Section 3.2 is standard inverse rendering. Equation 5 is a correct linear split for a dielectric, and using visibility from the Gaussian ray tracer to compute AO and base color is a rational design choice. The stress-test concern about interreflection is legitimate: Eqs. 6-8 only account for direct environment light, so bounce radiance from skin, clothing folds, and under-chin areas gets folded into base color and AO. The paper itself concedes this in Section 6, where it says the rendering equation is approximated and material results contain errors. That admission is honest, but it is not quantified, and the synthetic evaluation uses RenderPeople meshes that are comparatively smooth and low-interreflection. Real human performances in a dome are exactly where the approximation would bite. So the headline claim of a 'CG-friendly relightable asset' is somewhat overstated.\n\nOther soft spots: quantitative evidence is thin (two synthetic sequences, no error bars, metrics restricted to the human bounding box), the user-study percentages (95.65% and 87%) do not match integer denominators for 30 users, and the diffusion-based roughness is not validated against any physical measurement. No code or data is released, which limits reproducibility. These are normal issues for an applied graphics paper, not fatal flaws.\n\nThis paper is for practitioners and researchers in human performance capture, relighting, and real-time rendering. It deserves a serious peer review; it is the kind of integration paper that gets cited as a baseline for practical relightable Gaussian pipelines. If I were refereeing, I would ask for a relighting test under a novel HDRI that includes indirect illumination, a comparison of the diffusion roughness against a measured or fitted counterpart, and a cleaned-up user study.","headline":"BEAM's real contribution is the pipeline integration for CG-friendly relightable 4D Gaussians, but the material decomposition has an unquantified interreflection error that the paper admits but does not evaluate.","tokens_in":16519,"tokens_out":2859,"would_cite":false,"duration_ms":28679,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BEAM turns multi-view RGB footage of a moving person into relightable 4D Gaussians that render in standard CG engines.","keywords":["volumetric video","relightable 4D Gaussians","physically-based rendering","material decomposition","Gaussian ray tracing","ambient occlusion","human performance capture","deferred shading"],"falsifier":"Run BEAM on a synthetic human with a known ground-truth base color and a deliberately strong self-occluding pose (for example, forearms held close to the chest) under an environment map containing a bright colored region; if the recovered base color or ambient occlusion in shadowed areas changes when that bright region is moved or recolored, the distant-light, no-interreflection assumption is falsified.","tokens_in":1898,"feed_emoji":"🎥","tokens_out":2443,"duration_ms":96314,"temperature":0.7,"pith_summary":"BEAM is a pipeline that produces relightable volumetric video from multi-view RGB footage of a human performance. The central claim is that, once the dynamic geometry is tracked and several material properties are fixed, the remaining physically-based rendering attributes (ambient occlusion and base color) can be solved directly from the input images through accumulated ray-traced visibility rather than through expensive inverse rendering. The pivotal equation is $L_o = \\rho L_D^o + L_S^o$, which separates outgoing radiance into a diffuse part proportional to base color $\\rho$ and a specular part independent of it. This makes the recovered asset usable in existing CG pipelines, supporting real-time deferred shading at 100 FPS in 1080p and offline ray tracing. If the claim holds, it removes the need for controlled light-stage capture or neural relighting networks and gives artists a compact, editable, relightable representation of a moving person.","feed_headline":"Multi-view RGB becomes relightable 4D video","feed_subtitle":"BEAM recovers albedo, roughness, and occlusion so moving humans relight in CG engines at 1080p, 100 FPS.","key_machinery":"The load-bearing object is the simplified Disney BRDF rendering equation, $L_o(\\mathbf{x},\\boldsymbol{\\omega}_o) = \\rho(\\mathbf{x}) L_D^o(\\mathbf{x},\\boldsymbol{\\omega}_o) + L_S^o(\\mathbf{x},\\boldsymbol{\\omega}_o)$, where the diffuse residue $L_D^o$ and specular residue $L_S^o$ are evaluated by Monte Carlo integration over an environment map weighted by the visibility $V_{\\mathrm{env}}(\\mathbf{x},\\boldsymbol{\\omega}_i) = \\prod_i (1 - o_i G_i(\\mathbf{x}_i))$ accumulated along rays through the Gaussian cloud. Because the diffuse term is proportional to base color $\\rho$ and the specular term is independent of it, the same visibility computation yields both ambient occlusion and an algebraic solution for $\\rho$ from observed pixel radiance. The dynamic geometry is supplied by a dual Gaussian representation with joint Gaussians for motion tracking and skin Gaussians for surface detail, rendered through a geometry-aware rasterizer that emits depth and normal maps.","core_discovery":"The paper establishes that a 4D Gaussian sequence augmented with per-splat base color, roughness, and ambient occlusion can be relit under arbitrary environment lighting after a single capture. After tracking joint and skin Gaussians with normal-consistency constraints, BEAM assigns roughness from a generative material model, then computes 2D ambient-occlusion and base-color maps in input views. The key step is a simplification of the rendering equation, $L_o = \\rho L_D^o + L_S^o$, where the diffuse and specular residues are computed by Monte Carlo integration over visibility from a tailored Gaussian ray tracer, allowing the base color to be recovered by subtraction from the measured pixel radiance. These 2D maps are then optimized into the corresponding dense Gaussian attributes, and the view-dependent color attribute is discarded, leaving a compact PBR representation.","pith_inferences":["The paper does not investigate non-human scenes, but the same visibility-first material solve should apply to any predominantly diffuse or dielectric dynamic object captured under known distant lighting, so the method is likely to generalize beyond human performance capture.","Because the rendering equation used here drops interreflection, a direct stress test would be a synthetic subject with known albedo in a close self-contact pose under a bright colored environment source; if the recovered base color shifts toward the source color, that failure mode is confirmed.","The 2D-to-3D baking strategy means achievable albedo and ambient-occlusion detail is capped by the visibility sampling rate and the denoiser, not by the Gaussian representation itself; higher-frequency materials would require more samples per pixel or a learned visibility estimate.","Allowing base color to vary per Gaussian over time means the representation can also encode time-varying appearance changes such as skin tone shifts, an option the paper leaves unexplored."],"forward_implications":["A single multi-view RGB capture of a moving person yields a 4D asset that can be relit under any distant environment map in real time via deferred shading at 100 FPS in 1080p, or offline with ray tracing.","The final Gaussian representation drops the view-dependent SH color attribute, leaving position, orientation, scale, opacity, base color, roughness, and ambient occlusion, which reduces storage and render load in CG engines.","Material decomposition remains stable when the number of input cameras is reduced from 50 to 20 views, based on the paper's synthetic-data ablation.","On synthetic data with known ground truth, BEAM reports higher PSNR, SSIM, and lower LPIPS for ambient occlusion, base color, and relighting than static Gaussian relighting methods and a parametric-body avatar method.","The pipeline supports scene and lighting editing in a CG engine, and the resulting 4D sequences can be deployed to VR headsets for immersive rendering and interaction."],"supporting_citations":[{"why":"Supplies the Gaussian-based performance tracking that gives temporally consistent motion for the 4D sequence.","marker":"[35]"},{"why":"Provides the geometry-aware rasterizer used to render depth and normal maps for the recovered Gaussians.","marker":"[79]"},{"why":"Supplies the Gaussian ray tracer adapted to compute the visibility term used in Eqs. 4 and 8.","marker":"[50]"},{"why":"Provides the material diffusion model used to generate the roughness texture assigned to the Gaussians.","marker":"[80]"},{"why":"Establishes the rendering equation that the paper simplifies into Eq. 5 for material decomposition.","marker":"[39]"},{"why":"Provides the Disney BRDF model whose Lambertian-plus-specular decomposition justifies solving base color from Eq. 5.","marker":"[3]"},{"why":"Provides the Cook-Torrance specular term used in Eq. 7 for the specular residue.","marker":"[10]"},{"why":"Supplies the panorama stitcher used to build the environment map that provides $L_i^{\\mathrm{env}}$ in Eqs. 6-7.","marker":"[52]"},{"why":"Provides the underlying 3D Gaussian representation that the paper extends with physically-based rendering attributes.","marker":"[41]"}],"fun_headline_variants":["Relightable 4D video from multi-view RGB","BEAM: Gaussian PBR video with real-time relighting","RGB in, relightable Gaussian video out","Relightable 4D video via Gaussian splatting"],"cache_read_input_tokens":18560,"weakest_assumption_plain":"The whole material solve assumes that the only illumination is a known, distant environment map and that light bouncing off the person's own body can be ignored, so any significant self-reflection gets misattributed to albedo or occlusion.","fun_headline_variants_meta":{"raw":{"variants":["Relightable 4D video from multi-view RGB","BEAM: Gaussian PBR video with real-time relighting","RGB in, relightable Gaussian video out","Relightable 4D video via Gaussian splatting"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2598,"prompt_tokens":965,"completion_tokens":1633,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":1566}},"tokens_in":581,"tokens_out":1633,"duration_ms":13194,"temperature":1.0,"reasoning_tokens":1566,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T05:40:11.427920+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run BEAM on a synthetic human with a known ground-truth base color and a deliberately strong self-occluding pose (for example, forearms held close to the chest) under an environment map containing a bright colored region; if the recovered base color or ambient occlusion in shadowed areas changes when that bright region is moved or recolored, the distant-light, no-interreflection assumption is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian-based performance tracking that gives temporally consistent motion for the 4D sequence."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the geometry-aware rasterizer used to render depth and normal maps for the recovered Gaussians."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the material diffusion model used to generate the roughness texture assigned to the Gaussians."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the rendering equation that the paper simplifies into Eq. 5 for material decomposition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Cook-Torrance specular term used in Eq. 7 for the specular residue."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the panorama stitcher used to build the environment map that provides $L_i^{\\mathrm{env}}$ in Eqs. 6-7."}],"review_version":1}