{"id":"4b9afcf7-e4a3-443b-a097-3363cb88db31","arxiv_id":"2411.10825","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ARM is a feed-forward model that reconstructs a 3D mesh and PBR texture maps (albedo, roughness, metalness) from sparse-view images, improving texture sharpness and relighting quality over prior single-image-to-3D methods.","lead":"This paper trains a system called ARM that turns one or a few images of an object into a 3D mesh with detailed, relightable surface materials. It works by separating geometry from appearance and painting textures in UV space using diffusion-generated multi-view images plus a semantic material prior.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantitative 'outperforms' claim depends on an unspecified alignment-selection protocol (Supp. Sec. 14) that can inflate small Table 1/2 margins; this is more load-bearing than the atlas-correspondence issue.","rationale":"The headline claim is empirical, so the least secure link is the measurement procedure. Supp. Sec. 14 selects the alignment that maximizes the evaluation score, which is a form of optimizing on the test metric; with unspecified restarts and selection criteria, the small margins in Table 1 cannot be distinguished from alignment overfitting. This is not an integrity accusation; it is a missing-controls problem that makes the quantitative claim unverifiable from the preprint. The reader's chosen weakest assumption, atlas correspondence between offline pre-unwrapped training shapes and inference-time unwrapping, is plausible but weaker: the inputs to InstantAlbedo include positions, normals, and view directions, so a consistent unwrapping code at both stages would likely make the learned UV-space processing transferable. The paper does provide internal support for the UV-space design through the ablations in Table 3 and Fig. 6, and it acknowledges limitations such as upstream multi-view inconsistency and the indicative nature of PBR-map scores (Table 5). The absence of code, checkpoints, error bars, and a public relighting benchmark is exactly why a conditional rather than an accept verdict is appropriate. These concerns do not overturn the reader's verdict, so UNCHANGED is recommended, with the conditions strengthened to include a fixed alignment protocol and released evaluation scripts.","tokens_in":18740,"tokens_out":11287,"duration_ms":122782,"concrete_test":"Recompute Tables 1 and 2 with a fixed, pre-registered alignment: normalize all meshes to [-1,1]^3 and apply one canonical Procrustes/ICP alignment per method (no grid search over rotations/scales, no selection by evaluation score), then recompute F-Score, CD, PSNR, SSIM, and LPIPS on GSO and OmniObject3D, and the relighting metrics on the synthetic PBR set. If the ARM-vs-MeshFormer gaps (notably F-Score 0.968 vs 0.966 and PSNR 21.692 vs 20.500) shrink or reverse, the quantitative outperformance claim is not established. If the original protocol is kept, the authors should report the number of alignment restarts, the specific score used for selection, and per-metric alignment choices.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ARM outperforms existing methods rests on Tables 1 and 2. Supp. Sec. 14 states that each method's predicted mesh is aligned to ground truth by sampling rotations and scales, running ICP, and 'select[ing] the alignment with the highest evaluation score.' The number of restarts, the score used for selection, and whether the selected alignment is shared across all metrics are not specified. The margins in Table 1 are small (GSO F-Score 0.968 vs MeshFormer 0.966; PSNR 21.692 vs 20.500; LPIPS 0.137 vs 0.141), so metric-driven alignment search can plausibly account for the reported gaps, especially if methods differ in how much they overfit the alignment to the ground truth. Table 2 is based on an author-created 100-object relighting set, has only SF3D as a baseline, and reports no error bars or full protocol, so the relightable-appearance superiority claim cannot currently be independently checked. The atlas-correspondence assumption in Sec. 6 is a real secondary risk, but if the same unwrapping code is used offline and at inference it is less likely to break the comparison than an unverifiable evaluation protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ARM, a feed-forward image-to-3D reconstruction system that separates geometry from appearance. GeoRM predicts a density field through an LRM-style triplane synthesizer and extracts a mesh with differentiable marching cubes; GlossyRM predicts per-vertex roughness and metalness; InstantAlbedo back-projects six multi-view images and auxiliary data into an atlas UV texture space and uses a U-Net plus an FFC-Net with global receptive field to predict lighting-baked color and decomposed diffuse albedo, aided by a material-aware image encoder. Training uses rendered Objaverse subsets, including 55K pre-unwrapped GeoRM meshes for InstantAlbedo. Experiments on GSO and OmniObject3D report improved geometry and appearance metrics over LGM, CRM, InstantMesh, SF3D, and MeshFormer, and ablations show benefits from the back-projected measurements, the material prior, and the FFC-Net. A supplementary relighting benchmark compares ARM with SF3D on 100 PBR objects.","tokens_in":19013,"tokens_out":4172,"duration_ms":43398,"significance":"Relightable appearance reconstruction from sparse views is an important and timely problem, and ARM's architecture is a credible engineering contribution: it moves appearance synthesis into UV texture space to circumvent triplane resolution limits, fuses multi-view information with max-pooling, inpaints unseen texture regions with a global-receptive-field network, and addresses material/lighting ambiguity with a learned material prior. The design choices are individually motivated and the ablations in Table 3 and Figure 6 support their contribution. Training on 8 H100 GPUs is also a practical strength. If the evaluation protocol is made fully rigorous, the method would be a solid addition to the feed-forward 3D generation literature, with concrete value for relighting applications.","major_comments":[{"comment":"The alignment protocol is load-bearing for the central 'outperforms' claim and is underspecified. Supp. §14 states that rotations and scales are sampled, refined with ICP, and that the authors 'select the alignment with the highest evaluation score,' but it does not state how many restarts are used, which evaluation score (F-Score, CD, PSNR, LPIPS, or a combination) governs the selection, or whether the same selected alignment is used for all reported metrics. The Table 1 margins over MeshFormer are small (e.g., GSO F-Score 0.968 vs 0.966; PSNR 21.692 vs 20.500; LPIPS 0.137 vs 0.141), so a per-metric or test-set-tuned alignment search can plausibly account for the reported gaps. Please specify the exact number of restarts, the selection criterion, and whether the chosen alignment is fixed across all geometry and appearance metrics; if possible, report results under a single pre-registered, metric-independent alignment.","section":"§7.2 and Supp. §14"},{"comment":"The relightable-appearance superiority claim rests on an author-created benchmark whose protocol is not independently checkable. The 100-object PBR set, the environment maps, the object selection criteria, and the rendering pipeline are not fully specified, only SF3D is used as a baseline, and no error bars or per-object distributions are reported. Moreover, Supp. Table 5 explicitly labels the PBR-map comparison as 'only indicative,' yet the main text uses these results to support strong conclusions about material decomposition. Please release the benchmark or evaluate on an existing public PBR dataset, add at least one additional relightable baseline, and provide variance information or significance tests for the metrics in Tables 1 and 2.","section":"§7.2 and Table 2"},{"comment":"The UV-space approach assumes that the atlas unwrapping used to pre-unwrap the 55K training shapes from GeoRM matches the unwrapping applied to a new GeoRM mesh at inference. The paper does not state whether the same unwrapping implementation, chart-cutting parameters, and texel sampling are used in both places. If the inference-time unwrapping differs, the back-projected image measurements, auxiliary maps, and learned inpainting cues are misaligned with the texel-to-surface correspondence seen in training, and the reported texture-quality gains would not transfer. Please state explicitly that the training and inference unwrappings are identical, or train InstantAlbedo with the same on-the-fly unwrapping used at test time.","section":"§5.2 and §6"}],"minor_comments":[{"comment":"The manuscript contains several typographical errors (e.g., 'Conventionly' in the Introduction, 'illustred' in Section 4, 'latter processing' for 'later processing', and 'reflective properties' for 'reflectance properties'); a copyedit pass would improve readability.","section":"Throughout"},{"comment":"The relationship between the number of input views and the loss views is unclear: Section 6 says 'with 6 used as input' while Section 7.1 sets N = 10 for view selection, and Equations (2)–(4) sum over N views. Please clarify whether N refers to the number of supervision views, input views, or both, and how the six input views are chosen.","section":"§6 and §7.1"},{"comment":"Since the supplementary PBR-map comparison is explicitly described as 'only indicative,' the main text should carry this caveat when discussing material-decomposition quality in Section 7.2 and Figure 5, rather than presenting the comparison as a definitive quantitative result.","section":"Supp. Table 5 and §7.2"},{"comment":"The caption lists 'Density ρ m Roughness & Metalness' as outputs of the same MLP, but the pipeline assigns density to GeoRM and roughness/metalness to GlossyRM; the caption should separate the two models to avoid confusion.","section":"Fig. 2 caption"}],"recommendation":"major_revision","confidential_remarks":"This is a solid systems paper whose central claim is empirical rather than theoretical. My recommendation is driven by the need to pin down the evaluation protocol—especially the alignment-selection procedure and the author-created relighting benchmark—rather than by doubts about the architecture itself. If the authors provide the exact alignment protocol, release or fully specify the relighting benchmark, add a second baseline, and clarify the training/inference unwrapping consistency, I would support acceptance. The atlas-correspondence issue is secondary but should be answered explicitly because it is a core assumption of the UV-space design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this paper has a real new building block. The InstantAlbedo stage back-projects multi-view images into UV texture space and processes them with a global-receptive-field FFC-Net, plus a DINO-based material prior. That specific combination is not in the cited prior work, and it shows empirically: on GSO the appearance numbers beat MeshFormer by a solid margin (21.7 vs 20.5 PSNR), while the geometry gains are tiny (0.968 vs 0.966 F-Score). The decoupling of geometry and appearance is well-motivated, the ablations confirm that the back-projection, material prior, and FFC-Net each contribute, and the training cost is reasonable (8 H100s, 5 days). This is a competent feed-forward systems paper, not a breakthrough, but a real step forward.\n\nSoft spots are in the evaluation, not the method. The stress-test note is right: the alignment protocol in Supp. Sec. 14 is the load-bearing issue. Saying the alignment with the 'highest evaluation score' is selected, without specifying the number of restarts, the metric used for selection, or whether the same alignment is used across metrics, leaves the small Table 1 margins open to question. I'd want that fully specified before trusting the 'outperforms' claim. The atlas-correspondence issue (55K pre-unwrapped training shapes vs inference-time unwrapping) is a genuine secondary risk, but if the same unwrapping code is used both places it likely holds. Second, there are no error bars or significance tests, so some of the small LPIPS differences could be noise. Third, the relighting benchmark is author-created, has only SF3D as a baseline, and the supp. properly calls the PBR map scores 'indicative' because decomposition is ambiguous. That honesty is appreciated, but it keeps the relightable-appearance superiority provisional. Fourth, the 'first to perform material decomposition entirely in UV space' is a stretch given SF3D already does UV unwrapping with material prediction; the real novelty is the back-projection plus FFC-Net plus material prior combination.\n\nWho's this for: anyone building feed-forward image-to-3D pipelines or working on single-image PBR material estimation. It deserves a serious referee; the architecture is new enough, the gains are consistent, and the weaknesses are fixable with a full evaluation protocol and a released checkpoint. I'd cite it if I were in the space.","headline":"A competent but provisional feed-forward systems paper: the UV-space InstantAlbedo stage is the real new idea, and the evaluation protocol needs tightening before the 'outperforms' claim can be fully trusted.","tokens_in":19563,"tokens_out":4397,"would_cite":true,"duration_ms":39185,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By moving appearance reconstruction into UV texture space, ARM produces sharper, relightable 3D assets from sparse-view images than previous feed-forward methods.","keywords":["image-to-3D reconstruction","relightable appearance","UV texture space","Physically-Based Rendering maps","material decomposition","sparse-view reconstruction","large reconstruction model","material prior"],"falsifier":"One decisive test is to take a GeoRM mesh, unwrap it with two different atlas parameterizations, and run InstantAlbedo on both; if the predicted albedo and relit renders differ substantially in sharpness or show misaligned seams, the central assumption of transferable texel correspondence fails. A sharper test: at inference, replace the unwrapping code used to build the training atlas with an independent UV unwrapper, and check whether PSNR and LPIPS on relit renders degrade on GSO.","tokens_in":18523,"feed_emoji":"🎨","tokens_out":7555,"duration_ms":67367,"temperature":0.7,"pith_summary":"ARM is a feed-forward system that turns sparse-view images of an object into a ready-to-use 3D mesh with physically based appearance. Its central claim is that geometry and appearance should be handled by separate networks, and that appearance should be reconstructed directly in UV texture space rather than decoded from triplanes at fixed resolution. On that basis it back-projects the input views, masks, positions, view directions, and surface normals onto the texture atlas, fuses them with a global-receptive-field inpainting network, and adds a material-aware image prior to separate illumination from albedo. The paper reports that this yields sharper textures, spatially varying roughness and metalness, and better relighting than prior feed-forward models such as InstantMesh, MeshFormer, and SF3D, trained on 8 H100 GPUs in about five days. A sympathetic reader would take away that texture resolution and material disentanglement, not geometry, are the remaining bottleneck for practical single-image 3D generation.","feed_headline":"Moving texture work to UV space sharpens relightable 3D from one image","feed_subtitle":"Back-projecting six views onto a UV atlas plus a material prior yields sharp, relightable PBR assets from one image.","key_machinery":"The load-bearing mechanism is the UV texture-space appearance stage, which ARM calls InstantAlbedo. Instead of querying a triplane with an MLP, ARM first unwraps the predicted mesh into atlas charts, then explicitly back-projects each input view and its auxiliary data (mask, position, texture coordinate, view direction, normal, and material-encoded features) onto the texture map. Six per-view U-Net feature maps are fused by max-pooling, and an FFC-Net with a global receptive field inpaints unseen texels and refines the result; a material prior from a material-aware image encoder is injected through the back-projection so the network can tell material boundaries apart from lighting gradients. This machinery is what lets ARM produce spatially varying albedo, roughness, and metalness at texture resolution without scaling up the triplane transformer.","core_discovery":"On the paper's own terms, the discovery is that the blurriness and baked-in lighting that plague feed-forward image-to-3D models come from decoding appearance from triplanes, and that both problems disappear when appearance is processed in UV texture space. ARM's GeoRM predicts a density field and extracts a mesh with differentiable marching cubes; the mesh is unwrapped into atlas charts, and InstantAlbedo back-projects the six input views together with masks, positions, texture coordinates, view directions, and normals onto those charts. A per-view U-Net extracts features, max-pooling fuses the views, and an FFC-Net with a global receptive field inpaints unseen regions, producing both a lighting-baked color map and a decomposed diffuse albedo map. Roughness and metalness are predicted separately per vertex by GlossyRM, and a material-aware image encoder (a DINO ViT initialized on semantic material maps and fine-tuned jointly) is back-projected as a material prior so that strong lighting does not bake into the albedo. The authors report that this design outperforms LGM, CRM, InstantMesh, MeshFormer, and SF3D on GSO and OmniObject3D for both geometry and appearance, and beats SF3D on relit renders under novel lighting.","pith_inferences":["Pith inference: the UV-space stage is modular: any mesh with a consistent unwrapping could receive InstantAlbedo's texture refinement, so ARM's appearance gains may transfer to other geometry generators or to user-provided meshes.","Pith inference: the material prior's separation of albedo and lighting suggests a testable extension to unseen material categories or to in-the-wild photos with unknown lighting, where the prior would need to generalize beyond Objaverse's material distribution.","Pith inference: because the training atlas and the inference mesh are unwrapped by different code paths, the method's robustness to mesh perturbation and re-meshing is a risk worth measuring directly; a geometry generator that produces a slightly different triangulation could break texel correspondence.","Pith inference: pairing this texture-space pipeline with a video-based multi-view generator instead of Zero123++ could reduce the cross-view inconsistencies the paper names as its main limitation, since back-projection would then average over more consistent measurements."],"forward_implications":["Single-image 3D generation can produce assets with true PBR maps, not just vertex colors, so downstream engines can relight them under arbitrary environments.","Texture sharpness no longer scales with triplane resolution, because appearance is resolved on the UV atlas where measurements are directly visible.","A learned material prior can separate illumination from albedo even with only six views, reducing the baked-in-lighting artifacts that inverse-rendering losses leave behind.","Geometry and appearance can be trained separately, which keeps the memory footprint low enough that triplane resolution can be raised to 256 and the whole system trained on 8 GPUs.","The reported gains on GSO, OmniObject3D, and the relighting dataset imply that the UV-space pipeline transfers across object categories outside the Objaverse training subset."],"supporting_citations":[{"why":"Supplies the LRM triplane backbone that GeoRM and GlossyRM are built on.","marker":"[21]"},{"why":"Makes iso-surface extraction differentiable so GeoRM can train with geometry-only losses.","marker":"[76]"},{"why":"Motivates the max-pooling fusion of per-view UV feature maps into one texture.","marker":"[50]"},{"why":"Provides the Fourier convolution blocks with a global receptive field used to inpaint unseen texels.","marker":"[64]"},{"why":"Gives the base ViT architecture for the material-aware image encoder.","marker":"[6]"},{"why":"Supplies the multi-scale cascade design and the semantic material weights that initialize the encoder.","marker":"[58]"},{"why":"Generates the six consistent multi-view images from a single input used by ARM and its baselines.","marker":"[59]"},{"why":"Is the two-stage LRM training baseline whose strategy GeoRM follows and a comparison method.","marker":"[81]"},{"why":"Is the main PBR baseline for UV unwrapping, material prediction, and relit rendering comparisons.","marker":"[4]"},{"why":"Provides the Objaverse 150K subset that trains GeoRM, GlossyRM, and InstantAlbedo.","marker":"[12]"}],"fun_headline_variants":["UV-space appearance beats triplane decoding for relightable 3D","Back-projecting sparse views onto UV atlas makes relightable 3D","Material prior and UV-space appearance sharpen relightable 3D","UV-space appearance, not triplanes, for relightable 3D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the flattening of a newly predicted mesh onto a 2D texture atlas at inference matches the flattening that was used to prepare the pre-unwrapped training shapes; if those two unwrappings disagree, the image pixels and material features back-projected onto the texture land in the wrong spots, and the sharpness gains would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["UV-space appearance beats triplane decoding for relightable 3D","Back-projecting sparse views onto UV atlas makes relightable 3D","Material prior and UV-space appearance sharpen relightable 3D","UV-space appearance, not triplanes, for relightable 3D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001065,"raw_usage":{"total_tokens":4465,"prompt_tokens":944,"completion_tokens":3521,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":3441}},"tokens_in":560,"tokens_out":3521,"duration_ms":48689,"temperature":1.0,"reasoning_tokens":3441,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:15:34.629643+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One decisive test is to take a GeoRM mesh, unwrap it with two different atlas parameterizations, and run InstantAlbedo on both; if the predicted albedo and relit renders differ substantially in sharpness or show misaligned seams, the central assumption of transferable texel correspondence fails. A sharper test: at inference, replace the unwrapping code used to build the training atlas with an independent UV unwrapper, and check whether PSNR and LPIPS on relit renders degrade on GSO.","supporting_citations":[{"cited_title":"Materialistic: Se- lecting similar materials in images","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-scale cascade design and the semantic material weights that initialize the encoder."}],"review_version":1}