{"id":"29d4c1df-d02a-47ba-bf29-ce948aade3ba","arxiv_id":"2411.15732","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"DynamicAvatars reconstructs dynamic 3D head avatars from video and enables prompt-based editing via dual Gaussian tracking, semantic masks, and LLM-guided diffusion editing.","lead":"This paper describes a method for building moving 3D head avatars from video and then editing them with text prompts, such as changing a person's expression or adding an accessory. The approach combines Gaussian splat tracking with large language model guidance, aiming to improve editing accuracy and stability.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central dynamic-editing claim rests on an underspecified mask-warping mapping net and an undefined objective for lifting 2D edits back into 3D Gaussians; without OOD validation or a reproducible loss, the 67.5% success rate is unsupported.","rationale":"The paper promises photorealistic dynamic avatars with precise text-based editing. I read the method sections and experiments in good faith. Reconstruction (Table 1) may be plausible because it explicitly builds on GaussianAvatars, but the numbers are single-run and protocol details are absent. The bigger weakness is the editing claim, which requires two unstated mechanisms: (1) propagating a 2D edit mask correctly across arbitrary time/pose, and (2) converting per-image diffusion edits into coherent 3D Gaussian updates. The mapping net in §4.2 is the only mechanism for (1), and it is trained and tested on the same NeRSemble distribution; no generalization evidence is given. Mechanism (2) is not described at all: Eq. (17) invokes L_rgb but there is no ground truth for edited views, and the text never states whether the method uses score-distillation, per-image reconstruction against DALL-E outputs, or something else. This is not a stylistic disagreement; it is a gap that makes the headline result untestable. A single OOD mask-IoU experiment would settle whether the mask-propagation concern is real, and writing out the actual update rule would settle whether the editing pipeline exists as specified. The reader's REJECT verdict is appropriate; I would keep it.","tokens_in":9935,"tokens_out":4308,"duration_ms":38456,"concrete_test":"On a held-out NeRSemble sequence with expressions/poses not used in training, compute IoU between the mapping net's predicted masks M^t_p and the first-stage FCI masks for the same frames, across several times t and camera poses p. If mean IoU falls below ~0.8, the mask-propagation assumption fails, and the editing pipeline has no reliable way to know which splats to edit, undermining the temporal-consistency claim. As a secondary check, require the authors to write out the gradient step used to update Gaussian parameters from DALL-E edited images; if no such step is specified, Table 2 cannot be reproduced.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing pillar is the editing pipeline, not the reconstruction numbers alone. In §4.2 the method locates splats to edit via a 'mapping net' that warps a baseline mask M^{t0}_{p0} to M^t_p; the net is trained only on masks from the same NeRSemble training set, and the paper gives no architecture, loss, or validation of the warped masks. The text itself concedes (§4.2) that a labeled splat at t0 may not contribute to the same semantic region at t1; if the warp is inaccurate, edits will bleed or flicker across poses and expressions. The second editing stage (§4.3) is even less specified: GPT-4 rewrites prompts, DALL-E (per Fig. 4 caption) generates images, and 'latent manipulation' finalizes them, but no equation explains how the 2D edited images are lifted back to the 3D Gaussian model. Equation (17) defines L_edit with an L_rgb term, but no ground-truth edited image exists; the actual objective for updating splats from edited images is absent. Table 2's success metric is named but not defined, so the 67.5% number is unverifiable. Without these details, the central claim of precise dynamic editing cannot be tested.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DynamicAvatars, a two-stage method for reconstructing dynamic 3D head avatars from multi-view video and for performing text-based edits on those avatars. The first stage binds Gaussian splats to a FLAME mesh with a ``dual-tracking'' mechanism and semantic masks; the second stage uses an LLM-based prompt preprocessor, a mask-warping mapping network, and a GAN-based discriminator to propagate edits across time and pose. Experiments on NeRSemble report improved reconstruction (PSNR 30.1 vs. 29.3 for GaussianAvatars, SSIM 0.907 vs. 0.865) and substantially higher edit-success rates (67.5% vs. 36.4% for GenN2N). The central claim is that the method enables precise dynamic editing of facial avatars, but the paper provides only high-level descriptions of the editing mechanism and does not define the evaluation protocol for edit quality.","tokens_in":7,"tokens_out":4496,"duration_ms":104079,"significance":"If fully specified and validated, the proposed dynamic editing pipeline would be a useful contribution to the growing line of work on editable 3D Gaussian avatars. The paper identifies a real problem: static semantic masks fail in dynamic scenes where Gaussians move relative to the mesh, and existing prompt-based editing methods struggle with fine-grained spatial instructions. The reported reconstruction improvements are plausible and consistent with the qualitative figures. The paper also credits the use of an LLM for prompt restructuring, which is a reasonable design choice. However, the manuscript currently lacks the implementation details needed to assess the central editing claim: the mapping network is not described, the objective for lifting 2D edits back to 3D is missing, and the headline edit-success metric is undefined. The paper therefore has the shape of a system demonstration rather than a complete, reproducible method.","major_comments":[{"comment":"The dynamic mask propagation step is the backbone of the editing pipeline, but the mapping network is not specified. The text states ``We train this module by utilizing the mask of the training dataset at different time and poses generated at the first stage'' without giving the architecture, loss function, or validation procedure. There is also no quantitative evaluation of the predicted masks (e.g., mask IoU on held-out timesteps or camera poses). Because the paper itself concedes that ``a labeled splat at time t0 may not contribute to the color of the corresponding semantic region at time t1,'' the correctness of the warp is load-bearing for the whole editing claim. Without these details, the claim of temporally consistent editing cannot be verified.","section":"§4.2, Fig. 5"},{"comment":"The second editing stage is described only as ``latent manipulation,'' but no equation or algorithm explains how the edited 2D images update the 3D Gaussian model. Eq. (17) defines L_edit with L_rgb, L_gs, and L_G, yet there is no ground-truth edited image I_edited; the text does not specify which rendered frames are compared with which edited images, how per-frame edits are fused, or how gradients flow to the Gaussian splats. This omission is load-bearing because the paper's central contribution is ``precise editing'' of dynamic avatars. The authors should provide the exact optimization objective and the backpropagation path from edited images to splat parameters.","section":"§4.3, Fig. 6, Eq. (17)"},{"comment":"The edit-quality numbers are uninterpretable as reported. The metric ``Average Spacial Attribute'' is not defined, and no protocol is given: number of subjects, number and content of prompts, number of raters, definition of ``success,'' or confidence intervals. The headline claim of 67.5% versus 36.4% therefore has no documented basis. Please specify the exact metric, the evaluation setup, and provide per-prompt or per-subject breakdowns so the result can be reproduced and compared with baselines.","section":"§5.3, Table 2"},{"comment":"The ``specialized GAN algorithm'' is presented only as two generic adversarial losses. There is no discriminator architecture, no description of how D and G are trained alternately, no loss weights or training schedule, and no discussion of how the GAN interacts with the mapping network and the diffusion-based editing process. Since the paper attributes its style-editing quality and temporal consistency to this component, the reported improvements cannot be reproduced or assessed without these details.","section":"§4.4, Eqs. (14)–(15)"}],"minor_comments":[{"comment":"The terms ``SLM'' and ``Instruct-Pix2Pix metric'' are used without definition; please explain what SLM stands for and how the text-image direction similarity is computed.","section":"§5.1"},{"comment":"The tracking loss uses L_perceptron(l, bl), but the symbols l and bl are not defined before the equation.","section":"§4.4, Eq. (12)"},{"comment":"The text says the λ values ``have been tested to be effective during ablation study in our work,'' but the paper contains no ablation study; either add the ablation or remove the claim.","section":"§4.4, Eq. (11) context"},{"comment":"There are typos that should be corrected: ``Spacial'' should be ``Spatial,'' and ``ultilize'' in the Figure 3 caption should be ``utilize.'","section":"Table 2 and Figure 3 caption"},{"comment":"The subcaptions in Figure 7 are labeled (a)–(e) but the main text references Figure 8 and Figure 9 in a confusing order; please align the caption labels with the in-text references.","section":"Figure 7 captions"},{"comment":"Section 3.2 is a generic review of prompt-based diffusion editing and is not connected to the actual implementation of the method; either remove it or explicitly state how the described components are instantiated in DynamicAvatars.","section":"Section 3.2"},{"comment":"The weights λ1, λ2, λ3 in L_edit are not reported; please provide the values used in the experiments.","section":"Eq. (17)"},{"comment":"The reference list contains minor formatting issues, such as ``[42]'' being cited as ``[42]'' while some entries lack venue information; please standardize the bibliography.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like an early technical report rather than a complete paper: the central components of the editing stage are described only in figure captions and high-level text. The reconstruction results are positive, but the editing pipeline is not sufficiently specified to evaluate the main claim. I recommend major revision with the expectation that the authors provide full implementation details for the mapping network, the 3D lifting objective, and the edit-quality evaluation protocol. If those details cannot be provided, the paper's central claim would be unsupported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the reconstruction part is a modest, plausible extension of GaussianAvatars with semantic masks; the editing part, which is the paper's main sell, is underspecified to the point that the headline numbers cannot be checked.\n\nWhat's genuinely useful: the problem is real. Static 2D/3D masks break when a head moves or changes expression, and the proposed fix—learn a mapping net that warps a baseline mask to other timestamps and viewpoints, then track which splats contribute to the masked region—is a sensible direction. The authors also correctly note that FLAME doesn't cover accessories, so decoupling splats from the mesh for editing is a reasonable design choice. The related work citations are mostly appropriate.\n\nThe soft spots are serious. Section 4.2 says the mapping net takes t, pose, and the baseline mask and outputs the target mask, but gives no architecture, no loss, no training details, and no validation of the warped masks. The paper itself concedes that a splat labeled at t0 may not contribute to the same semantic region at t1; if the warp is wrong, edits will bleed or flicker. More importantly, Section 4.3 has no equation describing how the 2D edited images are lifted back to update the 3D Gaussian splats. Equation (17) defines L_edit with L_rgb, but there is no ground-truth edited image, so it's unclear what the supervision is. The 'success' metric in Table 2 is never defined. These aren't minor omissions—they are the load-bearing parts of the editing claim.\n\nI also note the reconstruction gain (PSNR 30.1 vs 29.3) is small and reported without error bars or significance; it could easily be noise. The editing gain (67.5% vs 36.4%) looks large but is unverifiable given the undefined metric.\n\nNet: this is a serious attempt at a real problem, and the authors engage with relevant prior work. But as a manuscript it's not ready for peer review. My recommendation is to desk reject and invite a resubmission after the method is fully specified—architecture, losses, training procedure, and a precise evaluation protocol. No code or data is included, so none of the numbers can be independently checked.","headline":"A plausible reconstruction system whose editing claims rest on an unspecified mapping net and an undefined 3D uplift objective; needs major method details before it can be evaluated.","tokens_in":10742,"tokens_out":2891,"would_cite":false,"duration_ms":25423,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DynamicAvatars reconstructs moving 3D head avatars from video and edits them with text prompts.","keywords":["dynamic head avatars","3D Gaussian splatting","text-guided editing","diffusion models","LLM-guided prompts","GAN supervision","FLAME mesh","novel view synthesis"],"falsifier":"Hold out one expression sequence or camera view from the NeRSemble training set, edit a spatially localized feature such as an earring using a prompt that names its position, and render the edited avatar across that held-out view; if the earring drifts, flickers, or disappears as the head moves, the claimed dynamic editing consistency is refuted.","tokens_in":9756,"feed_emoji":"🎭","tokens_out":8754,"duration_ms":74205,"temperature":0.7,"pith_summary":"DynamicAvatars is a two-stage system that turns multi-view video of a person's head into a moving 3D avatar, then lets the user edit that avatar with natural-language prompts. The central claim is that combining two tracking signals—Gaussian splats (small oriented 3D blobs rendered to images) anchored to the FLAME face mesh, and Gaussian splats labeled by facial semantic region—lets the same model reconstruct sharp faces and keep edits attached to the right area as the head moves. Editing is stabilized by an LLM that rewrites user prompts into structured guidance and a GAN-based discriminator that enforces consistent color across frames. On the NeRSemble benchmark the paper reports reconstruction quality of PSNR 30.1 versus 29.3 for GaussianAvatars and a prompt-edit success rate of 67.5% versus 36.4% for GenN2N. The intended payoff is controllable avatars for film, games, and virtual reality in which expression changes, style transfers, and accessory additions stay realistic over time.","feed_headline":"Text-based edits on 3D head avatars hit 67.5% success","feed_subtitle":"Pipeline also reconstructs faces at higher fidelity than prior baselines, PSNR 30.1 vs. 29.3.","key_machinery":"The load-bearing machinery is dual tracking over semantic-labeled 3D Gaussian splats—small oriented 3D blobs rendered into images—bound to the FLAME face mesh. In the modeling stage, each splat is first bound to a FLAME mesh triangle to preserve face structure, then tagged with a semantic label from a facial component identifier, so the same splats can be followed through time and across camera poses. A mapping network carries an editing mask from a baseline view to every target view by taking the target timestep and camera pose as inputs and applying bilinear interpolation on the time-pose plane, identifying which splats contribute to the edited region throughout the sequence. The editing stage then chains an LLM-based prompt preprocessing module (GPT-4) that turns a user prompt into structured instructions, a diffusion editor (DALL-E) that generates the edited images, and a conditional GAN discriminator that supervises color consistency. This chain is what lets edits apply to the whole dynamic avatar rather than to one frame.","core_discovery":"The paper claims that dynamic facial avatar reconstruction and precise editing should be built on one shared representation, not as separate steps. That representation is a set of 3D Gaussian splats bound to the FLAME mesh for structural rigidity, with each splat additionally labeled by a facial component identifier so that regions outside FLAME (hair, rings, hats) can also be tracked and edited consistently. During editing, a mapping network propagates a requested edit mask across time steps and camera poses, an LLM restructures the user's prompt into detailed instructions, a diffusion model renders the edited frames, and a discriminator removes color and texture artifacts around sensitive regions such as eyes and teeth. The authors state that this combination removes facial distortions and inaccurate head movements that hamper prior NeRF- and 3DGS-based avatars, and they support it with comparisons to GaussianAvatars, HeadStudio, PointAvatar, GaussianEditor, and GenN2N.","pith_inferences":["Beyond the paper, the mapping network's generalization is the part to test first: if it transfers to unseen identities, expressions, or camera poses, the same pipeline could likely run on sparser or monocular capture rather than dense multi-view rigs.","Beyond the paper, because the editing backbone uses off-the-shelf GPT-4 and DALL-E, edit fidelity should improve automatically as those base models improve, without changes to the avatar representation.","Beyond the paper, the reported edit metrics measure instruction-following, not frame-to-frame temporal stability; a direct flicker or temporal-consistency metric on long sequences would be a natural next evaluation.","Beyond the paper, the densification and splitting schedule is fixed and the authors note it is not tuned; an adaptive module that densifies where the semantic mask changes could further improve fine accessory edits."],"forward_implications":["Because reconstruction and editing share the same Gaussian splat representation, an edit made at any frame is defined over the whole moving sequence rather than a single image.","The semantic labeling of splats means accessories such as earrings, rings, and hats can be added or modified without breaking the FLAME-tracked face structure.","LLM prompt preprocessing and the GAN discriminator should allow prompts that specify relative positions and fine style details, where prior prompt-only editors fail.","On the reported benchmarks, the same pipeline improves both reconstruction fidelity and edit success, so dynamic editing no longer requires a separate edited-avatar retraining step per prompt."],"supporting_citations":[{"why":"Provides the 3D Gaussian Splatting primitive the avatar is built from and the rendering formulation used in training.","marker":"[17]"},{"why":"Defines the FLAME mesh whose triangles anchor the Gaussian splats in the dual-tracking scheme.","marker":"[18]"},{"why":"Is the main reconstruction baseline and the source of the mesh-Gaussian binding approach that dual tracking extends.","marker":"[31]"},{"why":"InstructPix2Pix defines the prompt-based editing task and the instruction-success metric used to evaluate edits.","marker":"[3]"},{"why":"GaussianEditor is the semantic-mask editing baseline whose static masks fail in dynamic scenes, motivating the mapping network.","marker":"[7]"},{"why":"GenN2N is the GAN-based NeRF editing baseline that the method compares against for edit quality.","marker":"[22]"},{"why":"SLD is the LLM-controlled diffusion baseline whose prompt-refinement idea the preprocessing module builds on.","marker":"[36]"}],"fun_headline_variants":["One 3D avatar model for both reconstruction and precise editing","DynamicAvatars: diffusion-driven head avatars with LLM-guided edits","Gaussian splats bound to FLAME enable editable dynamic head avatars","LLM plus diffusion gives precise edits on 3D head avatars","Reconstruct and edit dynamic facial avatars in a single pipeline"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the mapping network, trained on masks generated from the same NeRSemble training data, will correctly warp an editing mask to unseen time steps and camera poses; if that warping fails, the edited region will flicker, drift, or corrupt the avatar.","fun_headline_variants_meta":{"raw":{"variants":["One 3D avatar model for both reconstruction and precise editing","DynamicAvatars: diffusion-driven head avatars with LLM-guided edits","Gaussian splats bound to FLAME enable editable dynamic head avatars","LLM plus diffusion gives precise edits on 3D head avatars","Reconstruct and edit dynamic facial avatars in a single pipeline"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000726,"raw_usage":{"total_tokens":3238,"prompt_tokens":917,"completion_tokens":2321,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":2229}},"tokens_in":533,"tokens_out":2321,"duration_ms":14605,"temperature":1.0,"reasoning_tokens":2229,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:57:17.710483+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out one expression sequence or camera view from the NeRSemble training set, edit a spatially localized feature such as an earring using a prompt that names its position, and render the edited avatar across that held-out view; if the earring drifts, flickers, or disappears as the head moves, the claimed dynamic editing consistency is refuted.","supporting_citations":[{"cited_title":"Gaussianeditor: Swift and controllable 3d editing with gaussian splatting, 2023","cited_arxiv_id":null,"evidence_quote":"GaussianEditor is the semantic-mask editing baseline whose static masks fail in dynamic scenes, motivating the mapping network."},{"cited_title":"Genn2n: Generative nerf2nerf translation","cited_arxiv_id":null,"evidence_quote":"GenN2N is the GAN-based NeRF editing baseline that the method compares against for edit quality."}],"review_version":1}