{"id":"fc172c0b-ccd3-40db-9172-28bf339d89b3","arxiv_id":"2412.20148","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DEGSTalk combines per-embedding Gaussian deformations with a hair-preserving fusion step to synthesize talking faces that beat prior baselines on PSNR, LPIPS, LMD, and SSIM.","lead":"This paper presents a new 3D Gaussian-based system that animates a person's face from audio while keeping long hair looking natural. It reports better image quality and lip motion than several existing talking-face methods on six test videos.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (7) is not a valid alpha-compositing rule: the hair term is added unweighted and the mouth term is gated by (1-O_face), so the paper's defining hair-preserving mechanism is mis-specified as written.","rationale":"The reader's concern about baseline protocol is legitimate: Table I does not state whether AD-NeRF, ER-NeRF, SyncTalk, and TalkingGaussian were re-run under DEGSTalk's exact dataset split, preprocessing, pose, and metric protocol, and the reported margins are large. I do not dispute that concern. However, I find a more load-bearing issue internal to the method: the paper's headline contribution, hair-preserving rendering, is defined by Eq. (7), which as written is not a valid compositing operation. The hair term is added without opacity weighting, and the mouth term is gated by (1-O_face), which is the opposite of a mouth-over-face composite. Because the ablation study attributes a 3.4 dB PSNR improvement to the H component, the central empirical result cannot be interpreted until Eq. (7) is corrected or the released code is shown to implement a different, valid fusion rule. This is a correctness risk that the baseline-protocol concern does not capture. I therefore keep the reader's CONDITIONAL verdict: the paper is salvageable if the authors clarify Eq. (7), release code, and verify that the corrected composition reproduces the reported numbers, but the manuscript as written does not yet support the central claim.","tokens_in":7963,"tokens_out":9954,"duration_ms":105990,"concrete_test":"Run the released DEGSTalk code on one subject (e.g., 'May') and instrument the renderer to save C_hair, C_face, O_face, C_mouth, and the final image. Then: (1) check numerically whether Eq. (7) holds and whether more than 0.1% of pixels exceed the valid [0,1] range; (2) replace Eq. (7) by the standard alpha-over rule C = C_face*O_face + C_mouth*O_mouth*(1-O_face) + C_hair*(1-O_face), re-run Table II, and compare. If the implemented fusion differs from Eq. (7), or if the corrected rule changes the H-ablation margin substantially, the paper must be revised with a validated composition equation before its central claim can be accepted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of improved long-hair synthesis is carried by the Dynamic Hair-Preserving Portrait Rendering in Sec. II-C, but its defining equation (7) is not a coherent composition rule. As printed, C = C_hair + C_face*O_face + C_mouth*(1-O_face), where C_hair is the hair-region color (the 'Chair' term in the text). Two concrete problems arise. First, C_hair is added without any opacity modulation or multiplication by (1-O_face): in pixels where the face is opaque (O_face=1), the hair radiance is still added on top of the face color, double-counting energy and potentially producing out-of-range RGB values. Second, the mouth term is weighted by (1-O_face), so the mouth contribution is largest where the face is transparent, i.e., outside the face region, rather than inside the mouth. C_hair is never defined and no hair opacity term appears anywhere. The large H-versus-w/o-H gap in Table II (PSNR 33.89 vs 37.30, SSIM 0.9354 vs 0.9664) and the qualitative 'hair-preserving' claims are attributed to exactly this stage, so the central mechanism is formally underspecified. If Eq. (7) is a typo, the corrected rule must be stated; if it is literal, the rendered images are not standard alpha composites and the claimed advantage is not assessable from the text. This concern is independent of the baseline-protocol issue: even with perfectly matched baselines, Eq. (7) must be a valid image-formation model for the head-to-head numbers to support the paper's conclusion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DEGSTalk, a 3D Gaussian Splatting (3DGS) based method for audio-driven talking face synthesis with emphasis on long-hair preservation. The method introduces per-Gaussian learnable embeddings z used by a deformation network, and a Dynamic Hair-Preserving Portrait Rendering stage that fuses separately rendered face, mouth, and hair regions. Experiments on six talking-head videos report consistent improvements over AD-NeRF, ER-NeRF, SyncTalk, and TalkingGaussian on PSNR, LPIPS, LMD, and SSIM, with inference at 114 FPS.","tokens_in":8334,"tokens_out":3255,"duration_ms":32634,"significance":"If the technical details are validated, the paper would make a useful contribution to 3DGS-based talking face synthesis, especially for the under-addressed case of long hair. The paper explicitly provides the loss formulations, training schedule, and an ablation study, and it promises public code. The main strengths are the novelty of per-Gaussian embeddings in this setting and the attempt to preserve hair through a dedicated rendering composition. However, the central rendering equation is not a coherent image-formation model as written, and the quantitative comparison lacks the protocol details needed to verify the claimed gains. The contribution is promising but the current evidence is insufficient for acceptance.","major_comments":[{"comment":"The defining composition rule of the Dynamic Hair-Preserving Portrait Rendering is not a valid alpha-compositing formula as printed. In C = Chair + Cface × Oface + Cmouth × (1 − Oface), the hair color Chair is added without any opacity modulation or multiplication by (1 − Oface), so in pixels where the face is opaque the hair radiance is still added on top of the face color, which double-counts energy and can produce out-of-range RGB values. The mouth term is weighted by (1 − Oface), so it is strongest where the face is transparent, i.e., outside the face region, rather than in the mouth area. Moreover, Chair is never defined and no hair opacity term appears anywhere in the paper. Since the large H-versus-w/o-H gaps in Table II (PSNR 33.89 vs. 37.30, SSIM 0.9354 vs. 0.9664) and the qualitative hair-preservation claims are attributed to exactly this stage, Eq. (7) must be corrected and fully specified; otherwise the central mechanism is not assessable from the text.","section":"II-C, Eq. (7)"},{"comment":"The headline comparison in Table I is not verifiable as reported. The paper does not state whether AD-NeRF, ER-NeRF, SyncTalk, and TalkingGaussian were re-run by the authors under the same dataset, cropping, pose estimation, and metric protocol, or whether the numbers were taken from their original papers. Different test splits, preprocessing, or pose alignment can easily change the reported metric values by more than the margins shown here (for example, PSNR 33.73 vs. 35.66 and SSIM 0.9361 vs. 0.9678 for SyncTalk vs. the proposed method). The paper also reports no error bars or significance tests, so the consistent-looking gains could be within run-to-run variability. The authors should either state that all baselines were evaluated under identical conditions and provide the experimental setup, or re-run the baselines and report variances.","section":"III-B / Table I"},{"comment":"The ablation study contains internal inconsistencies that weaken its support for the component claims. The row 'w/o F' reports LPIPS of 0.150, which is an order of magnitude worse than the full model's 0.0140 and also much worse than every other ablation row; this looks like a typo, and without correction it undermines the ablation analysis. In addition, 'w/o E' achieves SSIM 0.9712 and 'w/o F' achieves SSIM 0.9692, both above the full model's 0.9664, yet the paper states 'Our method achieves strong performance across multiple evaluation metrics' without discussing these non-monotonic results. The paper should correct the table and either show that the full model is consistently best or temper the claim accordingly.","section":"III-D / Table II"}],"minor_comments":[{"comment":"The phrase 'head reconstruction setting' appears in the metric section and in Table I, but the setting itself is never explicitly defined; please clarify what it means and whether the same setting applies to all baselines.","section":"III-A"},{"comment":"The last row of Table II is labeled 'w E, F, R, H', but the standard notation used elsewhere in the table is 'w/o'; this should be corrected to 'w/ E, F, R, H' or a similar unambiguous label.","section":"III-D / Table II"},{"comment":"The sentence 'Initially, the face and mouth regions undergo dilate operations during training to better capture their features, while also preserving the hair region Chair' is confusing: 'Chair' appears to be a term rather than a region name, and the connection between dilation and the preservation of Chair is not explained.","section":"II-C"},{"comment":"The conclusion mentions 'Some noisy primitives may still occur' as future work, but this limitation is not introduced or discussed in the body; a short limitation paragraph would help contextualize the method's failure modes.","section":"IV"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and relevant problem for 3DGS-based talking face synthesis. The per-Gaussian embedding idea is a reasonable extension of deformable Gaussian fields, and the hair-preserving goal is well motivated. However, the central rendering equation is formally opaque and the baseline comparison is not yet rigorous enough to support the claimed state-of-the-art results. I would be willing to look at a revised version that fixes Eq. (7), reports the experimental protocol for baselines, and corrects the ablation table."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead DEGSTalk. The new idea is per-Gaussian learnable embeddings combined with a hair-preserving composition stage for 3DGS talking heads. That combination is not in the prior work they cite, and the reported numbers are consistently better than AD-NeRF, ER-NeRF, SyncTalk, and TalkingGaussian on all four metrics, with 114 FPS. The per-embedding deformation is a reasonable extension of TalkingGaussian's pipeline, and the ablation shows the hair-preserving stage matters (w/o H drops PSNR from 37.30 to 33.89). So this is a genuine engineering contribution, not a repackaging.\n\nThe soft spots are real. Equation (7), the defining rule, is not a valid alpha composite as printed. Chair is added without any opacity weighting, so in a fully opaque face pixel the hair radiance is still added on top, and the mouth term is weighted by (1−Oface), which makes the mouth contribution largest outside the face region. The text describes dilate operations and two extra rendering rounds, so possibly the equation is a sloppy summary of a more complex procedure, but as written the central mechanism is mis-specified. The paper also never defines Chair or a hair opacity. That has to be fixed before the head-to-head numbers can be trusted.\n\nThe evaluation is otherwise thin: no error bars, no significance tests, no statement of whether baselines were re-run under the same protocol. One ablation row is internally odd (w/o F has LPIPS 0.150 vs 0.0140 full, a tenfold jump that does not match its PSNR/SSIM). The benchmark is six videos, which is typical for this subfield but not a robust basis for claiming state-of-the-art.\n\nThe citation pattern looks fine. No circularity: metrics are against external videos.\n\nBottom line: the idea is worth a serious referee, but the paper needs a corrected derivation or a clear description of the fusion rule, and a proper evaluation protocol. I would accept it for review, not desk-reject, but I'd expect heavy revision. The stress-test concern about Eq. (7) is right on the money.","headline":"Promising incremental talking-face synthesis, but the defining hair-preserving fusion equation (7) is not a valid alpha composite as written and needs correction before the quantitative claims can be trusted.","tokens_in":8875,"tokens_out":3082,"would_cite":false,"duration_ms":29467,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 3D Gaussian talking-face model preserves long hair and beats four published baselines on all measured metrics.","keywords":["3D Gaussian Splatting","Talking Face Synthesis","Pre-Embedding","Hair-Preserving Rendering","Audio-Driven Animation","Deformable Gaussian Fields","Implicit 3DMM Coefficients","Real-Time Rendering"],"falsifier":"Re-run the four baselines and DEGSTalk on the same six clips using one shared preprocessing and metric implementation, and compare per-clip PSNR/LPIPS/LMD/SSIM; if the gap between DEGSTalk and TalkingGaussian shrinks to the noise level, the reported superiority is a protocol artifact rather than a property of the model. A complementary test is to train DEGSTalk on a held-out long-haired identity not among the six and inspect the hair/face boundary for ghosting or double edges, which would indicate the compositing does not generalize.","tokens_in":7755,"feed_emoji":"🗣️","tokens_out":12104,"duration_ms":105870,"temperature":0.7,"pith_summary":"This paper tries to establish that the two persistent failure modes of talking-face synthesis--distorted dynamic regions and broken long-hair rendering--can both be fixed inside a 3D Gaussian Splatting pipeline without losing real-time speed. The proposed DEGSTalk gives every Gaussian primitive a learnable embedding and deforms those primitives with audio plus implicit 3DMM expression coefficients, so subtle eye and mouth motion is not smeared by smooth neural fields. It then renders hair as a separately preserved region and composites it back into the face with an opacity-weighted formula, which the authors argue removes the double-chin and boundary artifacts seen in earlier methods. If the paper is right, long-haired talking avatars can be trained in about half an hour and rendered at 114 FPS while scoring highest on PSNR, LPIPS, LMD, and SSIM against four published baselines. The paper itself acknowledges that some noisy primitives may still occur.","feed_headline":"Per-Gaussian embeddings preserve long hair in talking-face video","feed_subtitle":"DEGSTalk separates hair from face, then fuses it back, beating four baselines at 114 FPS.","key_machinery":"The carrier is the Deformable Pre-Embedding Gaussian Field: each Gaussian primitive is assigned a learnable embedding $z \\in \\mathbb{R}^{32}$, and a shared MLP reads the encoded position-and-embedding combination $\\mathcal{H}(\\mu \\oplus z)$, concatenates audio features $f_a$ and implicit 3DMM coefficients $f_e$, and predicts the per-primitive deformation $\\{\\Delta \\mu, \\Delta s, \\Delta q\\}$, which is added to the canonical position, scale, and rotation before splatting. The second carrier is Dynamic Hair-Preserving Portrait Rendering: the face and mouth regions are reconstructed separately with hair excluded and then fused with the preserved hair by $C = C_{hair} + C_{face}O_{face} + C_{mouth}(1 - O_{face})$. The ablation study identifies this hair-preserving compositing as the largest single contributor to the reported quality.","core_discovery":"The paper's central claim is that the distortions and hair artifacts that plague talking-face synthesis can be removed by giving each 3D Gaussian primitive a learnable embedding and rendering hair as a separately preserved region. In DEGSTalk, audio features $f_a$ and implicit 3DMM coefficients $f_e = \\{\\psi_{id}, \\psi_{s}, \\psi_{exp}, \\psi_{eye}\\}$ drive an MLP that predicts per-Gaussian deformations $\\{\\Delta \\mu, \\Delta s, \\Delta q\\}$ from the primitive's position and embedding, so dynamic regions such as eyes and mouth are represented by primitives that know their own identity. Hair is reconstructed with the face and mouth regions excluded, then composited by the opacity-weighted formula $C = C_{hair} + C_{face}O_{face} + C_{mouth}(1 - O_{face})$, which the authors argue prevents the double-chin and boundary noise seen in earlier methods. On six HD long-hair portraits, the paper reports the best scores on all four measured metrics (PSNR, LPIPS, LMD, SSIM), with about 0.5 hours of training and 114 FPS inference, and it closes by noting that some noisy primitives may still occur.","pith_inferences":["Our inference: the per-embedding deformation mechanism is not inherently face-specific, so the same recipe could be tried on other region-structured dynamic scenes such as hands or clothed bodies, but the paper provides no evidence for that transfer.","Our inference: because the reported metrics are per-frame, temporal flicker at the hair boundary is not measured; a short-term consistency test on long sequences would be the natural next evaluation.","Our inference: the SOTA claim is data-limited--six clips from one set of speakers--so the margin should be re-checked on unseen identities and hairstyles before being treated as a general property; the paper's own closing remark about noisy primitives points in the same direction.","Our inference: the opacity-weighted compositing suggests a matting interpretation; replacing the hand-chosen fusion order with learned alpha compositing could be a testable extension."],"forward_implications":["Long-hair talking avatars can be trained in about half an hour and rendered at 114 FPS, removing the main practical barrier to real-time 3DGS avatars for subjects with long hair.","The ablation attributes the largest quality drop (PSNR from 37.30 to 33.89) to removing the hair-preserving rendering, so the compositing stage is the key component to reproduce in follow-up work.","The best-reported numbers indicate that, on the tested six-clip dataset, per-Gaussian embeddings and implicit 3DMM coefficients together push PSNR from 32.39 (TalkingGaussian) to 37.30.","Since fine-tuning updates only color parameters, the deformation and embedding modules are kept from overfitting during the final stage, which the paper says is what makes the fused result realistic."],"supporting_citations":[{"why":"Supplies the single-image 3D reconstruction that predicts the implicit 3DMM coefficients (identity, shape, expression, eyelid, jaw) used as the facial feature input.","marker":"[6]"},{"why":"AD-NeRF is the first baseline; its reported quality and 0.16 FPS speed anchor the comparison DEGSTalk's real-time claim is measured against.","marker":"[18]"},{"why":"One of the dataset sources for the six training/evaluation video clips, and a prior NeRF talking-portrait approach the method builds on.","marker":"[19]"},{"why":"ER-NeRF is a region-aware NeRF baseline whose face/mouth separation DEGSTalk extends with the hair-preserving compositing.","marker":"[20]"},{"why":"SyncTalk supplies the pose-estimation protocol for the experiments and is a baseline; its synchronization framework is the reference for head-pose handling.","marker":"[21]"},{"why":"GeneFace is another dataset source and a 3DMM-based talking-face baseline lineage that the implicit coefficient representation continues.","marker":"[22]"},{"why":"3D Gaussian Splatting is the underlying explicit representation and rasterizer through which the deformed primitives are rendered.","marker":"[24]"},{"why":"TalkingGaussian is the direct 3DGS baseline and structural template: DEGSTalk inherits its three-stage training and separate face/mouth reconstruction, adding per-Gaussian embeddings and hair-preserving fusion.","marker":"[25]"}],"fun_headline_variants":["Hair-preserving AI talking faces with per-Gaussian embeddings","DEGSTalk: hair keeps moving in AI face videos","Per-embedding 3D Gaussians fix hair in talking faces","Talking faces that keep hair real, 114 FPS","Decomposed Gaussians save long hair in talk videos"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported advantage over AD-NeRF, ER-NeRF, SyncTalk, and TalkingGaussian assumes all four baselines were measured on the same clips with the same crop, pose, and metric code; the paper does not say whether these baselines were re-run under that protocol.","fun_headline_variants_meta":{"raw":{"variants":["Hair-preserving AI talking faces with per-Gaussian embeddings","DEGSTalk: hair keeps moving in AI face videos","Per-embedding 3D Gaussians fix hair in talking faces","Talking faces that keep hair real, 114 FPS","Decomposed Gaussians save long hair in talk videos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000559,"raw_usage":{"total_tokens":2673,"prompt_tokens":976,"completion_tokens":1697,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":1610}},"tokens_in":592,"tokens_out":1697,"duration_ms":12062,"temperature":1.0,"reasoning_tokens":1610,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:29:39.423121+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the four baselines and DEGSTalk on the same six clips using one shared preprocessing and metric implementation, and compare per-clip PSNR/LPIPS/LMD/SSIM; if the gap between DEGSTalk and TalkingGaussian shrinks to the noise level, the reported superiority is a protocol artifact rather than a property of the model. A complementary test is to train DEGSTalk on a held-out long-haired identity not among the six and inspect the hair/face boundary for ghosting or double edges, which would indicate the compositing does not generalize.","supporting_citations":[{"cited_title":"3d facial expressions through analysis-by-neural-synthesis,","cited_arxiv_id":null,"evidence_quote":"Supplies the single-image 3D reconstruction that predicts the implicit 3DMM coefficients (identity, shape, expression, eyelid, jaw) used as the facial feature input."},{"cited_title":"Ad-nerf: Audio driven neural radiance fields for talking head synthesis,","cited_arxiv_id":null,"evidence_quote":"AD-NeRF is the first baseline; its reported quality and 0.16 FPS speed anchor the comparison DEGSTalk's real-time claim is measured against."},{"cited_title":"Efficient region-aware neural radiance fields for high-fidelity talking portrait synthesis,","cited_arxiv_id":null,"evidence_quote":"ER-NeRF is a region-aware NeRF baseline whose face/mouth separation DEGSTalk extends with the hair-preserving compositing."},{"cited_title":"SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis","cited_arxiv_id":"2311.17590","evidence_quote":"SyncTalk supplies the pose-estimation protocol for the experiments and is a baseline; its synchronization framework is the reference for head-pose handling."},{"cited_title":"Geneface: Generalized and high-fidelity audio-driven 3d talking face synthesis,","cited_arxiv_id":null,"evidence_quote":"GeneFace is another dataset source and a 3DMM-based talking-face baseline lineage that the implicit coefficient representation continues."}],"review_version":1}