{"id":"e02cfc83-f4c2-42a5-8c88-26a536a2690f","arxiv_id":"2508.08248","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A diffusion-based avatar generator that uses a timestep-aware audio adapter, audio-adaptive guidance, and weighted sliding-window fusion to produce audio-synced talking-head videos several minutes long with reduced identity and quality drift.","lead":"StableAvatar adds a timestep-aware audio adapter and a modified guidance scheme to a video diffusion model so that talking-head videos stay sharp, identity-consistent, and lip-synced for minutes instead of seconds. The authors report beating 14B-parameter rivals with a 1.3B backbone, but the 'infinite-length' claim is tested only up to about five minutes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'without post-processing' claim is contradicted by the paper's own Dynamic Weighted Sliding-Window Strategy (Sec.","rationale":"The reader's weakest assumption was fairness of the SOTA comparison on the private Long100 benchmark. That concern is valid and important: all competitors were retrained on the authors' private 1200-hour dataset, and no code, data, or training recipes are released, so the reported 80-85% gains over OmniAvatar on Long100 cannot be independently verified. However, I found a more direct threat to the central claim. The paper explicitly promises 'without post-processing' and 'end-to-end' synthesis, but Sec. 3.3's Dynamic Weighted Sliding-Window Strategy is an inference-time latent-fusion mechanism that blends overlapping windows. The authors' own Table 6 ablation shows that removing this fusion substantially degrades all metrics (FVD 718 vs 532, CSIM 0.842 vs 0.853, Sync-C 8.18 vs 8.20). Thus DWSW is not a minor detail; it is the stitching mechanism that makes long videos smooth. Calling the result 'without post-processing' is therefore misleading unless one adopts an unusually narrow definition of post-processing that excludes latent-space fusion during denoising. Additionally, Algorithm 1 takes a fixed total latent length L as input and never describes how to extend generation beyond that L, so the 'infinite-length' claim is an extrapolation from 3-5 minute demonstrations rather than a property of the algorithm. I give credit where due: the timestep-aware audio adapter, the guidance mechanism, and the internal ablations are plausible and mutually consistent, and the qualitative results are suggestive. But the headline claim overstates what is demonstrated. This does not require rejecting the paper; it requires rewording the claims and releasing code or a reproducible evaluation protocol. Since the reader already assigned CONDITIONAL, my analysis does not move the verdict.","tokens_in":19403,"tokens_out":10762,"duration_ms":126240,"concrete_test":"Set the overlap length m=0 in Algorithm 1 (i.e., disable the weighted fusion, keeping all other modules) and generate the same Long100 clips with the released checkpoints; if FVD/CSIM/Sync-C degrade toward the w/o DWSW row of Table 6 and stitching artifacts appear at window boundaries, DWSW is performing post-hoc latent stitching, falsifying the 'without post-processing' part of the central claim. If instead the no-fusion variant matches the reported quality, the claim may survive but the paper's own ablation is then misleading.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Abstract and Sec. 4.2 claim that StableAvatar is the first end-to-end video diffusion transformer synthesizing infinite-length videos without post-processing. The method's own inference procedure contradicts this. Algorithm 1 and Sec. 3.3 describe DWSW, which denoises overlapping windows and fuses their latents at every timestep using a logarithmic weighting function. This is an explicit inference-time fusion/stitching step: overlapping latents from separately processed windows are blended and injected back into adjacent windows. The authors' Table 6 shows this fusion is necessary: removing DWSW raises FVD from 532 to 718 and lowers CSIM and Sync-C. So the 'without post-processing' claim is at best a semantic distinction between decoding-time and latent-space post-processing. Moreover, Algorithm 1 is defined for a fixed total latent length L; no streaming or autoregressive extension protocol is given, so 'infinite-length' is supported only by 3-5 minute demos, not by the algorithm. The central claim should be revised to: finite-length long-video generation with a latent sliding-window fusion step.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"StableAvatar proposes an audio-driven avatar video generation system built on the Wan2.1-1.3B video diffusion transformer, with three components: a Timestep-aware Audio Adapter that refines Wav2Vec embeddings through timestep-conditioned modulation and cross-attention with latents; an Audio Native Guidance mechanism intended to replace classifier-free guidance using a joint audio-latent score; and a Dynamic Weighted Sliding-Window Strategy (DWSW) that fuses overlapping window latents during inference. The paper reports state-of-the-art results on HDTF, AVSpeech, and a privately collected Long100 benchmark, with quantitative gains in identity consistency (CSIM) and lip synchronization (Sync-C) over competitors including Wan2.1-14B-based OmniAvatar. The authors claim that StableAvatar is the first end-to-end diffusion transformer capable of infinite-length, post-processing-free video generation, supported by qualitative demos of videos exceeding 3500 frames.","tokens_in":19633,"tokens_out":3379,"duration_ms":35652,"significance":"If the empirical claims hold, this is a practically significant contribution: it would demonstrate that a 1.3B-parameter video diffusion model can maintain identity and audio-lip synchronization for multi-minute avatar videos, at substantially lower inference cost than 14B-parameter competitors. The ablations are informative and support the qualitative claim that each proposed component degrades performance when removed, and the supplementary limitation section (Sec. A.10) and failure-case figure are honest disclosures. However, the headline claims of 'infinite-length' and 'without post-processing' are not supported by the algorithm as presented, and the SOTA comparison rests on an unverifiable retraining protocol on a private dataset. The paper is a solid engineering contribution in need of claim revision and additional validation before it can be accepted.","major_comments":[{"comment":"The claim 'infinite-length high-quality videos without post-processing' is contradicted by the paper's own inference procedure. Algorithm 1 takes a fixed total latent length L as input, denoises overlapping windows, and at each timestep fuses latents from adjacent windows with a logarithmic weighting and injects the fused latents back into both windows. This is an explicit inference-time stitching/fusion step performed in latent space. Table 6 shows the step is not cosmetic: removing DWSW raises FVD from 532 to 718. The central claim should be revised to 'finite-length long-video generation with a latent sliding-window fusion step'; no streaming or autoregressive extension protocol is given, so 'infinite-length' is not supported by the algorithm or by the 3-5 minute demos.","section":"Abstract; Sec. 3.3 and Algorithm 1"},{"comment":"The Audio Native Guidance derivation contains unstated approximations and notational inconsistencies. Eq. (6) factorizes the modified sampling density into pθ(A|[zt,ā])^α pθ(ā|zt,A)^β without stating the assumed conditional-independence structure. In going from Eq. (7) to Eq. (9), the score function mixes three different conditioning sets: ∇log pθ([zt,ā]|A), ∇log pθ([zt,ā]), and ∇log pθ(zt|A). The final form in Eq. (10), however, implements the last term as D([zt,∅], y, I, A;θ), whose conditioning includes text and image inputs and whose first argument [zt,∅] does not match pθ(zt|A). The authors should state precisely which density each term denotes and justify the approximation that ∇log pθ(zt|A) can be evaluated by the network with the audio as the only conditioning signal.","section":"Sec. 3.2, Eqs. (6)-(10)"},{"comment":"The quantitative superiority claim rests on a comparison protocol that cannot be independently verified. All baselines are retrained on the authors' private 1200-hour dataset and evaluated on the private Long100 benchmark, but no details are given about training budgets, hyperparameters, checkpoints, or the selection and prompt protocol for Long100. The reported margins on Long100 (CSIM 0.849 vs. 0.471; Sync-C 8.24 vs. 4.45) could be inflated by undertuned baselines or by benchmark choices that favor the proposed design. Additionally, no error bars or statistical significance tests are provided for any table. The authors should release the Long100 protocol or report multiple seeds and variance, and should provide per-baseline training details.","section":"Sec. 4.1-4.2 and Table 1"}],"minor_comments":[{"comment":"The same 'Ours' configuration on Long100 reports FVD=504 in Table 2 but FVD=532 in Tables 5 and 6; the discrepancy is not explained and should be clarified (e.g., different seeds, evaluation subsets, or corrected numbers).","section":"Tables 2, 5, 6"},{"comment":"The text says 'previous sling-window denoising strategy'; 'sling' should be 'sliding'. Also, the pseudo-code in Algorithm 1 uses 'np.linspace(0, 1, num samples=m)' and a subsequent normalization that is formatted ambiguously; the notation should be cleaned up.","section":"Sec. 3.3"},{"comment":"The contributions list skips item (3): items are numbered (1), (2), (4). This is a minor editorial issue.","section":"Sec. 1, contributions"},{"comment":"The row label 'w/o Aduio Adapter' contains a typo and should read 'w/o Audio Adapter'.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's central mechanism is plausible and the ablations are somewhat convincing, but the discrepancy between the 'infinite-length, no post-processing' claim and the actual DWSW algorithm is a matter of presentation that the authors can fix. The more serious concern is the Long100 comparison: retraining all competitors on a private dataset with no released protocol or error bars makes the reported SOTA margins unverifiable. I would encourage the editor to ask for either release of the Long100 evaluation protocol or a comparison on a public benchmark with fixed baselines. The derivation issue in Eqs. (6)-(10) is fixable with a cleaner formulation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the core new component—a timestep-aware audio adapter that modulates audio embeddings with the diffusion timestep—is a sensible fix for drift in long audio-driven avatars, and the ablations show it matters. Second, the headline claim of \"infinite-length without post-processing\" is overstated: their Dynamic Weighted Sliding-Window (DWSW) step fuses overlapping latents at inference time, which is post-processing by any ordinary reading, and Algorithm 1 is defined for a fixed total length L with no streaming extension. The stress-test note is right on this.\n\nWhat the paper does well: the adapter design is plausible, and the ablations support it (removing it drops CSIM from 0.849 to 0.457 on Long100). The audio-native guidance beats CFG in their Table 5. The DWSW weighting helps smoothness relative to fixed weights. The efficiency story is real: a 1.3B backbone beats 14B-based competitors on their long benchmark while using half the GPU memory.\n\nSoft spots, in proportion:\n\n- \"Without post-processing\" is doing illegitimate work. DWSW blends and re-injects latents across windows; that is stitching at the latent level, not single-pass generation.\n- \"Infinite-length\" is not supported beyond 3–5 minute demos. There is no autoregressive or streaming mechanism; the algorithm requires a pre-specified L.\n- The Long100 comparison retrains all competitors on the authors' private 1200-hour dataset. If baselines are undertuned or the benchmark favors their design, the large CSIM/Sync-C gains are suspect. No code, no data, no error bars—a real reproducibility gap for a claims-heavy systems paper.\n- The guidance derivation (Eq. 6–10) is loose: the step from a joint distribution to a score combination is heuristic, not derived. Acceptable as a trick, but it should be labeled as such.\n\nMinor: the contribution list skips item (3), and \"first\" is a strong claim against a fast-moving literature.\n\nBottom line: the core mechanism is credible and the ablation evidence is the strongest part. The paper deserves serious refereeing, but the claims need scaling back and the evaluation should be public or shared with reviewers. I would recommend conditional acceptance after major revision. I would bring it to a reading group for the adapter mechanism and the framing debate, but I would not cite it yet without code.","headline":"The audio adapter idea is real and the ablations back it, but the 'infinite-length without post-processing' claim collapses on contact with the method's own sliding-window fusion.","tokens_in":20147,"tokens_out":2889,"would_cite":false,"duration_ms":31192,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"StableAvatar generates infinite-length, audio-synchronized avatar videos end-to-end, preserving reference identity beyond 3500 frames.","keywords":["audio-driven avatar video generation","video diffusion transformer","infinite-length video generation","latent distribution error accumulation","timestep-aware audio adapter","audio native guidance","lip synchronization","identity consistency"],"falsifier":"Generate one audio-driven video of 3500+ frames with StableAvatar and one with the same backbone but direct audio injection, then compare the denoised-latent distributions of frames 1-200 and frames 3500-3700 with FID or a divergence measure: the paper's error-accumulation story predicts near-stationarity for StableAvatar and clear drift for the baseline. A public head-to-head using released competitor checkpoints on a public long-video set, without the private retraining protocol, would separately settle whether the reported margins survive.","tokens_in":19213,"feed_emoji":"🎤","tokens_out":16876,"duration_ms":155969,"temperature":0.7,"pith_summary":"StableAvatar sets out to remove the roughly 15-second ceiling on audio-driven avatar video generation. The paper's diagnosis is that existing models degrade on long videos because they inject off-the-shelf audio embeddings directly into a diffusion backbone that has no audio priors, so every generated segment leaves the latent distribution slightly off target and the error accumulates across segments until faces distort, colors drift, and identity washes out. Its proposed cure is a Timestep-aware Audio Adapter that conditions the audio representation on the denoising state itself, making the model learn a joint audio-latent distribution at each step, supported by an inference-time Audio Native Guidance that sharpens lip-sync and a dynamic weighted sliding-window fusion that smooths clip boundaries. If the claim holds, avatar videos of minutes to hours can be generated in a single pass from one reference image and an audio track, with identity and lip-sync preserved past 3500 frames, using a 1.3-billion-parameter model rather than a 14-billion one. The supporting experiments report large margins over the strongest competitor on the authors' private Long100 benchmark (identity similarity 0.849 versus 0.471).","feed_headline":"One diffusion model keeps avatar identity stable past 3500 frames","feed_subtitle":"Audio-synced talking-head video runs minutes to hours from a single image, no stitching needed.","key_machinery":"The load-bearing object is the Timestep-aware Audio Adapter. Raw Wav2Vec embeddings are concatenated with neighboring audio context frames, projected by an MLP, and then used as Query in a cross-attention block whose Key and Value are the noisy latents $z_t$; the outputs are scaled and shifted by the same timestep embeddings the DiT uses internally. Because diffusion pretraining ties each timestep embedding to a specific latent distribution, this modulation implicitly binds the audio representation to the latent distribution of the current denoising step, producing refined embeddings $\\bar{a}_t$ that are then injected into the backbone by cross-attention. The inference mechanisms build on the same idea: Audio Native Guidance rewrites the denoising score as a weighted combination of the joint audio-latent prediction, the unconditional prediction, and the audio-conditioned latent prediction, treating $\\bar{a}_t$ as an extra model output; the Dynamic Weighted Sliding-Window strategy blends overlapping latent windows with logarithmically spaced weights so both edges of every window are fused from both sides. The adapter is what carries the argument: it converts audio conditioning from an external signal the backbone distrusts into a function of the model's own evolving latent state.","core_discovery":"The central claim, put as the authors would put it, is that the long-video failure of audio-driven avatar models is not a defect of the video backbone but of the audio conditioning path. Since diffusion backbones carry no audio priors, embeddings from an off-the-shelf extractor that are injected by cross-attention push each successive video segment's latents slightly off the target distribution; across many segments the error accumulates, and identity, color, and lip-sync collapse. StableAvatar's answer is to make the audio conditioning depend on the denoising state: the Timestep-aware Audio Adapter lets audio tokens cross-attend with the noisy latents and modulates them with the timestep embeddings, so the conditioning signal is forged from the joint audio-latent distribution at every step rather than handed in from outside. At inference, Audio Native Guidance replaces classifier-free guidance by steering the sampling score toward that same joint distribution with the refined audio embedding treated as an additional prediction, and a dynamic weighted sliding-window strategy fuses overlapping latent windows to keep long videos smooth. On Long100, the paper's benchmark of 100 unseen videos of two to five minutes, the 1.3B-parameter StableAvatar is reported to keep identity similarity at 0.849 and lip-sync at 8.24 while the strongest Wan2.1-14B-based competitor, OmniAvatar, drops to 0.471 and 4.45; the model is reported stable past 3500 frames in a single pass.","pith_inferences":["If the causal story is right, the recipe generalizes: any diffusion model that injects off-the-shelf embeddings from a second modality into latent space without cross-modal priors should show the same cumulative drift in long-form generation, and a timestep-aware adapter is a portable cure for text-, pose-, or emotion-conditioned video.","The paper's own 'w/Random modulation' ablation (CSIM 0.632 versus 0.849) isolates the timestep coupling as the source of most of the gain; re-running that ablation across datasets and seeds would confirm it is the coupling, not the extra parameters, doing the work.","The paper itself reports a boundary case: given a reference far outside human morphology, such as a fantastical creature, the model cannot locate the lips and lip-sync fails; the infinite-length guarantee is therefore implicitly scoped to human-like references and would need an explicit reference-semantics module to extend further.","A decisive independent check would re-run the comparison with released competitor checkpoints on a public long-video benchmark, since the reported margins come from retraining all models on the authors' private 1200-hour dataset and scoring on the private Long100."],"forward_implications":["Avatar videos longer than 15 seconds no longer require clip stitching or post-processing: the authors report quality that stays flat across thousands of frames, making video length a matter of compute and memory rather than drift.","A 1.3B-parameter model can beat 14B-based systems on long-form avatar video, because the binding constraint is the conditioning path, not scale.","Inference-time smoothing alone is not enough: the ablations show motion-frame and conventional sliding-window strategies still collapse to FVD values near 1800-2000 and CSIM near 0.4, whereas the full model reaches 504 and 0.849.","Audio Native Guidance outperforms classifier-free guidance at no training cost (Sync-C 8.20 versus 7.62) and skips image and text as guidance inputs, saving memory.","The same recipe claims roughly half the GPU memory and about ten times the inference speed of the leading competitor, because the effective fix is architectural, not a bigger backbone."],"supporting_citations":[{"why":"Supplies the off-the-shelf Wav2Vec audio embeddings whose direct cross-attention injection is diagnosed as the source of latent error accumulation; the adapter also takes them as input.","marker":"[1]"},{"why":"The Wan2.1 video diffusion transformer that StableAvatar adapts; its lack of audio priors is the premise of the diagnosis, and the 1.3B base sets up the scaling comparison with 14B competitors.","marker":"[66]"},{"why":"OmniAvatar, the leading Wan2.1-14B-based competitor whose Long100 scores (CSIM 0.471, Sync-C 4.45) form the main baseline the paper claims to beat.","marker":"[14]"},{"why":"Classifier-free guidance, the standard inference-time conditioning that Audio Native Guidance replaces and is ablated against.","marker":"[23]"},{"why":"Hallo3 supplies training data and the motion-frame long-video strategy that the ablations show still collapses; also a comparison competitor.","marker":"[10]"},{"why":"Sonic provides the conventional sliding-window long-video strategy that DWSW is compared against and outperforms.","marker":"[27]"},{"why":"SyncNet defines the Sync-C/Sync-D lip-sync metrics behind the synchronization claims and is also used to filter the training data.","marker":"[9]"},{"why":"SadTalker is the source of the CSIM identity-similarity metric and a GAN-based competitor in the comparisons.","marker":"[81]"}],"fun_headline_variants":["Audio conditioning fix stops avatar drift past 3500 frames","StableAvatar ends long-video lip-sync and identity collapse","Timestep-aware audio adapter keeps avatar faces stable in long video","Why audio-driven avatars drift: cross-attention error, not backbone","Diffusion model holds avatar identity past 3500 frames, no stitching"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline margins rest on the assumption that retraining every competitor on the authors' private 1200-hour dataset and judging them on the private Long100 benchmark is a genuinely fair comparison, rather than a protocol that happens to favor StableAvatar's design.","fun_headline_variants_meta":{"raw":{"variants":["Audio conditioning fix stops avatar drift past 3500 frames","StableAvatar ends long-video lip-sync and identity collapse","Timestep-aware audio adapter keeps avatar faces stable in long video","Why audio-driven avatars drift: cross-attention error, not backbone","Diffusion model holds avatar identity past 3500 frames, no stitching"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1644,"prompt_tokens":1085,"completion_tokens":559,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":701,"completion_tokens_details":{"reasoning_tokens":468}},"tokens_in":701,"tokens_out":559,"duration_ms":6422,"temperature":1.0,"reasoning_tokens":468,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:36:06.754629+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate one audio-driven video of 3500+ frames with StableAvatar and one with the same backbone but direct audio injection, then compare the denoised-latent distributions of frames 1-200 and frames 3500-3700 with FID or a divergence measure: the paper's error-accumulation story predicts near-stationarity for StableAvatar and clear drift for the baseline. A public head-to-head using released competitor checkpoints on a public long-video set, without the private retraining protocol, would separately settle whether the reported margins survive.","supporting_citations":[{"cited_title":"Hallo3: Highly dynamic and realistic portrait image animation with video diffusion transformer","cited_arxiv_id":null,"evidence_quote":"Hallo3 supplies training data and the motion-frame long-video strategy that the ablations show still collapses; also a comparison competitor."},{"cited_title":"Sonic: Shifting focus to global audio perception in portrait animation","cited_arxiv_id":null,"evidence_quote":"Sonic provides the conventional sliding-window long-video strategy that DWSW is compared against and outperforms."},{"cited_title":"Out of time: auto- mated lip sync in the wild","cited_arxiv_id":null,"evidence_quote":"SyncNet defines the Sync-C/Sync-D lip-sync metrics behind the synchronization claims and is also used to filter the training data."},{"cited_title":"Sadtalker: Learning realistic 3d motion coefficients for stylized audio- driven single image talking face animation","cited_arxiv_id":null,"evidence_quote":"SadTalker is the source of the CSIM identity-similarity metric and a GAN-based competitor in the comparisons."}],"review_version":1}