{"id":"ee0333a1-bea6-4a5b-b6ca-7b41ba13baf9","arxiv_id":"2411.16810","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Sign-D2C uses a conditional diffusion model with random masking and linear interpolation padding to generate transition poses between discrete signs, but its evaluation is largely a same-task reconstruction rather than a realistic discrete-to-continuous test.","lead":"This paper trains a diffusion model to fill in missing sign-language pose frames between discrete sign segments, with the aim of producing continuous signing videos from word-level clips. It relies on random masking of long videos for training and linear interpolation for initialization, but the quantitative evidence mostly measures reconstruction of masked frames rather than the actual transition-generation task.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim—that Sign-D2C generates smooth transitions between discrete sign segments—is never quantitatively tested: Tables 1–5 measure masked-frame reconstruction inside continuous videos, and the only discrete-segment experiment (USTC-SLR500) is qualitative.","rationale":"In good faith, the paper proposes a sensible self-supervised proxy: mask portions of continuous signing videos and train a diffusion model to recover the missing poses. If the proxy were faithful, the method could plausibly generate transitions between discrete signs. But the central claim is stated as generating smooth, contextually accurate transitions for continuous sign videos assembled from discrete segments, and the evidence never measures that outcome quantitatively. Every table evaluates reconstruction of frames removed from the same continuous video, which is a substantially easier and different task than synthesizing a transition between two independently recorded clips. The discrepancy is not merely a matter of dataset domain; it affects the fundamental validity of the training signal and the evaluation metric. The USTC-SLR500 experiment is precisely the real-world discrete-to-continuous case, yet it is presented only through visual examples, with no DTW, FID, MPJPE, BLEU, WER, or human study. The baselines G2P-DDM and VQ-GCDM are also not designed for observation-conditioned inbetweening, so the reported gains over them do not establish superiority on the stated task. I do not claim the method cannot work; the masking idea is reasonable and the architecture is plausible. But as submitted, the manuscript does not support the central application. The reader's verdict of REJECT is appropriate: the method could become viable with a quantitative discrete-segment evaluation and fair baselines, but the current evidence does not demonstrate the central claim.","tokens_in":47,"tokens_out":2368,"duration_ms":59002,"concrete_test":"Run a quantitative evaluation on USTC-SLR500 (or an equivalent isolated-sign dataset) in which two separately recorded word signs are concatenated and the model must generate the transition frames, with signers or word pairs held out from any masked-frame training. Compare against a linear-interpolation baseline and an observation-conditioned inpainting baseline under identical conditions, using DTW, FID, MPJPE, and either a semantic metric (BLEU/WER via NSLT) or human ratings of naturalness. If the model does not clearly beat these controls on the concatenated-sign task, the discrete-to-continuous claim is unsupported. At minimum, report the USTC-SLR500 results numerically rather than only qualitatively.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's stated task is to assemble isolated word-level signs into continuous signing by generating transition poses between discrete segments (Sec. 1, inference phase). The training and all quantitative evaluations do not test that task. Section 3.2 simulates missing transitions by randomly masking portions of a single continuous sign sequence, and Tables 1–5 report how well the model reconstructs those masked frames from surrounding frames of the same video. That is conditional inpainting or interpolation within a continuous take, not discrete-to-continuous assembly. Frames adjacent to a masked gap in one continuous sentence share the same signer, recording conditions, and grammatical context; transitions between separately recorded lexicon videos involve different boundary poses, coarticulation, and potentially different signers and camera conditions. The only experiment on the actual discrete-to-continuous scenario is USTC-SLR500 (Sec. 4.4, Fig. 4), which reports no metrics, only visual examples. Consequently, the headline numbers—BLEU-1 of 69.64 and DTW of 1.92 on USTC-CSL100 under 20/10—support only masked-frame reconstruction, not the central claim of smooth discrete-to-continuous generation. The baseline comparison is also not evidence for the central claim: G2P-DDM and VQ-GCDM are gloss-conditioned sign pose generators, not observation-conditioned inbetweeners, so their poor results largely reflect task mismatch. The load-bearing premise is that random masking of continuous video faithfully mimics transitions between discrete signs; the manuscript provides no quantitative check of that premise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Sign-D2C is a conditional diffusion framework intended to generate smooth transition frames between discrete sign-language pose segments so that isolated signs can be assembled into continuous signing. During training, the authors randomly mask segments of continuous signing videos and train the model to reconstruct the masked frames from the remaining observed frames. At inference, a linearly interpolated padding initializes the missing gap, which the diffusion model then refines. The paper reports ablations of masking ratio and padding strategy, comparisons against G2P-DDM and VQ-GCDM on PHOENIX14T and USTC-CSL100, and qualitative examples on USTC-SLR500.","tokens_in":13450,"tokens_out":7937,"duration_ms":67421,"significance":"If the proposed method actually delivered smooth, meaning-preserving transitions between independently recorded word-level signs, it would address a genuine bottleneck in sign-language production and synthesis, where continuous sentence-level data are scarce. The idea of converting the unsupervised transition-generation problem into a supervised masked-reconstruction task is appealing, and the intended use of a conditional diffusion model is reasonable. The paper also provides ablations on masking ratios and padding strategies, and the inclusion of three datasets is appropriate. However, the current experimental design does not quantitatively evaluate the stated discrete-to-continuous task, and the numerical results therefore cannot substantiate the central claim.","major_comments":[{"comment":"The training and all quantitative evaluations operate on a single continuous signing video, masking frames and reconstructing them from surrounding frames of the same video. The paper's stated task (Sec. 1 and the inference phase in Fig. 1) is instead to synthesize transitions between discrete, separately recorded sign segments. Because observed and masked frames in the same continuous sequence share signer, camera, and grammatical context, performance on the masked-reconstruction benchmarks does not measure generalization to the discrete-to-continuous setting. The only experiment on that setting, Sec. 4.4 and Fig. 4 for USTC-SLR500, reports no quantitative metrics. The headline numbers, e.g., BLEU-1 of 69.64 and DTW of 1.92 in Table 5, thus support only conditional inpainting within continuous videos, not the paper's central claim of generating smooth transitions between discrete segments. This must be addressed with either a quantitative evaluation of the discrete-to-continuous scenario or a substantial reframing of the paper's claims.","section":"Sec. 3.2 and Tables 1–5"},{"comment":"The comparison against G2P-DDM and VQ-GCDM is not informative for the transition-generation task. These baselines are gloss-to-pose generators that synthesize an entire pose sequence from a gloss input, rather than observation-conditioned in-betweeners that must respect the surrounding pose observations. Their poor BLEU and high DTW may simply reflect that they are solving a different, harder problem under the authors' adaptation. The paper should compare against observation-conditioned inpainting baselines, including the trivial linear interpolation baseline and a frame-interpolation method, and should report how the gloss-to-pose baselines were adapted to the masked-setting protocol.","section":"Sec. 4.3 and Tables 3–5"},{"comment":"The formula labeled \"linear interpolation padding\" does not implement linear interpolation. The iterate f(i) = sum_{k=0}^{i} 1/(k+1) produces cumulative harmonic weights, so the intermediate poses pc(i) are spaced by decreasing increments rather than uniformly between ps and pe. As written, the equation contradicts the surrounding text and the \"Linear Padding\" experimental results in Table 2. The authors must correct the equation or clarify the actual initialization used in inference.","section":"Sec. 3.3, Eq. (8)"},{"comment":"Because only 10 of 30 frames are predicted in the main 20-observed/10-predicted setting, the reported DTW and MPJPE over the full sequence may be dominated by the observed frames, which are identical to ground truth. A simple copy-observed baseline would likely already achieve very low DTW. The paper should report coherence metrics computed only on the masked (predicted) frames and should include a no-generation baseline such as direct linear interpolation on the same metrics, to assess the incremental value of the diffusion refinement.","section":"Sec. 4.2, Table 2 and Sec. 4.1"}],"minor_comments":[{"comment":"The text states that the \"w/o Padding\" approach yields a TEST DTW of 10.36, but the table reports 11.60; the numbers are inconsistent.","section":"Sec. 4.2, Table 2"},{"comment":"The text says \"As shown in Table 4\" when referring to the 20-observed/10-predicted comparison, but Table 3 is the 20/10 table; the citation appears to be wrong.","section":"Sec. 4.3"},{"comment":"In the USTC-CSL100 paragraph, the text mentions an \"FID score of 3.21\" for the 20/10 setting, while Table 5 reports FID 0.31 for Ours; the text likely has a decimal error.","section":"Sec. 4.3"},{"comment":"The contribution list spells the dataset \"UCST-CSL100\" while the rest of the paper uses \"USTC-CSL100\".","section":"Sec. 1"},{"comment":"The noise-adding notation is inconsistent: Eq. (1) uses sqrt(1 - alpha_t) while Eq. (4) uses sqrt(1 - alpha_t^2), with no definition of alpha_t in either place.","section":"Sec. 3.3, Eqs. (1) and (4)"}],"recommendation":"major_revision","confidential_remarks":"The evaluation gap is the central concern. If the authors cannot provide quantitative evidence for the discrete-to-continuous setting (for example, a user study or a downstream-recognition measure on stitched sequences), the paper should not be published as is. The baseline comparison also needs to be rethought."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper has a sensible method and a serious evaluation gap. The random-masking approach to supervising transition generation is a neat idea, and the linear interpolation padding is a reasonable initialization for a diffusive refiner. The writing is clear and the related work is appropriately cited. Credit where due: the masking scheme makes a genuinely unsupervised problem supervised, and the ablation in Table 2 shows the padding helps.\n\nThe problem is that every quantitative result in Tables 1–5 tests masked-frame reconstruction inside continuous sign videos, not the task advertised in the title. Masking 10 frames out of 30 in a continuous sentence and reconstructing them from 20 observed frames is conditional inpainting, not assembling two isolated vocabulary signs into a sentence. The observed frames dominate the semantic metrics; with 20 of 30 frames observed, even a frozen frame copy would give high BLEU. The baselines G2P-DDM and VQ-GCDM are gloss-to-pose models, not observation-conditioned inbetweeners, so their poor numbers are unsurprising and do not demonstrate an advantage. The only experiment on the real discrete-to-continuous scenario (USTC-SLR500, Fig. 4) is qualitative. That is a load-bearing gap.\n\nThere are also smaller technical problems. Equation 8 is mathematically wrong as written: f(i) is defined as a harmonic sum, which does not give linear interpolation, and pc(0) does not equal ps. Table references are mislabeled in Section 4.3. These are fixable, but they need fixing.\n\nI don't think this is a reject because the method is silly—the core idea is plausible and could be useful if evaluated properly. But as it stands, the manuscript doesn't demonstrate what it claims. The right fix is a quantitative experiment on the actual stitching task: take isolated signs from a dataset like USTC-SLR500, concatenate them, generate transitions, and measure something (pose distance to a human-provided transition, or a signer/observer study). Also add an interpolation-only baseline, which would show whether the diffusion refiner adds value over linear blending.\n\nWho this is for: researchers in sign language production and synthesis, especially those working on dictionary-to-sentence assembly. It's a reasonable incremental contribution, but the evaluation needs a major overhaul before the central claim is credible.","headline":"Plausible diffusion-based transition generator, but the evaluation measures masked-frame reconstruction, not the claimed discrete-to-continuous stitching task.","tokens_in":14076,"tokens_out":2881,"would_cite":false,"duration_ms":27653,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Randomly masking frames in continuous sign videos turns transition generation into a supervised problem, and a conditional diffusion model learns to fill the gaps smoothly.","keywords":["sign language production","transition pose generation","conditional diffusion model","random masking","pose synthesis","video generation","sign language synthesis"],"falsifier":"One concrete check would be to assemble a test set of continuous sentences for which ground-truth videos exist, artificially split each sentence into two isolated segments at a cut point inside a transition, run Sign-D2C to reconnect them, and measure the mismatches (DTW, MPJPE, and human fluency ratings) between the generated bridging frames and the actual frames at that cut; if the error is no better than simple interpolation for transitions that cross signer or recording-session boundaries, the masking assumption would fail to transfer to realistic concatenation.","tokens_in":12950,"feed_emoji":"🤟","tokens_out":4797,"duration_ms":40540,"temperature":0.7,"pith_summary":"The paper proposes Sign-D2C, a framework that generates the missing transition poses needed to stitch isolated word-level signs into continuous sign language sentences. The key trick is to convert the unsupervised task of transition generation into a supervised one: during training, random segments of long continuous signing videos are masked out, and a conditional diffusion model learns to reconstruct those masked frames from the surrounding visible frames. At inference, missing frames between two discrete sign segments are initialized by linear interpolation between the boundary poses and then refined by the same diffusion model. On sentence-level benchmarks the method reports semantic accuracy close to ground truth (BLEU-1 of 69.64% on USTC-CSL100) and far lower mismatch (DTW) than the two compared generation baselines, and it shows qualitative stitching of isolated signs on USTC-SLR500.","feed_headline":"Masked frames teach a diffusion model to join signs smoothly","feed_subtitle":"A conditional diffusion model recreates missing transition poses, bringing word-level signs closer to fluent continuous sentences.","key_machinery":"The load-bearing mechanism is a conditional diffusion model in latent space, paired with two training and inference strategies. A pre-trained encoder and decoder handle pose sequences, while a denoiser with self-attention and cross-attention layers removes noise conditionally on observed frames. The random masking strategy simulates the absence of transitions by erasing random frame blocks during training, turning the unsupervised stitching problem into supervised reconstruction. The linear interpolating padding strategy initializes the missing frames between boundary poses at inference, giving the diffusion process a stable starting point. Together these components let the model generate temporally coherent transition poses that preserve surrounding sign context.","core_discovery":"The central discovery is that transition poses between signs are learnable in a fully self-supervised way from continuous sign video alone, without any manually labeled transition data. By randomly masking a contiguous block of frames in a long video and conditioning the denoiser on the remaining observed frames, the model learns smooth, contextually appropriate bridging motion purely from reconstruction. During inference, the same model fills the gap between two discrete sign segments, using linear interpolation between the boundary frames as a warm start that the iterative denoising process then refines. The paper demonstrates this on PHOENIX14T and USTC-CSL100, where the generated transition frames match the semantics and smoothness of the original continuous sequences, and on USTC-SLR500, where it qualitatively joins isolated vocabulary signs into a continuous signing video.","pith_inferences":["A likely extension not explored in the paper is to make the masking ratio and mask location adaptive to pose velocity, so transitions are simulated where real coarticulation is hardest, which may improve generalization to variable-length transitions.","If the transition model is conditioned on gloss or text embeddings of the two signs being joined, it could produce transitions that are semantically aware of the words' meanings rather than purely pose-driven.","The BLEU and DTW numbers on sentence-level datasets are measured on masked-frame reconstruction; a stitched sequence of separately recorded vocabulary clips would be a sterner test because it introduces signer and recording variations, so the real-world gains over baselines remain to be quantified."],"forward_implications":["If the framework works as claimed, word-level sign dictionaries become building blocks for sentence-level video synthesis without collecting new paired sentence data.","The masking-based supervision recipe could apply to other motion-continuity tasks, such as action stitching or gesture-to-gesture transitions, where only long continuous recordings are available.","The near-ground-truth semantic accuracy on USTC-CSL100 suggests the generated transitions preserve enough information for downstream sign-language translation after reconstruction via the NSLT evaluation pipeline.","The linear interpolation warm start reduces the difficulty of the diffusion refinement, so shorter denoising schedules may be sufficient for practical stitching."],"supporting_citations":[{"why":"Supplies the denoising diffusion probabilistic model that the conditional denoiser is built upon.","marker":"[14]"},{"why":"G2P-DDM is a discrete diffusion baseline for sign pose generation from gloss sequences and serves as a primary comparison.","marker":"[47]"},{"why":"VQ-GCDM is a multi-hypothesis diffusion baseline for sign pose generation and serves as a primary comparison.","marker":"[43]"},{"why":"Provides the PHOENIX14T dataset and the NSLT method used to convert generated poses back into text for semantic evaluation.","marker":"[2]"},{"why":"Provides the USTC-CSL100 sentence-level dataset used for supervised transition generation evaluation.","marker":"[18]"},{"why":"Provides the USTC-SLR500 isolated word-level dataset used to test stitching discrete signs into continuous sequences.","marker":"[17]"},{"why":"OpenPose extracts 2D skeleton keypoints from all raw sign videos as the initial pose representation.","marker":"[4]"},{"why":"Supplies the skeleton correction model that refines the 2D OpenPose keypoints into 3D coordinates used as model input.","marker":"[50]"}],"fun_headline_variants":["Diffusion model learns sign transitions from masked frames","Self-supervised bridging: smooth sign joins without labels","Smooth sign transitions via masked-frame diffusion training","From discrete signs to fluent signing with diffusion","Masked-to-continuous: diffusion fills sign gaps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that randomly masking frames from continuous signing videos produces training and evaluation conditions that faithfully mimic the real transition task of concatenating two isolated word-level signs.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion model learns sign transitions from masked frames","Self-supervised bridging: smooth sign joins without labels","Smooth sign transitions via masked-frame diffusion training","From discrete signs to fluent signing with diffusion","Masked-to-continuous: diffusion fills sign gaps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1359,"prompt_tokens":896,"completion_tokens":463,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":391}},"tokens_in":512,"tokens_out":463,"duration_ms":5556,"temperature":1.0,"reasoning_tokens":391,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:04:54.127367+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete check would be to assemble a test set of continuous sentences for which ground-truth videos exist, artificially split each sentence into two isolated segments at a cut point inside a transition, run Sign-D2C to reconnect them, and measure the mismatches (DTW, MPJPE, and human fluency ratings) between the generated bridging frames and the actual frames at that cut; if the error is no better than simple interpolation for transitions that cross signer or recording-session boundaries, the masking assumption would fail to transfer to realistic concatenation.","supporting_citations":[{"cited_title":"Denoising diffusion probabilistic models","cited_arxiv_id":null,"evidence_quote":"Supplies the denoising diffusion probabilistic model that the conditional denoiser is built upon."},{"cited_title":"G2p-ddm: Generating sign pose se- quence from gloss sequence with discrete diffusion model","cited_arxiv_id":null,"evidence_quote":"G2P-DDM is a discrete diffusion baseline for sign pose generation from gloss sequences and serves as a primary comparison."},{"cited_title":"Gloss-driven conditional diffusion mod- els for sign language production","cited_arxiv_id":null,"evidence_quote":"VQ-GCDM is a multi-hypothesis diffusion baseline for sign pose generation and serves as a primary comparison."},{"cited_title":"Neural sign language trans- lation","cited_arxiv_id":null,"evidence_quote":"Provides the PHOENIX14T dataset and the NSLT method used to convert generated poses back into text for semantic evaluation."},{"cited_title":"Video-based sign language recognition without temporal segmentation","cited_arxiv_id":null,"evidence_quote":"Provides the USTC-CSL100 sentence-level dataset used for supervised transition generation evaluation."},{"cited_title":"Attention-based 3d-cnns for large-vocabulary sign language recognition","cited_arxiv_id":null,"evidence_quote":"Provides the USTC-SLR500 isolated word-level dataset used to test stitching discrete signs into continuous sequences."},{"cited_title":"Realtime multi-person 2d pose estimation using part affinity fields","cited_arxiv_id":null,"evidence_quote":"OpenPose extracts 2D skeleton keypoints from all raw sign videos as the initial pose representation."},{"cited_title":"Neural sign language synthe- sis: Words are our glosses","cited_arxiv_id":null,"evidence_quote":"Supplies the skeleton correction model that refines the 2D OpenPose keypoints into 3D coordinates used as model input."}],"review_version":1}