{"id":"75fc0f2b-4033-40a5-9c50-05b8ac48c303","arxiv_id":"2601.00296","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"TimeColor supports variable multi-reference video colorization via temporal latent concatenation and masked spatiotemporal attention, improving fidelity and consistency on Sakuga-42M.","lead":"TimeColor is a sketch-based video colorization model that handles any number of heterogeneous references by treating them as extra latent frames concatenated in time. This could let animators and editors use character sheets, backgrounds, or multiple frames for more consistent results without changing model size.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No direct ablation isolates whether masked attention and modality-disjoint RoPE are required to prevent leakage or if gains come from extra reference frames alone.","rationale":"The reader's weakest assumption correctly flags the unverified reliability of the binding mechanisms. The proposed ablation directly tests whether those mechanisms are necessary for the claimed improvements, moving the verdict from UNVERDICTED to CONDITIONAL pending the result.","tokens_in":1661,"tokens_out":331,"duration_ms":19276,"concrete_test":"Re-train the model on the multi-reference Sakuga-42M protocol with two variants: (1) replace correspondence-masked attention with vanilla spatiotemporal attention and (2) replace modality-disjoint RoPE with standard joint RoPE; keep all other hyperparameters and reference count identical. If either variant closes more than 70 % of the reported gap to the full TimeColor model on the identity-consistency and palette-leakage metrics, the mechanisms are not the primary driver.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline claim rests on the assertion that temporal concatenation plus spatiotemporal correspondence-masked attention and modality-disjoint RoPE together eliminate shortcutting and cross-identity palette leakage. The reported gains in color fidelity, identity consistency, and temporal stability on Sakuga-42M are aggregate scores; they do not separate the contribution of the binding mechanisms from the simple effect of supplying more reference pixels. If standard (unmasked) attention with joint RoPE already yields comparable numbers when the same extra frames are concatenated, the specific engineering choices are not load-bearing and the central explanation for the improvement is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes TimeColor, a sketch-based video colorization diffusion model that supports an arbitrary number of heterogeneous references (e.g., character sheets, background images, or additional colorized frames) by encoding them as extra latent frames that are temporally concatenated with the target sequence. This keeps the parameter count fixed while allowing concurrent processing. The model adds spatiotemporal correspondence-masked attention to enforce subject-reference binding and modality-disjoint RoPE indexing to reduce shortcutting and cross-identity palette leakage. Experiments on Sakuga-42M under single- and multi-reference protocols report gains in color fidelity, identity consistency, and temporal stability relative to prior baselines.","tokens_in":1782,"tokens_out":547,"duration_ms":23856,"significance":"If the reported gains hold under rigorous controls and the binding mechanisms prove necessary rather than incidental, the work would meaningfully extend reference-based colorization to practical multi-reference scenarios common in animation pipelines. The fixed-parameter design via temporal concatenation is a clean architectural choice that could generalize beyond colorization.","major_comments":[{"comment":"The central claim that spatiotemporal correspondence-masked attention and modality-disjoint RoPE are required to mitigate shortcutting and cross-identity leakage rests on aggregate improvements over baselines. No ablation is described that compares the full model against a control using standard (unmasked) attention and joint RoPE while still supplying the same extra reference frames; without this isolation, it remains possible that gains derive primarily from additional reference pixels rather than the proposed binding mechanisms.","section":"Experiments / §4"},{"comment":"Quantitative results are summarized at a high level (improvements in fidelity, consistency, and stability) without reported metrics, error bars, baseline details, or per-scene breakdowns. This makes it impossible to assess effect sizes or whether the method remains stable across diverse real-world reference sets without per-scene tuning.","section":"Experiments / §4"}],"minor_comments":[{"comment":"The abstract states empirical improvements but supplies no numerical values, making it difficult for readers to gauge the magnitude of gains before reading the full results section.","section":"Abstract"},{"comment":"Notation for the masked attention and RoPE variants should be introduced with explicit equations or pseudocode in the method section to clarify how masking is applied across the concatenated temporal dimension.","section":"Method"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's evaluation relies on a single dataset (Sakuga-42M) with limited detail on reference diversity; this may limit generalizability claims. Citation pattern appears appropriate for the subfield."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive comments on our manuscript. We address each major point below and will revise the paper to strengthen the experimental validation as suggested.","responses":[{"response":"We agree that the current manuscript does not include a controlled ablation isolating the spatiotemporal correspondence-masked attention and modality-disjoint RoPE from the mere provision of additional reference frames. Our existing comparisons are against prior single-reference baselines, which leaves open the possibility that gains stem primarily from extra conditioning pixels. In the revised version we will add a dedicated ablation: the full TimeColor model versus an otherwise identical variant that uses standard (unmasked) spatiotemporal attention and joint RoPE while receiving the same set of extra reference frames. This will directly test whether the proposed binding mechanisms are necessary to reduce shortcutting and cross-identity palette leakage.","revision_made":"yes","referee_comment":"[Experiments / §4] The central claim that spatiotemporal correspondence-masked attention and modality-disjoint RoPE are required to mitigate shortcutting and cross-identity leakage rests on aggregate improvements over baselines. No ablation is described that compares the full model against a control using standard (unmasked) attention and joint RoPE while still supplying the same extra reference frames; without this isolation, it remains possible that gains derive primarily from additional reference pixels rather than the proposed binding mechanisms."},{"response":"We acknowledge that the experimental section currently presents results at a summary level. The revised manuscript will expand §4 to report the concrete metrics employed (PSNR, SSIM, LPIPS for fidelity; identity-consistency scores and temporal-warping error for stability), include error bars computed over multiple random seeds, provide full baseline implementation details, and add per-scene performance tables. These additions will enable readers to evaluate effect sizes and assess robustness across heterogeneous reference sets without per-scene hyper-parameter tuning.","revision_made":"yes","referee_comment":"[Experiments / §4] Quantitative results are summarized at a high level (improvements in fidelity, consistency, and stability) without reported metrics, error bars, baseline details, or per-scene breakdowns. This makes it impossible to assess effect sizes or whether the method remains stable across diverse real-world reference sets without per-scene tuning."}],"tokens_in":1341,"tokens_out":480,"duration_ms":42436,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"TimeColor concatenates variable references as extra latent frames in the diffusion process and adds spatiotemporal correspondence-masked attention plus modality-disjoint RoPE to keep subjects bound and stop color leakage across identities. This lets the model take character sheets, background plates, or any mix of references without changing parameter count or retraining for each new count. The approach is a direct response to the single-reference limit in prior sketch-based video colorization work, and the architecture description is clear enough that someone could reimplement the core pieces from the text. On Sakuga-42M it reports better fidelity, identity consistency, and temporal stability in both single- and multi-reference settings, which matches what you would expect from giving the model more conditioning pixels when the binding works. The practical upside is real for animation pipelines that already produce multiple reference images. The main gap is the missing evidence. The abstract supplies no metrics, no listed baselines, no error bars, and no ablation that isolates the masked attention or the special RoPE from the simple effect of feeding more frames. If standard attention on the concatenated sequence already captures most of the gain, then the central claim that these mechanisms prevent shortcutting and leakage rests on an untested assumption. That is the exact point the stress-test note raises, and nothing in the provided description contradicts it. The paper is aimed at people who build or tune reference-conditioned video models. A reader working on diffusion video pipelines would pick up the concatenation trick and the masking pattern quickly. I would send it to peer review because the problem is concrete, the fix is lightweight, and the missing controls are straightforward to add. Referees can ask for the ablations and the actual scores without the work being fundamentally broken.","headline":"TimeColor adds temporal concatenation and masked attention for multi-reference video colorization, but without numbers or ablations it's unclear if the binding tricks are necessary or if extra frames alone would suffice.","tokens_in":2265,"tokens_out":425,"would_cite":false,"duration_ms":51700,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/RealityFromDistinction.lean","rs_theorem":"reality_from_one_distinction","paper_passage":"TimeColor encodes references as additional latent frames which are concatenated temporally... uses spatiotemporal correspondence-masked attention to enforce subject–reference binding in addition to modality-disjoint RoPE indexing. These mechanisms mitigate shortcutting and cross-identity palette leakage."},{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"We propose TimeColor, a DiT-based framework for sketch video colorization supporting variable-count, heterogeneous multi-reference conditioning with explicit region-level control."}],"headline":"TimeColor's temporal concatenation + masked attention for reference binding is orthogonal to RS forcing chain","alignment":"orthogonal","rationale":"The paper's core machinery (DiT temporal token concat, modality-disjoint RoPE, correspondence-masked 3D attention to enforce subject-reference binding and prevent palette leakage) operates entirely within generative CV / diffusion modeling. It neither invokes nor parallels any RS element: no J-cost functional, no φ-ladder ratios, no 8-tick periodicity, no parameter-free constant derivation, and no recognition-cost reasoning. The domain (sketch video colorization on Sakuga-42M) lies outside the RS canon; the framework has no theorems about attention masking or reference conditioning.","tokens_in":52499,"confidence":"high","tokens_out":346,"duration_ms":15165,"cache_read_input_tokens":32896,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"TimeColor treats any number of reference images as extra frames concatenated in time to colorize video sketches with fixed model size.","keywords":["video colorization","multi-reference colorization","sketch-based generation","diffusion models","temporal concatenation","masked attention","rotary position embedding","anime video"],"falsifier":"A multi-reference test sequence in which adding a second reference produces visible color mixing between characters or sudden palette shifts between frames.","tokens_in":2567,"feed_emoji":"🎨","tokens_out":631,"duration_ms":42188,"temperature":0.7,"pith_summary":"The paper presents TimeColor as a diffusion-based method for sketch video colorization that accepts heterogeneous references of varying count, such as character sheets or background images, rather than restricting to a single first-frame reference. It achieves this by encoding each reference as an additional latent frame and concatenating them temporally with the target sequence so the model processes them together in every diffusion step. Spatiotemporal correspondence-masked attention and modality-disjoint RoPE indexing enforce correct binding and block palette leakage between identities. Experiments on the Sakuga-42M dataset under single- and multi-reference protocols report gains in color accuracy, character consistency, and frame-to-frame stability.","feed_headline":"Video colorization accepts any number of references via temporal frame concat","feed_subtitle":"References become extra latent frames processed together, raising color accuracy and identity consistency on Sakuga-42M.","key_machinery":"Temporal concatenation of reference latents processed with correspondence-masked spatiotemporal attention and modality-disjoint RoPE indexing.","core_discovery":"TimeColor encodes references as additional latent frames which are concatenated temporally, permitting them to be processed concurrently in each diffusion step while keeping the model's parameter count fixed. It uses spatiotemporal correspondence-masked attention to enforce subject-reference binding in addition to modality-disjoint RoPE indexing. These mechanisms mitigate shortcutting and cross-identity palette leakage.","pith_inferences":["The approach could be tested on live-action video to see whether the same mechanisms reduce reference leakage in non-animated footage.","Production pipelines might use it to feed multiple artist-provided references in a single pass instead of sequential single-reference steps.","Extending the temporal concatenation idea to other conditioning signals such as depth maps or motion sketches would be a direct next experiment."],"forward_implications":["The same fixed-parameter model handles one reference or many without retraining or architecture changes.","Color fidelity, identity consistency, and temporal stability all increase under both single- and multi-reference evaluation protocols.","References can be arbitrary colorized frames, character sheets, or background images rather than only the first frame.","Explicit per-reference region assignment becomes possible without altering the diffusion backbone."],"fun_headline_variants":["TimeColor concatenates references temporally without extra params","Flexible multi-reference video colorization via temporal frame concat","Temporal concat of latent frames supports heterogeneous color references","Correspondence-masked attention prevents palette leakage in TimeColor","TimeColor improves fidelity with variable count reference encoding"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The masking and indexing will reliably tie each reference to its intended subject across real-world inputs without creating new artifacts or needing per-scene adjustments.","fun_headline_variants_meta":{"raw":{"variants":["TimeColor concatenates references temporally without extra params","Flexible multi-reference video colorization via temporal frame concat","Temporal concat of latent frames supports heterogeneous color references","Correspondence-masked attention prevents palette leakage in TimeColor","TimeColor improves fidelity with variable count reference encoding"]},"model":"grok-4.3","cost_usd":0.00435,"raw_usage":{"total_tokens":2147,"prompt_tokens":599,"num_sources_used":0,"completion_tokens":71,"cost_in_usd_ticks":43499500,"prompt_tokens_details":{"text_tokens":599,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1477,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":599,"tokens_out":71,"duration_ms":23128,"temperature":1.0,"reasoning_tokens":1477,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-16T18:00:57.170757+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A multi-reference test sequence in which adding a second reference produces visible color mixing between characters or sudden palette shifts between frames.","supporting_citations":[],"review_version":1}