{"id":"47a189ab-76ce-4c1f-bd50-40d33012d52f","arxiv_id":"2412.00156","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A latent-diffusion solver with pseudo-batch sampling and DDIM-inversion initialization reconstructs high-definition video from spatio-temporal degradations on a single GPU.","lead":"VISION-XL uses a pretrained image diffusion model (SDXL) in latent space to restore high-definition videos from blur, low resolution, missing pixels, and combined spatio-temporal degradations. It introduces pseudo-batch sampling and measurement-based initialization, reporting reconstructions above 1280x720 in about six seconds per frame on one GPU.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3 itself undercuts the 'state-of-the-art across a wide range' claim: on Inpaint+ and Inpaint, SVI beats VISION-XL on FVD, PSNR, and SSIM, so the central claim needs narrowing before acceptance.","rationale":"I examined the low-pass filter assumption highlighted by the reader. While the filter is underspecified and tuned by ablation, it is an explicit, testable component with a demonstrated effect, so it is not the most load-bearing problem for the central claim. The more direct problem is that the paper's own quantitative table contradicts the headline claim: on the Inpaint+ and Inpaint task families, SVI is better or comparable on multiple metrics, including FVD, which the paper emphasizes for temporal consistency. The central claim would require VISION-XL to dominate across all listed degradation families, and Table 3 shows it does not. This is an internal, checkable inconsistency rather than a disagreement with field consensus. I would not reject the paper: the pseudo-batch framework, efficiency results, and ablations are valuable, and the overclaim is correctable. However, conditional acceptance should explicitly require claim revision and code release so the stated SOTA can be independently verified.","tokens_in":13888,"tokens_out":5211,"duration_ms":46918,"concrete_test":"Release code and recompute the full Table 3 with the stated protocol (same videos, same FVD/LPIPS code), especially Inpaint+ DAVIS and Inpaint DAVIS/vertical rows. If SVI remains better on any task family, narrow the abstract/conclusion claim to 'best on temporal deblurring/SR and competitive on inpainting,' and report error bars or significance across dataset folds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires VISION-XL to be state-of-the-art across the listed spatio-temporal inverse problems. Table 3 contains direct counterexamples. On Inpaint+ (DAVIS), SVI scores FVD 208.6 vs Ours 241.1, LPIPS 0.238 vs 0.242, PSNR 29.60 vs 28.81, SSIM 0.848 vs 0.815; SVI also wins FVD/PSNR on vertical Inpaint+ and on DAVIS/vertical Inpaint (e.g., Inpaint DAVIS PSNR 30.20 vs 29.74). Thus the broad SOTA claim is false as written, independent of any assumptions about the low-pass filter. The supporting comparison is also narrow: DiffIR2VR is only run on SR, Warped Diffusion is excluded, and code is not released, so 'state-of-the-art' rests on two applicable baselines, one of which wins on a full task family.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents VISION-XL, a plug-and-play solver for video inverse problems built on SDXL. The algorithm initializes latents by DDIM-inverting the encoded measurement, denoises frame-wise via Tweedie, enforces data consistency with l-step CG in pixel space, applies a scheduled low-pass filter before re-encoding, and re-noises to continue the diffusion loop. Experiments cover six degradations (Deblur, SR, Inpaint, and their '+' frame-averaging variants) on DAVIS and Pexels at three aspect ratios, with ablations for initialization timestep, CG steps, and low-pass filter strength. The paper claims state-of-the-art reconstruction, multi-aspect support, and under 6 seconds per frame on one NVIDIA 4090 GPU.","tokens_in":14154,"tokens_out":5719,"duration_ms":47815,"significance":"If the results held, VISION-XL would be a meaningful step: it is the first latent-diffusion video inverse solver in the comparison set, it handles memory-intensive SDXL via pseudo-batch sampling, and pseudo-batch inversion provides large FVD gains over random or batch-synchronized noise initialization. The ablations are informative and the efficiency numbers in Table 4 are concrete and useful. However, the central state-of-the-art claim is not supported by the paper's own Table 3, because SVI beats VISION-XL on every metric for DAVIS Inpaint+ and DAVIS Inpaint. The contribution remains worthwhile, but the claims need substantial narrowing before publication.","major_comments":[{"comment":"The abstract and Section 4.2 claim 'state-of-the-art' reconstruction and state that the method 'consistently outperforms baseline approaches' across spatio-temporal inverse problems, but Table 3 directly contradicts this. On DAVIS Inpaint+, SVI achieves FVD 208.6 vs Ours 241.1, LPIPS 0.238 vs 0.242, PSNR 29.60 vs 28.81, and SSIM 0.848 vs 0.815; on DAVIS Inpaint, SVI wins all four metrics as well. The claim must be narrowed to the tasks and datasets where the method actually wins (e.g., Deblur+ and SR+), and the sentence about a 'significant reduction in FVD compared to the runner-up across all datasets' should be removed or qualified.","section":"Section 4.2, Table 3"},{"comment":"The scheduled low-pass filter is a load-bearing component: Table 7 shows that removing it increases FVD by roughly 30 points and lowers PSNR by about 1 dB. Yet h_sigma_t is never defined (kernel shape, support, or boundary treatment are all missing), and no derivation is given for why sigma_t = lambda * sqrt(1 - alpha_bar_t) keeps re-encoded latents on the clean manifold. The heuristic is validated only on SR+ Pexels landscape and then transferred to every other task and aspect ratio. Please specify the filter explicitly and provide either a derivation or a multi-task validation; as written, the method is not fully reproducible at this step.","section":"Section 3, Step 4, Eq. (7)"},{"comment":"The hyperparameters tau = 0.3T, lambda = 2, and l = 10 are selected by ablations on a single task (SR+, Pexels landscape) and then applied to every other degradation, aspect ratio, and dataset, with no error bars, confidence intervals, or repeated-run statistics. Given the sensitivity shown in Table 6 (l = 5 vs l = 10 changes FVD by about 56 points) and Table 7 (lambda = 1 vs lambda = 2 changes LPIPS and PSNR), the broad-applicability claim needs either additional evidence across tasks or a more conservative statement of the method's tuning requirements.","section":"Section 4.3, Tables 5-7"},{"comment":"The state-of-the-art claim is also weakened by the comparison set: Warped Diffusion is not evaluated, and DiffIR2VR is run only on SR, so the comparison against prior video inverse problem solvers rests largely on SVI, which wins on the inpainting tasks in Table 3. Please either add the missing baseline or explicitly state that the comparison covers only the methods that could be run; the current wording implies a broader comparison than the experiments support.","section":"Section 4.1, Baselines"}],"minor_comments":[{"comment":"The loop header 'for t = tau : 2 do' is ambiguous; please write it as a descending loop, e.g., 'for t = tau downto 2'.","section":"Algorithm 1"},{"comment":"The convolution in Eq. (7) does not specify boundary conditions or kernel support; adding these details would improve reproducibility.","section":"Section 3, Step 4"},{"comment":"The blind video deblurring extension is presented only qualitatively; a quantitative evaluation of the two-round PSF refinement would make the extension more convincing.","section":"Section 7, Algorithm 2"},{"comment":"No code release is mentioned; given the number of unspecified details, releasing the inference code would substantially help reproducibility.","section":"General"},{"comment":"The statement that DiffIR2VR artifacts are 'likely due to inaccuracies in optical flow estimation' is speculative; either provide supporting evidence or soften the wording.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a coherent algorithm and useful ablations, but the headline claim is currently refuted by the authors' own Table 3. The revision should focus on narrowing the claims, specifying the low-pass filter, and adding robustness or variance information. I do not see a fundamental flaw in the method itself, so I recommend major rather than minor revision; the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a serious look for the pseudo-batch tricks. The idea of splitting a video batch into individual frames and denoising them in parallel through SDXL, while doing data consistency in pixel space with CG, is a clean solution to the memory bottleneck that has kept latent diffusion out of video inverse problems. The pseudo-batch inversion initialization—DDIM-inverting the encoded measurement to get a temporally informative starting latent—is also a nice trick, and the ablation showing a 3x FVD drop is convincing. The efficiency numbers are real: 2.5 minutes for a 25-frame video at 768x1280 on a single 4090, with 12.7 GB memory, versus SVI's 15 minutes and 18.5 GB. That alone is a contribution.\n\nThe paper is less solid when it claims state-of-the-art across a wide range. Table 3 is the direct evidence: on Inpaint+ for DAVIS, SVI beats VISION-XL on all four metrics. For spatial Inpaint on DAVIS and vertical, SVI also wins on FVD and PSNR. The stress-test's specific example on vertical Inpaint+ is mistaken—VISION-XL wins there—but it does not matter; the DAVIS Inpaint+ result alone contradicts the SOTA sentence in the abstract and conclusion. The real story is: VISION-XL dominates on the temporal degradations (Deblur+, SR+) and on SR, but inpainting is a mixed bag, and the paper's own qualitative text admits it is only 'competitive' with SVI there. That admission should be reflected in the abstract.\n\nOther soft spots: the low-pass filter in Step 4 is load-bearing—without it, VAE drift ruins the latent manifold—but it is a heuristic with a single ablation (lambda on SR+ on one dataset) and no kernel specification. That is a reproducibility gap. No code release, no error bars, and hyperparameters (tau, lambda, l) are tuned on one task and applied everywhere. DiffIR2VR is only run on SR, and Warped Diffusion is excluded because it is not open-source, so the baseline comparison is narrower than the 'SOTA' claim needs.\n\nThe mathematical framing is sound: it is an empirical algorithm with clear steps, and the ablations support the components. No circularity. The paper is honest about the inpainting limitation in the qualitative section, which is why I would give this a conditional rather than a reject.\n\nFor peer review: yes, send it. The pseudo-batch ideas are useful and the efficiency result is solid. The authors need to narrow the SOTA claim, release code, and specify the low-pass filter kernel. A good referee can turn this into a solid paper.\n\nI would bring this to reading group and would cite the pseudo-batch sampling if I worked in this subfield.","headline":"Useful practical advance in video inverse problems with latent diffusion, but the paper's own Table 3 undercuts the broad 'state-of-the-art' claim.","tokens_in":14626,"tokens_out":3453,"would_cite":true,"duration_ms":25512,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Latent diffusion models can solve high-definition video inverse problems without fine-tuning, by pairing a pseudo-batch sampling loop with measurement-derived initialization and a low-pass filter that keeps the latents on the clean…","keywords":["video inverse problems","latent diffusion models","SDXL","pseudo-batch sampling","pseudo-batch inversion","temporal consistency","low-pass filter","conjugate gradient"],"falsifier":"Run the method with and without the low-pass filter on a degradation whose spectrum is not low-frequency-dominated, such as high-frequency-patterned blur or a non-smooth mask, and compare FVD and PSNR. If removing the filter does not degrade the output on such a task, or if the optimal filter width must be re-tuned per task, the central mechanism is not general as claimed.","tokens_in":13693,"feed_emoji":"🎬","tokens_out":5315,"duration_ms":43620,"temperature":0.7,"pith_summary":"The paper is trying to establish that latent image diffusion models can solve high-definition video inverse problems—situations where the only measured video is a temporal average of clean frames that are also blurred, downsampled, or partially masked. The proposed framework, VISION-XL, adds two mechanisms to the SDXL latent sampler: pseudo-batch sampling, which denoises one frame at a time to fit on a single GPU, and pseudo-batch inversion, which initializes the latent path from the measurement rather than from noise. It also applies a scheduled low-pass filter before re-encoding the data-consistent batch into latent space, preventing the VAE's encoder-decoder round trip from kicking the reconstruction off the clean image manifold. The paper reports state-of-the-art perceptual quality (FVD) across multiple aspect ratios and several inverse problems, at resolutions above 1280×720 in under 6 seconds per frame on a single NVIDIA 4090 GPU. If true, this means progress in latent image generation transfers directly to video restoration without task-specific training.","feed_headline":"Latent diffusion solves HD video degradation in 6 s/frame","feed_subtitle":"Pseudo-batch inversion and a scheduled low-pass filter turn SDXL into a training-free video inverse solver.","key_machinery":"The load-bearing mechanism is the alternating loop of latent denoising and pixel-space data consistency, glued by a scheduled low-pass filter. Each iteration: Tweedie-denoise each latent frame separately (pseudo-batch), decode, run an $l$-step conjugate-gradient refinement against the measurement $Y$, apply a low-pass filter $h_{\\sigma_t}$ with width $\\sigma_t = \\lambda\\sqrt{1-\\bar\\alpha_t}$, re-encode, and re-noise with batch-consistent noise. The filter width scales with the noise level and vanishes as $t\\to 0$, so early iterations suppress the high-frequency artifacts that VAE re-encoding introduces, while final iterations preserve detail. Pseudo-batch inversion sets the starting latent $z_\\tau$ by encoding the measurement and running DDIM inversion from $z_0$.","core_discovery":"The central claim is that the failures of latent diffusion models in video inverse problems are not inherent to the latent space but are fixable by two changes to the sampling loop. First, instead of generating a batch of frames from synchronized noise, the latents are initialized by encoding the measured frames and DDIM-inverting them to a mid-noise timestep. Second, after enforcing data consistency on the decoded batch with conjugate-gradient updates, the batch is low-pass filtered with a cutoff that shrinks with the noise scale, then re-encoded. Together these keep the latents on the clean manifold and yield temporally consistent reconstructions, turning SDXL into a state-of-the-art video inverse solver. The method is train-free, works across degradation types, and supports landscape, vertical, and square aspect ratios.","pith_inferences":["The success of the low-pass filter suggests a general recipe for latent inverse solvers: any operation that pushes the latent representation off the training manifold—such as aggressive data-consistency updates, editing, or classifier guidance—might benefit from a scale-matched frequency truncation before re-encoding, extending beyond video to high-resolution single-image restoration.","The method relies on a single tuned width parameter $\\lambda=2$; whether that cutoff can be derived from the noise schedule or adapted per degradation is an open question, and if it requires per-task tuning the robustness across unseen degradations is uncertain.","Since the initialization is only DDIM inversion of the measurement frames, a natural test is whether a stronger inversion (for example, with guidance or multiple inversion steps) further improves temporal consistency, with the supplementary material already sketching a two-round blind-deblurring extension.","The reported speed implies that diffusion-based restoration could move from offline batch processing into interactive tools, but this depends on the low-pass filter and inversion remaining stable at higher frame counts than the tested 25 frames."],"forward_implications":["Any latent image diffusion model can be turned into a video inverse solver without fine-tuning, so advances in text-to-image models transfer directly to video restoration.","Computational cost scales with a single frame rather than a large batch, making HD video reconstruction feasible on a single consumer GPU.","Because the solver avoids optical flow, it does not suffer from flow-estimation failures under severe degradation and can handle spatio-temporal degradations that optical-flow-based methods cannot address.","Frame averaging can be combined with deblurring, super-resolution, or inpainting in one forward model and solved in the same alternating loop.","The approach inherently supports multiple aspect ratios, including landscape, vertical, and square formats, without reconfiguration.","The reported efficiency (under 6 s/frame at 1280×768 on a 4090) suggests the method is practical for interactive or near-real-time use in video editing pipelines."],"supporting_citations":[{"why":"Baseline SVI whose batch-consistent sampling and pixel-space conjugate-gradient protocol the paper extends to the latent domain.","marker":"[13]"},{"why":"SDXL, the latent diffusion model whose strong prior and high-resolution synthesis capability the method builds on.","marker":"[17]"},{"why":"Source of the multi-step conjugate-gradient data-consistency update used in Step 3.","marker":"[4]"},{"why":"Cited as the basis for the informative latent initialization via inversion of the measurement.","marker":"[30]"},{"why":"Spectral-diffusion frequency analysis that motivates the scheduled low-pass filter in Step 4.","marker":"[32]"},{"why":"DDIM inversion, used to obtain the informative starting latents in Step 1, and renoising in Step 5.","marker":"[21]"},{"why":"Tweedie's formula, used in Step 2 for one-step denoised latents from the noise prediction.","marker":"[7]"}],"fun_headline_variants":["Video inverse solver via SDXL latents: 6s per frame, train-free","Pseudo-batch sampling turns SDXL into a fast video inverse solver","HD video reconstruction from degraded frames in under 6s/frame","Train-free SDXL latent tricks achieve HD video reconstruction in 6s","Training-free method turns SDXL into HD video inverse solver"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole performance gain depends on the heuristic scheduled low-pass filter: if the chosen filter width $\\sigma_t = 2\\sqrt{1-\\bar\\alpha_t}$ fails to generalize to a new degradation, aspect ratio, or model version without re-tuning, the claimed advantage over pixel-space methods disappears.","fun_headline_variants_meta":{"raw":{"variants":["Video inverse solver via SDXL latents: 6s per frame, train-free","Pseudo-batch sampling turns SDXL into a fast video inverse solver","HD video reconstruction from degraded frames in under 6s/frame","Train-free SDXL latent tricks achieve HD video reconstruction in 6s","Training-free method turns SDXL into HD video inverse solver"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00094,"raw_usage":{"total_tokens":3993,"prompt_tokens":893,"completion_tokens":3100,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":3005}},"tokens_in":509,"tokens_out":3100,"duration_ms":19163,"temperature":1.0,"reasoning_tokens":3005,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:06:36.595069+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the method with and without the low-pass filter on a degradation whose spectrum is not low-frequency-dominated, such as high-frequency-patterned blur or a non-smooth mask, and compare FVD and PSNR. If removing the filter does not degrade the output on such a task, or if the optimal filter width must be re-tuned per task, the central mechanism is not general as claimed.","supporting_citations":[{"cited_title":"Solving video inverse problems using image diffusion models","cited_arxiv_id":null,"evidence_quote":"Baseline SVI whose batch-consistent sampling and pixel-space conjugate-gradient protocol the paper extends to the latent domain."},{"cited_title":"De- composed diffusion sampler for accelerating large-scale in- verse problems","cited_arxiv_id":null,"evidence_quote":"Source of the multi-step conjugate-gradient data-consistency update used in Step 3."},{"cited_title":"Dreamclean: Restoring clean image using deep diffusion prior","cited_arxiv_id":null,"evidence_quote":"Cited as the basis for the informative latent initialization via inversion of the measurement."},{"cited_title":"Diffusion probabilistic model made slim","cited_arxiv_id":null,"evidence_quote":"Spectral-diffusion frequency analysis that motivates the scheduled low-pass filter in Step 4."},{"cited_title":"Denois- ing diffusion implicit models","cited_arxiv_id":null,"evidence_quote":"DDIM inversion, used to obtain the informative starting latents in Step 1, and renoising in Step 5."},{"cited_title":"Tweedie’s formula and selection bias","cited_arxiv_id":null,"evidence_quote":"Tweedie's formula, used in Step 2 for one-step denoised latents from the noise prediction."}],"review_version":1}