Pith. sign in

REVIEW 4 major objections 8 minor 1 cited by

Self-supervised ControlNet with Spatio-Temporal Mamba for Real-world Video Super-resolution

T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a self-supervised ControlNet paired with a spatial-temporal continuous Mamba scan lets latent diffusion models super-resolve real-world video with fewer artifacts and more stable frames.

desk verdict A solid engineering recipe for diffusion video super-resolution, but the real-world evaluation is weaker than the abstract claims. read the letter →

arxiv 2506.01037 v1 pith:A2JBZMZS submitted 2025-06-01 cs.CV

Shijun Shi , Jing Xu , Lijing Lu , Zhihang Li , Kai Hu This is my paper · ORCID
classification cs.CV
keywords videosuper-resolutionlatentdiffusionmodelMambastatespaceself-supervisedcontrastivelearningControlNettemporalconsistencyreal-worlddegradation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the main obstacle to diffusion-based real-world video super-resolution is not resolution itself but two coupled failures: generative randomness invents details that flicker across frames, and unknown degradations corrupt the conditioning signal. It proposes SCST, a system built on a pretrained latent diffusion model, that attacks both problems at once. A spatial-temporal continuous Mamba block gives the denoiser a global view of the whole clip at linear cost, while a self-supervised ControlNet uses patch-level contrastive learning to pull degradation-insensitive features from low-resolution frames, using high-resolution targets as keys. The authors report that this combination beats prior real-world VSR methods on perceptual quality and temporal-consistency metrics across synthetic benchmarks and on the real-world VideoLQ benchmark. If the finding holds, video super-resolution can move from sharp-but-unstable outputs to outputs that are both detailed and temporally coherent, which matters for display, surveillance, and archival restoration.

What carries the argument

The load-bearing object is the Spatial-Temporal Continuous Scan inside the 3D-Mamba Block, a state-space model that flattens each video into six continuous scanning trajectories (three patterns, each with a flipped counterpart). Unlike a 3D sweep scan that resets the hidden state at frame boundaries, the continuous scan keeps an uninterrupted path over intra-frame and inter-frame positions, so global 3D context accumulates across the whole clip at linear cost. The second mechanism is MoCoCtrl, a momentum-contrast ControlNet: a query encoder processes LR frames, a momentum-updated key encoder processes HR frames, and a patch-level contrastive loss aligns each LR patch with its HR counterpart while pushing against a memory queue of negatives. These feed into a frozen pretrained latent diffusion U-Net, and the three-stage HR/LR curriculum stabilizes training by learning reconstruction before super-resolution before temporal modeling.

What would settle it

Train two SCST variants on the same footage, one with the paper's synthetic degradation pipeline and one with real camera-captured LR-HR pairs, then evaluate both on VideoLQ and on paired real data; the claimed real-world advantage fails if the simulator-trained model does not stay ahead on no-reference quality and temporal-smoothness metrics.

Watch

Extended reading notes

Core claim

The central claim is that a latent-diffusion video super-resolver can be made noise-robust and temporally coherent by adding two components on top of a frozen pretrained U-Net. The first is the Spatial-Temporal Continuous Mamba (STCM), a 3D state-space block whose six scanning paths run continuously through space and across frames rather than resetting between frames, giving each video patch global contextual information with linear computational complexity. The second is MoCoCtrl, a ControlNet-style encoder trained with a patch-level momentum-contrast loss in which low-resolution patches are the queries and high-resolution patches are positive keys, so the extracted features are insensitive to degradation. A three-stage training schedule mixes HR and LR videos, first teaching reconstruction, then contrastive alignment, then temporal modeling with LR only. On REDS4, UDM10, SPMCS, and YouHQ40, the paper reports the best LPIPS and DISTS among compared methods, and on VideoLQ it reports the best CLIP-IQA, MUSIQ, NIQE, and DOVER. That is the evidence the authors use to conclude that both proposed modules and the training strategy are effective.

Load-bearing premise

The whole training and most evaluation rely on a synthetic degradation simulator that stands in for real-world video degradation; if that simulator is not faithful, the reported quality gains may weaken on genuine footage.

Editorial extensions

If this is right

  • Because STCM models global 3D context without full cubic attention, diffusion-based VSR can handle longer clips at practical computational cost.
  • The contrastive ControlNet makes the conditioning features degradation-insensitive, so artifacts caused by unknown real-world degradation are reduced during sampling.
  • The three-stage curriculum lets reconstruction, contrastive alignment, and temporal modeling be learned separately, stabilizing diffusion VSR training.
  • On the reported benchmarks, perceptual quality improves while temporal smoothness also improves, as measured by warp error.
  • The design works with a frozen pretrained diffusion U-Net, preserving the generative prior while adding video-specific structure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper's real-world claim depends on the synthetic degradation simulator used in training and in four of the five test sets; if that simulator is not faithful, the synthetic-benchmark gains may not transfer to genuine footage, and the VideoLQ results alone would not settle the question.
  • Editorial inference: the continuous-scan idea could transfer to other video restoration and generation tasks where temporal coherence matters, such as deblurring, frame interpolation, or video editing, without retraining the full 2D diffusion backbone.
  • Editorial inference: a natural next ablation would isolate whether the contrastive benefit comes mainly from HR positives, LR negatives, or HR negatives in the memory queue, since the paper does not separate those contributions.
  • Editorial inference: evaluating the same model on videos with device-specific real degradations, using paired camera-captured data, would be a direct way to test whether the reported VideoLQ improvement generalizes beyond the simulator.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes SCST, a diffusion-based video super-resolution method that combines a Spatial-Temporal Continuous Mamba (STCM) module inserted into a pretrained Stable Diffusion U-Net, a self-supervised MoCo-based ControlNet (MoCoCtrl) that aligns LR and HR patch features via contrastive learning, and a three-stage HR/LR hybrid training strategy. The authors report state-of-the-art LPIPS and DISTS on four synthetic test sets, best no-reference metrics on the real-world VideoLQ set, and ablation studies on YouHQ showing that both proposed modules improve quality and that STCM improves warping error over local attention and a 3D sweep-scan Mamba baseline.

Significance. If fully substantiated, SCST would be a useful contribution to real-world VSR: the 3D continuous scan is a plausible alternative to expensive full 3D attention, and the patch-level momentum-contrastive ControlNet is an interesting adaptation of MoCo to restoration. The paper also provides informative ablations, including a comparison of temporal-modeling strategies and temporal-profile visualizations. However, the evidence as presented does not yet support the central real-world superiority claim: the full-reference benchmarks are in-distribution with the training degradation pipeline, the only real-world benchmark is assessed with no-reference metrics that cannot verify fidelity, and temporal consistency against state-of-the-art methods is never quantified. The paper does not release code or trained models, and all quantitative results are single-run point estimates.

major comments (4)
  1. [§4.1, Table 1] All four synthetic test sets (REDS4, UDM10, SPMCS, YouHQ40) are generated with the same RealBasicVSR degradation pipeline used to create training pairs, so the LPIPS/DISTS improvements in Table 1 are in-distribution with respect to the degradation model and do not by themselves demonstrate generalization to unseen real-world degradation. The only real-world evidence is VideoLQ, which is scored exclusively with no-reference metrics (CLIP-IQA, MUSIQ, NIQE, DOVER). These metrics can reward sharper, more natural-looking output even when generated details are not faithful to the source content, and they do not measure fidelity. Because the abstract claims superiority on real-world VSR benchmark datasets, this is load-bearing: please add an evaluation on a real-world paired or pseudo-paired dataset (for example, real LR frames with registered HR references) or a human perceptual study with fidelity controls, and explicitly discuss what VideoLQ no-reference gains can and cannot establish.
  2. [§4.2, Tables 1 and 3] The paper repeatedly claims temporal consistency as a central advantage, yet no temporal consistency metric is reported for comparisons against MGLD, Upscale-A-Video, or RealBasicVSR. Warping Error appears only in the internal ablation of Table 3. Without a warping-error (or equivalent) measurement on the same test clips for all compared methods, the temporal-consistency component of the state-of-the-art claim is unquantified. Please add such a comparison, ideally for both synthetic and VideoLQ outputs.
  3. [Table 1; §4.3] All quantitative results are single-run point estimates with no error bars or significance tests. Several of the claimed advantages are modest (for example, REDS4 LPIPS 0.2518 versus 0.2545 for RealBasicVSR, DISTS 0.1094 versus 0.1171 for MGLD, and YouHQ40 SSIM 0.6759 versus 0.6758 for RealViformer). To support state-of-the-art claims, please report results over multiple seeds for at least the main benchmarks and the YouHQ ablations, or provide statistical significance tests.
  4. [§3.3, Eq. (4)] The contrastive loss and memory queue are underspecified. The queue is written as Q={E_k(x_h_j), (E_k(x_l_j) | j in {0,1,...,K/2}, j != i)}; it is not clear how j indexes the queue versus the batch, whether LR and HR features of the same other video j both serve as negatives, how the projection head produces P x P patch features, and whether all patches contribute equally to the loss. Because MoCoCtrl is a central contribution, this level of detail is needed for reproducibility.
minor comments (8)
  1. [§3.1] The sentence 'the training samples x_h are drown from a HR video dataset' contains a typo: 'drown' should be 'drawn'.
  2. [§3.4] The statement that the HR/LR mixture ratio 'starts at 1 and gradually decreases to 0.3' is ambiguous: specify whether this is the HR:LR ratio or the fraction of HR videos, and give the schedule used in training.
  3. [§4.1] The use of Panda-70M to generate text prompts is mentioned but not explained; please clarify how prompts are used during training and inference and whether test videos are given fixed or empty prompts.
  4. [Figure 2] The circled labels 1, 2, and 3 in Figure 2 are not explained in the caption; please define them or remove them.
  5. [References] References [9] and [10] are the same paper (RealBasicVSR) and should be merged or cross-referenced consistently.
  6. [Table 3] The warping-error metric is not defined; please state how it is computed, which flow method is used, and confirm that lower values are better.
  7. [§3.2.2] The novelty claim for the spatial-temporal continuous scan would be stronger if the text explicitly contrasted it with existing video Mamba scans (for example, SegMamba's 3D sweep scan and any video-Mamba methods in related work) rather than only citing image/medical applications.
  8. [General] No code or trained-model release is mentioned; adding a reproducibility statement or a planned release would help the community verify the state-of-the-art claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's contributions are empirical architectural and training-strategy changes, and no claimed prediction reduces to a fitted constant or self-citation chain.

full rationale

This paper does not contain a derivation chain in which a predicted quantity is equivalent to an input by construction. The central claims are empirical: the proposed SCST framework, composed of the Spatial-Temporal Continuous Mamba (STCM) and self-supervised MoCoCtrl, is evaluated against prior methods on synthetic and real-world benchmarks. No parameter is fitted to a test set and then reported as a prediction; no uniqueness theorem or load-bearing self-citation is invoked; and the architecture choices are supported by ablations rather than by a self-referential argument. The closest issue to circularity is that the synthetic test sets are generated with the same RealBasicVSR degradation pipeline used for training, as stated in Section 4.1: 'four synthetic testing datasets ... follow the same degradation pipeline in training to generate LQ videos.' This means the full-reference perceptual gains on those sets are in-distribution relative to the training degradation model. However, this is a generalization and benchmark-validity limitation, not a logical circularity: the LPIPS and DISTS numbers are still measured against ground-truth HR frames that are external to the network's fitted parameters, and the VideoLQ results provide an independent, though no-reference, real-world evaluation. The weakness of no-reference metrics for verifying fidelity is a correctness or evidence concern, not a circular-reasoning step. No manuscript passage asserts a limitation that reveals a definitional loop, and no quoted equation reduces one claimed result to another by construction. Therefore the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The paper is empirical; no analytic constants are fitted. The free parameters listed are hand-chosen training hyperparameters that the reported performance depends on but that are only partially specified. No new physical or theoretical entities are introduced.

free parameters (6)
  • Stage-1 HR/LR mixture ratio schedule = starts at 1, decays to 0.3
    Chosen by hand in Section 3.4; no ablation on this schedule, though it defines the training curriculum.
  • Contrastive loss temperature tau
    Required in Equation 4; value and sensitivity are not reported.
  • MoCo memory queue size K
    Defined in the memory queue in Section 3.3; exact size and its effect on negative sampling are not reported.
  • Patch size P x P of projection head
    Patch-level contrastive loss in Eq. 4 uses P^2 patches; P is never specified.
  • Number of scan paths K in STCM = 6
    Set to 6 in Section 3.2.2; no ablation on the number of scan directions.
  • Loss weighting for contrastive loss
    Total loss combining Eq. 1 and Eq. 4 is not written; the balance is unspecified.
assumptions (5)
  • domain assumption RealBasicVSR degradation pipeline approximates real-world video degradations.
    Used to generate LQ-HQ pairs for training and for synthetic test sets, Section 4.1.
  • domain assumption Stable Diffusion V2.1 provides a sufficiently strong generative prior for video texture synthesis.
    The U-Net is frozen and used as the denoiser, Section 3.1.
  • domain assumption MoCo-style contrastive learning transfers from representation learning to patch-level super-resolution feature alignment.
    MoCoCtrl directly adapts MoCo to LR/HR patch features, Section 3.3.
  • ad hoc to paper A fixed set of six scan paths captures global spatio-temporal context.
    The continuous scan strategy is introduced in Section 3.2.2 with no formal coverage or ablation on path count.
  • ad hoc to paper The three-stage training order (mix, contrastive, temporal) is necessary for stability.
    Motivated empirically in Section 3.4; no comparison to other curricula or simultaneous training.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-supervised ControlNet with Spatio-Temporal Mamba for Real-world Video Super-resolution." pith.science (2026). https://pith.science/paper/A2JBZMZS

@misc{pith2026250601037,
  author       = {Pith},
  title        = {Pith review of: Self-supervised ControlNet with Spatio-Temporal Mamba for Real-world Video Super-resolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A2JBZMZS}},
  note         = {Machine review of arXiv:2506.01037}
}
read the original abstract

Existing diffusion-based video super-resolution (VSR) methods are susceptible to introducing complex degradations and noticeable artifacts into high-resolution videos due to their inherent randomness. In this paper, we propose a noise-robust real-world VSR framework by incorporating self-supervised learning and Mamba into pre-trained latent diffusion models. To ensure content consistency across adjacent frames, we enhance the diffusion model with a global spatio-temporal attention mechanism using the Video State-Space block with a 3D Selective Scan module, which reinforces coherence at an affordable computational cost. To further reduce artifacts in generated details, we introduce a self-supervised ControlNet that leverages HR features as guidance and employs contrastive learning to extract degradation-insensitive features from LR videos. Finally, a three-stage training strategy based on a mixture of HR-LR videos is proposed to stabilize VSR training. The proposed Self-supervised ControlNet with Spatio-Temporal Continuous Mamba based VSR algorithm achieves superior perceptual quality than state-of-the-arts on real-world VSR benchmark datasets, validating the effectiveness of the proposed model design and training strategies.

Figures

Figures reproduced from arXiv: 2506.01037 by the authors.

Figure 2
Figure 2. Overview of the proposed SCST framework for real-world VSR. SCST consists of several modules, including Spatial-Temporal [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Diagram of Temporal-Spatial Continuous Scan Strat [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 5
Figure 5. Multi-stage HR-LR hybrid training strategy. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figures from the paper (3 more)
Figure 6
Figure 6. Figure 6: Qualitative comparisons on synthetic low-quality videos. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparisons on real-world test videos in VideoLQ. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 9
Figure 9. Figure 9: Different Spatial-temporal modeling approaches [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VEMamba: Efficient Isotropic Reconstruction of Volume Electron Microscopy with Axial-Lateral Consistent Mamba

    cs.CV 2026-03 conditional novelty 5.0 of 10

    VEMamba applies Mamba state-space models with axial-lateral chunked scanning and MoCo-based degradation learning to achieve efficient isotropic reconstruction of volume electron microscopy data.

Reference graph

Works this paper leans on

75 extracted references · 40 canonical work pages · cited by 1 Pith paper

  1. [1]

    Masked siamese networks for label-efficient learning

    Mahmoud Assran, Mathilde Caron, Ishan Misra, Piotr Bo- janowski, Florian Bordes, Pascal Vincent, Armand Joulin, Mike Rabbat, and Nicolas Ballas. Masked siamese networks for label-efficient learning. InEuropean Conference on Com- puter Vision, pages 456–473. Springer, 2022. 3

    work page 2022

  2. [2]

    Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models.arXiv preprint arXiv:2405.04233, 2024

    Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to-video generator with diffusion models.arXiv preprint arXiv:2405.04233, 2024. 4

    arXiv 2024

  3. [3]

    Video super-resolution transformer.arXiv preprint arXiv:2106.06847, 2021

    Jiezhang Cao, Yawei Li, Kai Zhang, and Luc Van Gool. Video super-resolution transformer.arXiv preprint arXiv:2106.06847, 2021. 2

    arXiv 2021

  4. [4]

    Unsupervised learning of visual features by contrasting cluster assignments.Ad- vances in neural information processing systems, 33:9912– 9924, 2020

    Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Pi- otr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments.Ad- vances in neural information processing systems, 33:9912– 9924, 2020. 3

    work page 2020

  5. [5]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 3

    2021

  6. [6]

    Diffusart: Enhancing line art coloriza- tion with conditional diffusion models

    Hernan Carrillo, Micha ¨el Cl ´ement, Aur ´elie Bugeau, and Edgar Simo-Serra. Diffusart: Enhancing line art coloriza- tion with conditional diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3486–3490, 2023. 1

    work page 2023

  7. [7]

    Basicvsr: The search for essential compo- nents in video super-resolution and beyond

    Kelvin CK Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. Basicvsr: The search for essential compo- nents in video super-resolution and beyond. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4947–4956, 2021. 2

    2021

  8. [8]

    Basicvsr++: Improving video super- resolution with enhanced propagation and alignment

    Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Basicvsr++: Improving video super- resolution with enhanced propagation and alignment. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5972–5981, 2022. 2

    2022

Show all 75 references
  1. [9]

    Investigating tradeoffs in real-world video super-resolution

    Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Investigating tradeoffs in real-world video super-resolution. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5962–5971, 2022. 5, 6

    2022

  2. [10]

    Investigating tradeoffs in real-world video super-resolution

    Kelvin CK Chan, Shangchen Zhou, Xiangyu Xu, and Chen Change Loy. Investigating tradeoffs in real-world video super-resolution. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5962–5971, 2022. 2, 7

    2022

  3. [11]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. InInternational conference on ma- chine learning, pages 1597–1607. PMLR, 2020. 3

    2020

  4. [12]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. InProceedings of the IEEE/CVF Conferen...

    2024

  5. [13]

    Image quality assessment: Unifying structure and texture similarity.IEEE transactions on pattern analysis and ma- chine intelligence, 44(5):2567–2581, 2020

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P Simoncelli. Image quality assessment: Unifying structure and texture similarity.IEEE transactions on pattern analysis and ma- chine intelligence, 44(5):2567–2581, 2020. 6

    2020

  6. [14]

    Adaptive soft contrastive learning

    Chen Feng and Ioannis Patras. Adaptive soft contrastive learning. In2022 26th International Conference on Pattern Recognition (ICPR), pages 2721–2727. IEEE, 2022. 3

    2022

  7. [15]

    Maskcon: Masked con- trastive learning for coarse-labelled dataset

    Chen Feng and Ioannis Patras. Maskcon: Masked con- trastive learning for coarse-labelled dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 19913–19922, 2023. 3

    2023

  8. [16]

    Ssr: An efficient and robust framework for learning with un- known label noise.arXiv preprint arXiv:2111.11288, 2021

    Chen Feng, Georgios Tzimiropoulos, and Ioannis Patras. Ssr: An efficient and robust framework for learning with un- known label noise.arXiv preprint arXiv:2111.11288, 2021. 3

    2021 arXiv

  9. [17]

    Self-supervised representation learning with cross-context learning between global and hypercolumn features

    Zheng Gao, Chen Feng, and Ioannis Patras. Self-supervised representation learning with cross-context learning between global and hypercolumn features. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1773–1783, 2024. 3

    2024

  10. [18]

    Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023. 2, 4

    2023 arXiv

  11. [19]

    Efficiently modeling long sequences with structured state spaces.arXiv preprint arXiv:2111.00396, 2021

    Albert Gu, Karan Goel, and Christopher R ´e. Efficiently modeling long sequences with structured state spaces.arXiv preprint arXiv:2111.00396, 2021. 2

    2021 arXiv

  12. [20]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning.arXiv preprint arXiv:2307.04725, 2023. 8

    2023 arXiv

  13. [21]

    Diagonal state spaces are as effective as structured state spaces.Advances in Neural Information Processing Systems, 35:22982–22994,

    Ankit Gupta, Albert Gu, and Jonathan Berant. Diagonal state spaces are as effective as structured state spaces.Advances in Neural Information Processing Systems, 35:22982–22994,

  14. [22]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 9729–9738, 2020. 2, 3, 5

    2020

  15. [23]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 3

    2022

  16. [24]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  17. [25]

    Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022. 1 9

    arXiv

  18. [26]

    Long movie clip classification with state-space video models

    Md Mohaiminul Islam and Gedas Bertasius. Long movie clip classification with state-space video models. InEuropean Conference on Computer Vision, pages 87–104. Springer,

  19. [27]

    Video super-resolution with recurrent structure-detail network

    Takashi Isobe, Xu Jia, Shuhang Gu, Songjiang Li, Shengjin Wang, and Qi Tian. Video super-resolution with recurrent structure-detail network. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XII 16, pages 645–660. Springer,

    2020

  20. [28]

    Video super-resolution with temporal group attention

    Takashi Isobe, Songjiang Li, Xu Jia, Shanxin Yuan, Gregory Slabaugh, Chunjing Xu, Ya-Li Li, Shengjin Wang, and Qi Tian. Video super-resolution with temporal group attention. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 8008–8017, 2020

    2020

  21. [29]

    Re- visiting temporal modeling for video super-resolution.arXiv preprint arXiv:2008.05765, 2020

    Takashi Isobe, Fang Zhu, Xu Jia, and Shengjin Wang. Re- visiting temporal modeling for video super-resolution.arXiv preprint arXiv:2008.05765, 2020. 2

    2008 arXiv

  22. [30]

    Dynamic filter networks

    Xu Jia, Bert De Brabandere, Tinne Tuytelaars, and Luc Van Gool. Dynamic filter networks. InNIPS, pages 667–675,

  23. [31]

    Deep video super-resolution network using dynamic upsampling filters without explicit motion compen- sation

    Younghyun Jo, Seoung Wug Oh, Jaeyeon Kang, and Seon Joo Kim. Deep video super-resolution network using dynamic upsampling filters without explicit motion compen- sation. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 3224–3232, 2018. 2

    2018

  24. [32]

    A new approach to linear filtering and prediction problems

    Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. 1960. 4

    1960

  25. [33]

    Denoising diffusion restoration models.Advances in Neural Information Processing Systems, 35:23593–23606,

    Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models.Advances in Neural Information Processing Systems, 35:23593–23606,

  26. [34]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6007–6017, 2023. 1

    2023

  27. [35]

    Musiq: Multi-scale image quality transformer

    Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. InProceedings of the IEEE/CVF international conference on computer vision, pages 5148–5157, 2021. 6

    2021

  28. [36]

    A method for stochastic optimization

    D Kinga, Jimmy Ba Adam, et al. A method for stochastic optimization. InInternational conference on learning rep- resentations (ICLR), page 6. San Diego, California;, 2015. 6

    2015

  29. [37]

    Open-sora-plan, 2024

    PKU-Yuan Lab and Tuzhan AI etc. Open-sora-plan, 2024. 4

    2024

  30. [38]

    Learning blind video temporal consistency

    Wei-Sheng Lai, Jia-Bin Huang, Oliver Wang, Eli Shechtman, Ersin Yumer, and Ming-Hsuan Yang. Learning blind video temporal consistency. InECCV, pages 170–185, 2018. 8

    2018

  31. [39]

    Mamba- nd: Selective state space modeling for multi-dimensional data.arXiv preprint arXiv:2402.05892, 2024

    Shufan Li, Harkanwar Singh, and Aditya Grover. Mamba- nd: Selective state space modeling for multi-dimensional data.arXiv preprint arXiv:2402.05892, 2024. 3

    2024 arXiv

  32. [40]

    Mucan: Multi-correspondence aggregation net- work for video super-resolution

    Wenbo Li, Xin Tao, Taian Guo, Lu Qi, Jiangbo Lu, and Jiaya Jia. Mucan: Multi-correspondence aggregation net- work for video super-resolution. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part X 16, pages 335–351. Springer,

    2020

  33. [41]

    Point- mamba: A simple state space model for point cloud analysis

    Dingkang Liang, Xin Zhou, Xinyu Wang, Xingkui Zhu, Wei Xu, Zhikang Zou, Xiaoqing Ye, and Xiang Bai. Point- mamba: A simple state space model for point cloud analysis. arXiv preprint arXiv:2402.10739, 2024. 3

    2024 arXiv

  34. [42]

    Recurrent video restoration trans- former with guided deformable attention.Advances in Neu- ral Information Processing Systems, 35:378–393, 2022

    Jingyun Liang, Yuchen Fan, Xiaoyu Xiang, Rakesh Ranjan, Eddy Ilg, Simon Green, Jiezhang Cao, Kai Zhang, Radu Timofte, and Luc V Gool. Recurrent video restoration trans- former with guided deformable attention.Advances in Neu- ral Information Processing Systems, 35:378–393, 2022. 2

    2022

  35. [43]

    Vrt: A video restoration transformer.IEEE Transactions on Image Processing, 2024

    Jingyun Liang, Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, and Luc Van Gool. Vrt: A video restoration transformer.IEEE Transactions on Image Processing, 2024. 2

    2024

  36. [44]

    On bayesian adaptive video super resolution.IEEE transactions on pattern analysis and ma- chine intelligence, 36(2):346–360, 2013

    Ce Liu and Deqing Sun. On bayesian adaptive video super resolution.IEEE transactions on pattern analysis and ma- chine intelligence, 36(2):346–360, 2013. 2

    2013

  37. [45]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11461–11471, 2022. 1

    2022

  38. [46]

    completely blind

    Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Mak- ing a “completely blind” image quality analyzer.IEEE Sig- nal processing letters, 20(3):209–212, 2012. 6

    2012

  39. [47]

    Ntire 2019 challenge on video deblurring and super- resolution: Dataset and study

    Seungjun Nah, Sungyong Baik, Seokil Hong, Gyeongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee. Ntire 2019 challenge on video deblurring and super- resolution: Dataset and study. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition works...

    2019

  40. [48]

    S4nd: Modeling images and videos as multidimensional signals with state spaces.Advances in neural information processing systems, 35:2846–2861, 2022

    Eric Nguyen, Karan Goel, Albert Gu, Gordon Downs, Preey Shah, Tri Dao, Stephen Baccus, and Christopher R ´e. S4nd: Modeling images and videos as multidimensional signals with state spaces.Advances in neural information processing systems, 35:2846–2861, 2022. 2

    2022

  41. [49]

    Deep blind video super-resolution

    Jinshan Pan, Haoran Bai, Jiangxin Dong, Jiawei Zhang, and Jinhui Tang. Deep blind video super-resolution. InProceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 4811–4820, 2021. 7

    2021

  42. [50]

    Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih- Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720,

    arXiv

  43. [51]

    Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents.arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 1

    2022 arXiv

  44. [52]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  45. [53]

    Photorealistic text-to-image diffusion models with deep 10 language understanding.Advances in neural information processing systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep 10 language understanding.Advances in neural informati...

    2022

  46. [54]

    Simplified state space layers for sequence modeling

    Jimmy TH Smith, Andrew Warrington, and Scott W Linder- man. Simplified state space layers for sequence modeling. arXiv preprint arXiv:2208.04933, 2022. 2

    2022 arXiv

  47. [55]

    Detail-revealing deep video super-resolution

    Xin Tao, Hongyun Gao, Renjie Liao, Jue Wang, and Jiaya Jia. Detail-revealing deep video super-resolution. InPro- ceedings of the IEEE international conference on computer vision, pages 4472–4480, 2017. 6

    2017

  48. [56]

    Ex- ploring clip for assessing the look and feel of images

    Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Ex- ploring clip for assessing the look and feel of images. InPro- ceedings of the AAAI Conference on Artificial Intelligence, pages 2555–2563, 2023. 6

    2023

  49. [57]

    Selective structured state-spaces for long-form video understanding

    Jue Wang, Wentao Zhu, Pichao Wang, Xiang Yu, Linda Liu, Mohamed Omar, and Raffay Hamid. Selective structured state-spaces for long-form video understanding. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6387–6397, 2023. 2

    2023

  50. [58]

    Exploiting diffusion prior for real-world image super-resolution.International Journal of Computer Vision, pages 1–21, 2024

    Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin CK Chan, and Chen Change Loy. Exploiting diffusion prior for real-world image super-resolution.International Journal of Computer Vision, pages 1–21, 2024. 1, 6

    2024

  51. [59]

    Exploiting diffusion prior for real-world image super-resolution.International Journal of Computer Vision, pages 1–21, 2024

    Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin CK Chan, and Chen Change Loy. Exploiting diffusion prior for real-world image super-resolution.International Journal of Computer Vision, pages 1–21, 2024. 1

    2024

  52. [60]

    Edvr: Video restoration with enhanced deformable convolutional networks

    Xintao Wang, Kelvin CK Chan, Ke Yu, Chao Dong, and Chen Change Loy. Edvr: Video restoration with enhanced deformable convolutional networks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 0–0, 2019. 2, 5

    2019

  53. [61]

    Real-esrgan: Training real-world blind super-resolution with pure synthetic data

    Xintao Wang, Liangbin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super-resolution with pure synthetic data. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 1905–1914,

    1905

  54. [62]

    Exploring video quality assessment on user gener- ated contents from aesthetic and technical perspectives

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jing- wen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user gener- ated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conferenc...

    2023

  55. [63]

    Mitigating artifacts in real-world video super-resolution models

    Liangbin Xie, Xintao Wang, Shuwei Shi, Jinjin Gu, Chao Dong, and Ying Shan. Mitigating artifacts in real-world video super-resolution models. InProceedings of the AAAI Conference on Artificial Intelligence, pages 2956–2964,

  56. [64]

    Simmim: A simple framework for masked image modeling

    Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu. Simmim: A simple framework for masked image modeling. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9653–9663, 2022. 3

    2022

  57. [65]

    Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation

    Zhaohu Xing, Tian Ye, Yijun Yang, Guang Liu, and Lei Zhu. Segmamba: Long-range sequential modeling mamba for 3d medical image segmentation. InInternational Conference on Medical Image Computing and Computer-Assisted Interven- tion, pages 578–588. Springer, 2024. 4

    2024

  58. [66]

    Video enhancement with task-oriented flow.International Journal of Computer Vision, 127:1106– 1125, 2019

    Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with task-oriented flow.International Journal of Computer Vision, 127:1106– 1125, 2019. 2

    2019

  59. [67]

    Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization.arXiv preprint arXiv:2308.14469, 2023

    Tao Yang, Rongyuan Wu, Peiran Ren, Xuansong Xie, and Lei Zhang. Pixel-aware stable diffusion for realistic image super-resolution and personalized stylization.arXiv preprint arXiv:2308.14469, 2023. 1

    2023 arXiv

  60. [68]

    Real- world video super-resolution: A benchmark dataset and a de- composition based learning scheme

    Xi Yang, Wangmeng Xiang, Hui Zeng, and Lei Zhang. Real- world video super-resolution: A benchmark dataset and a de- composition based learning scheme. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4781–4790, 2021. 2

    2021

  61. [69]

    Motion- guided latent diffusion for temporally consistent real-world video super-resolution.arXiv preprint arXiv:2312.00853,

    Xi Yang, Chenhang He, Jianqi Ma, and Lei Zhang. Motion- guided latent diffusion for temporally consistent real-world video super-resolution.arXiv preprint arXiv:2312.00853,

    arXiv

  62. [70]

    Progressive fusion video super-resolution network via exploiting non-local spatio-temporal correlations

    Peng Yi, Zhongyuan Wang, Kui Jiang, Junjun Jiang, and Ji- ayi Ma. Progressive fusion video super-resolution network via exploiting non-local spatio-temporal correlations. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 3106–3115, 2019. 2, 5

    2019

  63. [71]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 3

    2023

  64. [72]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 586–595, 2018. 6

    2018

  65. [73]

    Realviformer: Investigat- ing attention for real-world video super-resolution.arXiv preprint arXiv:2407.13987, 2024

    Yuehan Zhang and Angela Yao. Realviformer: Investigat- ing attention for real-world video super-resolution.arXiv preprint arXiv:2407.13987, 2024. 7

    2024 arXiv

  66. [74]

    Upscale-a-video: Temporal- consistent diffusion model for real-world video super- resolution

    Shangchen Zhou, Peiqing Yang, Jianyi Wang, Yihang Luo, and Chen Change Loy. Upscale-a-video: Temporal- consistent diffusion model for real-world video super- resolution. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2535– 2545, 2024...

    2024

  67. [75]

    Vision mamba: Efficient visual representation learning with bidirectional state space model.arXiv preprint arXiv:2401.09417, 2024

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model.arXiv preprint arXiv:2401.09417, 2024. 3 11

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.