Pith. sign in

REVIEW 3 major objections 4 minor 67 references

Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes DCKD, a knowledge-distillation framework that makes the lower bound of the solution space dynamic via EMA-generated negatives and aligns teacher–student pixel category distributions with a codebook, reporting…

desk verdict A broadly tested but incremental KD method whose central "dynamic" claim is undermined by a missing control experiment. read the letter →

arxiv 2412.08939 v2 pith:C67OGVU6 submitted 2024-12-12 cs.CV

Yunshuai Zhou , Junbo Qiao , Jincheng Liao , Wei Li , Simiao Li , Jiao Xie , Yunhang Shen , Jie Hu
show 1 more author
Shaohui Lin
This is my paper · ORCID
classification cs.CV
keywords imagerestorationknowledgedistillationcontrastivelearningdynamiclowerbounddistributionmappingsuper-resolutiondeblurringderaining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that knowledge distillation for image restoration is held back by fixed solution spaces, where the lower-bound constraint weakens as the student improves. It proposes DCKD, which generates negative examples from an exponential-moving-average copy of the student applied to randomly degraded inputs, and uses contrastive learning to keep the student close to the teacher and ground truth while repelling it from these dynamic negatives. A second module, DMM, aligns pixel-level category distributions between teacher and student outputs through a pretrained VQGAN codebook and cross-entropy loss. Across super-resolution, deblurring, and deraining, the paper reports consistent PSNR/SSIM gains over existing KD methods on CNN and Transformer backbones.

What carries the argument

The carrier of the argument is the Dynamic Negative Sample Generator (DNSG): a degradation module paired with an exponential-moving-average history model of the student. Given a low-quality input, it produces $N$ negative images that define a moving lower bound in feature space, and the dynamic contrastive loss $L_{\mathrm{dcl}}$ in Eq. 5 enforces a ratio between the distance to the positive and the summed distances to the negatives. The second carrier is the Distribution Mapping Module (DMM): a pretrained VQGAN encoder and codebook convert both teacher and student outputs into per-pixel soft category distributions aligned by cross-entropy. Together these replace a static solution space with one that tightens as the student learns.

What would settle it

Train the same DCKD pipeline but replace the EMA history model with a frozen random network, the teacher network, or the current student itself; if PSNR/SSIM do not drop materially for the frozen or teacher replacements, the dynamic tracking of the student's state is not the source of the gain. A second check is to record the feature distances in Eq. 5 over training and confirm the negatives stay below the anchor, since the argument requires the anchor to move away from a retreating lower bound.

Watch

Extended reading notes

Core claim

The central claim is that making the lower bound of the distillation solution space dynamic, rather than fixed, is what unlocks better student performance. Concretely, the student anchor is contrasted against negatives produced by applying random degradations to the low-quality input and reconstructing with a history model that tracks the student by exponential moving average; the dynamic contrastive loss pulls the student toward the teacher/ground-truth positive and pushes it away from these negatives. The paper also introduces distribution mapping, which turns teacher and student outputs into pixel-level category distributions via a VQGAN codebook and aligns them with cross-entropy. The combined loss is $L = L_{\mathrm{rec}} + L_{\mathrm{kd}} + \lambda_{\mathrm{dcl}} L_{\mathrm{dcl}} + \lambda_{\mathrm{ce}} L_{\mathrm{ce}}$, and the paper shows this recipe improves student PSNR/SSIM over fixed lower-bound methods and remains compatible with upper-bound-improving methods such as DUKD.

Load-bearing premise

The method assumes that randomly degrading the input and running the EMA copy of the student produces negatives that actually represent the lower boundary of the student's current output space at every training stage; the paper does not measure this directly, and its ablation shows the result is sensitive to the history-model update schedule.

Editorial extensions

If this is right

  • Students trained with DCKD outperform students trained with logits-only or fixed-lower-bound contrastive KD on super-resolution, deblurring, and deraining benchmarks.
  • Because DCKD is structure-agnostic, its losses can be attached to both CNN and Transformer restoration backbones without architectural changes.
  • DCKD can be composed with upper-bound-optimizing distillation methods, and the combination yields larger gains than either component alone.
  • The DMM's pixel-level category alignment gives low-level vision a distribution-level distillation signal that global distribution matching does not provide.
  • The main gains come from the dynamic lower bound, since DCKD without any upper-bound enhancement still beats the fixed-lower-bound baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The EMA history model behaves like a soft negative curriculum; an untested extension is to anneal the degradation severity or the EMA update rate over training to control how far the negatives trail the anchor.
  • Because the negatives are derived from the input image itself rather than from a fixed database, the same mechanism should transfer to blind restoration settings where a degradation model is available.
  • The reliance on a fixed pretrained VQGAN codebook raises the question of whether finer or task-specific codebooks would change the outcome; that is a testable variant the paper does not run.
  • The repulsion term in Eq. 5 is a sum of distances; whether a max-margin or triplet form of the same dynamic lower bound behaves differently is left open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DCKD, a knowledge distillation framework for image restoration that combines dynamic contrastive regularization (DCR) with a distribution mapping module (DMM). DCR generates negative samples by applying random degradations to the low-quality input and passing them through an EMA-updated history model of the student, then uses a contrastive loss (Eq. 5) to pull student features toward the teacher output while pushing them away from these negatives. DMM extracts pixel-level category distributions from teacher and student outputs using a pretrained VQGAN encoder and codebook, and aligns them with cross-entropy loss (Eq. 9). The total loss (Eq. 10) adds these terms to reconstruction and vanilla KD losses. Experiments on super-resolution (SwinIR, RCAN), deblurring (NAFNet, Restormer), and deraining (Restormer) report PSNR/SSIM improvements over the compared methods, with ablations on components, degradation types, balancing weights, number of negatives, and update step.

Significance. If the reported results are reproducible, DCKD provides a potentially useful plug-in for image-restoration knowledge distillation, with a concrete mechanism for using learned lower bounds and distribution-level alignment. The paper's strengths include evaluation across three tasks and multiple backbone architectures, ablations of the main components, and a promised public code release. However, the central claim of 'dynamic' adjustment is not directly evidenced: the EMA mechanism is never compared against a static-history counterpart, and the closest contrastive baselines (CSD, Luo et al.) are absent from the main tables. The improvements over the strongest baselines are often modest (about 0.1 dB) and no variance information is given, so the significance at this stage is conditional on filling these gaps.

major comments (3)
  1. [Methodology, Eq. (4)-(6); Experiments, Fig. 8] The manuscript's central claim is that the EMA history model in Eq. (6) tracks the student's learning state and thereby dynamically adjusts the lower bound of the solution space. However, no experiment isolates this dynamic property. In the ablations (Tab. 5, Fig. 8), the authors vary the initial update step and the number of negatives, but they never compare against a control with a static history model (e.g., F_his frozen at initialization or replaced by a fixed pretrained network). Under such a static-history control, the contrastive loss would still penalize deviation of the student from older outputs on degraded inputs, so any gain in that control would indicate that the improvement comes from the degradation-based contrastive augmentation rather than from tracking the student's state. Because the paper's novelty and title rest on the word 'dynamic,' this omitted control is load-bearing. I request adding this ablation, and ideally a plot of the distance between negatives and the student anchor over training to substantiate the 'perceive the student's learning state' claim.
  2. [Experiments, Tabs. 2-4 and Table 8] The related work identifies CSD (Wang et al. 2021d) and Luo et al. (2023) as the closest contrastive KD baselines with fixed lower bounds, yet neither appears in the main comparison tables (Tabs. 2-4). Table 8 compares only MCLIR on super-resolution; the claim of 'significantly outperforms the state-of-the-art KD methods' cannot be evaluated without these direct competitors on the same tasks and backbones. In addition, DCKD* in Tab. 2 is presented as the combination of DCKD with DUKD, but no DUKD-only result is reported; without that baseline, the gain of DCKD* over DCKD cannot be attributed to the upper-bound combination. Please add these experiments or temper the claims accordingly.
  3. [Experimental Settings, Implementation Details] The reported improvements over the strongest baseline are small (e.g., about 0.1 dB on Urban100 in Tab. 2, 0.17 dB in Tab. 3, and 0.55 dB on Rain100L in Tab. 4), and no error bars, repeated-run statistics, or significance tests are provided. Moreover, the hyperparameters λdcl, λce, N, and the initial update step are selected on the validation sets and then fixed across all tasks (Tabs. 7, Fig. 8), which raises a selection-overfitting concern. For the main claims of consistent improvement, at least three repeated runs with mean and standard deviation (or paired tests) should be reported for the key comparisons.
minor comments (4)
  1. [Figure 8] The Figure 8 caption contains a corrupted placeholder string ('/uni00000016/...') and the figure labels are unreadable; the figure and caption need to be regenerated.
  2. [Table 4] Several entries in Table 4 have inconsistent spacing (e.g., '30.54 /0.8969' and '38.02 /0.9762'); please format uniformly.
  3. [Appendix, Implementation Details] In the NAFNet and Restormer training details, 'weight deacy' should read 'weight decay'.
  4. [Eq. (5)] The denominator in Eq. (5) sums over N negative samples, but the scaling behavior of the loss with N is not discussed; a brief explanation of whether and how the loss is normalized would improve clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DCKD is an empirical, benchmark-evaluated training framework with no equation-level reduction or load-bearing self-citation.

full rationale

The paper makes no formal derivation claim that reduces a predicted quantity to a fitted input. Eq. 4 defines negative samples through random degradation plus the EMA history model; Eq. 5 defines the dynamic contrastive loss; Eq. 10 is the total training loss. These are construction choices for a training objective, not derivations whose conclusion is equivalent to their premise. The central claim, that DCKD improves student PSNR/SSIM across restoration tasks, is tested on held-out external benchmarks (Set5, Set14, BSD100, Urban100, GoPro, Rain100H, etc.) with frozen teacher and pretrained VQGAN encoders. No fitted parameter is renamed as a prediction: the balancing weights and update-step schedule are reported as ablations, and the measured gains come from external test-set metrics rather than from the training objective by construction. Some self-citations appear (MiPKD and DUKD share authors with this paper, and DCKD* builds on DUKD), but these are used as comparative baselines and combination components, not as an authority that forces the framework's choice. No uniqueness theorem from the authors' prior work is invoked, and no ansatz is smuggled in solely through self-citation. The absence of a static-history control in the experiments is an empirical validation concern, not a circularity, because it does not make the reported results true by definition. Overall, the derivation chain is self-contained and empirically falsifiable.

Assumptions & free parameters 7 free parameters · 5 assumptions · 3 invented entities

The framework rests on several tuned hyperparameters and on assumptions about EMA history states, degraded negatives, and VQGAN codebook semantics. These are not derived from theory and require independent validation.

free parameters (7)
  • lambda_dcl = 0.1
    Balancing weight for the dynamic contrastive loss in Eq. 10; selected by validation ablation in Tab. 7 and fixed as a default across tasks.
  • lambda_ce = 0.001
    Balancing weight for the cross-entropy distribution loss in Eq. 10; selected by validation ablation in Tab. 7.
  • number_of_negative_samples N = 5
    Chosen as the best trade-off between PSNR and training cost in Fig. 8(a).
  • initial_history_update_step = 1000
    Chosen as the best initial step for updating the EMA history model in Fig. 8(b).
  • EMA decay alpha = 0.1
    Set in the Appendix implementation details; not ablated in the main paper.
  • VQGAN layer weights lambda_i = 1/32, 1/16, 1/8, 1/4, 1
    Feature-layer weights for Ldcl in Eq. 5, set in the Appendix without an ablation.
  • EMA update-step schedule s = gradually increases
    Eq. 6 states that s gradually increases during training, but the exact schedule is not specified, leaving a free implementation choice.
assumptions (5)
  • ad hoc to paper Random degradation of the low-quality input followed by the EMA history model produces negative samples that represent the lower bound of the student's current solution space.
    Central to Eq. 4 and Ldcl; the paper provides no formal characterization of 'solution space', and Fig. 8 shows performance varies with the update step.
  • domain assumption The VQGAN encoder and codebook pretrained on ImageNet provide meaningful features and category distributions for image-restoration outputs.
    Used in Eqs. 7-9; transfer of ImageNet codebook semantics to pixel-level restoration is assumed rather than demonstrated.
  • ad hoc to paper Cross-entropy between soft pixel-category distributions of teacher and student is a suitable alignment objective for low-level vision.
    DMM design choice in Eq. 9; no comparison with alternative distribution alignments is provided.
  • domain assumption EMA parameters of the student approximate its historical state and therefore encode its learning state.
    Eq. 6 uses a standard EMA, but the leap from EMA weights to a well-defined 'learning state' is heuristic.
  • domain assumption Standard degradations (blur, noise, resize) cover the variation needed for informative negative samples.
    Adopted from Real-ESRGAN degradation modeling; not validated as a sampler for the restoration solution space.
invented entities (3)
  • EMA history model F_his_S
    purpose: Generate negative image samples from degraded inputs using stale student weights.
    Introduced as part of the dynamic negative sample generator; only in-paper ablations support it, with no external verification.
  • Dynamic Negative Sample Generator (DNSG)
    purpose: Produce N negative images that define the dynamic lower bound for contrastive distillation.
    New component described in the Methodology; its effectiveness depends on heuristics for degradation and EMA update scheduling.
  • Pixel-level category distribution over the VQGAN codebook
    purpose: Provide distribution supervision for the cross-entropy loss in DMM.
    The paper introduces this concept without evidence that codebook categories correspond to perceptually meaningful restoration classes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration." pith.science (2026). https://pith.science/paper/C67OGVU6

@misc{pith2026241208939,
  author       = {Pith},
  title        = {Pith review of: Dynamic Contrastive Knowledge Distillation for Efficient Image Restoration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C67OGVU6}},
  note         = {Machine review of arXiv:2412.08939}
}
read the original abstract

Knowledge distillation (KD) is a valuable yet challenging approach that enhances a compact student network by learning from a high-performance but cumbersome teacher model. However, previous KD methods for image restoration overlook the state of the student during the distillation, adopting a fixed solution space that limits the capability of KD. Additionally, relying solely on L1-type loss struggles to leverage the distribution information of images. In this work, we propose a novel dynamic contrastive knowledge distillation (DCKD) framework for image restoration. Specifically, we introduce dynamic contrastive regularization to perceive the student's learning state and dynamically adjust the distilled solution space using contrastive learning. Additionally, we also propose a distribution mapping module to extract and align the pixel-level category distribution of the teacher and student models. Note that the proposed DCKD is a structure-agnostic distillation framework, which can adapt to different backbones and can be combined with methods that optimize upper-bound constraints to further enhance model performance. Extensive experiments demonstrate that DCKD significantly outperforms the state-of-the-art KD methods across various image restoration tasks and backbones.

Figures

Figures reproduced from arXiv: 2412.08939 by the authors.

Figure 1
Figure 1. Difference between our DCKD and existing KD [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed Dynamic Contrastive Knowledge Distillation framework. Our DCKD consists of two [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the Image-level Distribution and our [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Visual comparison for image deblurring. We deliberately use the most straightforward approach to demonstrate DCKD, showing that dynamic lower-bound constraints can yield strong results even without improv￾ing upper-bound constraints. To demonstrate that DCKD can be com…
Figure 5
Figure 5. Figure 5: The visual comparison of distilling SwinIR on Urban100 for [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The visual comparison of distilling RCAN on Urban100 for [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Visual comparison for image deraining. rained images. DCKD further enhances the student’s abil￾ity to remove rain streaks compared to the logits distillation method [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Ablation studies on the number of negative sam [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: More visual comparison of image super-resolution at [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: More visual comparison of image deblurring. [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: More visual comparison of image deraining. [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 34 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ahn, N.; Kang, B.; and Sohn, K.-A. 2018. Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of the European conference on computer vision (ECCV), 252--268

    2018

  4. [4]

    Ayazoglu, M. 2021. Extremely lightweight quantization robust real-time single-image super resolution for mobile devices. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2472--2479

    work page 2021

  5. [5]

    Chen, H.; Wang, Y.; Guo, T.; Xu, C.; Deng, Y.; Liu, Z.; Ma, S.; Xu, C.; Xu, C.; and Gao, W. 2021. Pre-trained image processing transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12299--12310

    2021

  6. [6]

    Chen, L.; Chu, X.; Zhang, X.; and Sun, J. 2022 a . Simple baselines for image restoration. In European conference on computer vision, 17--33. Springer

    2022

  7. [7]

    Chen, X.; Cao, Q.; Zhong, Y.; Zhang, J.; Gao, S.; and Tao, D. 2022 b . Dearkd: data-efficient early knowledge distillation for vision transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12052--12062

    work page 2022

  8. [8]

    Chen, X.; Wang, X.; Zhou, J.; Qiao, Y.; and Dong, C. 2023. Activating more pixels in image super-resolution transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 22367--22377

    2023

Show all 67 references
  1. [9]

    Chen, Z.; Zhang, Y.; Gu, J.; Kong, L.; Yuan, X.; et al. 2022 c . Cross aggregation transformer for image restoration. Advances in Neural Information Processing Systems, 35: 25478--25490

    2022

  2. [10]

    C.; He, K.; and Tang, X

    Dong, C.; Loy, C. C.; He, K.; and Tang, X. 2015. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2): 295--307

    2015

  3. [11]

    Du, Z.; Liu, J.; Tang, J.; and Wu, G. 2021. Anchor-based plain net for mobile image super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2494--2502

    2021

  4. [12]

    Esser, P.; Rombach, R.; and Ommer, B. 2021. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12873--12883

    2021

  5. [13]

    Fan, Y.; Yu, J.; Mei, Y.; Zhang, Y.; Fu, Y.; Liu, D.; and Huang, T. S. 2020. Neural sparse representation for image restoration. Advances in Neural Information Processing Systems, 33: 15394--15404

    2020

  6. [14]

    Fu, H.; Zhou, S.; Yang, Q.; Tang, J.; Liu, G.; Liu, K.; and Li, X. 2021. LRC-BERT: latent-representation contrastive knowledge distillation for natural language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 12830--12838

    2021

  7. [15]

    Fu, X.; Huang, J.; Zeng, D.; Huang, Y.; Ding, X.; and Paisley, J. 2017. Removing rain from single images via a deep detail network. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3855--3863

    2017

  8. [16]

    J.; and Tao, D

    Gou, J.; Yu, B.; Maybank, S. J.; and Tao, D. 2021. Knowledge distillation: A survey. International Journal of Computer Vision, 129(6): 1789--1819

    2021

  9. [17]

    Gou, Y.; Li, B.; Liu, Z.; Yang, S.; and Peng, X. 2020. Clearer: Multi-scale neural architecture search for image restoration. Advances in neural information processing systems, 33: 17129--17140

    2020

  10. [18]

    Hahn, S.; and Choi, H. 2019. Self-knowledge distillation in natural language processing. arXiv preprint arXiv:1908.01851

    2019 arXiv

  11. [19]

    He, W.; Yao, Q.; Yokoya, N.; Uezato, T.; Zhang, H.; and Zhang, L. 2022. Spectrum-aware and transferable architecture search for hyperspectral image restoration. In European Conference on Computer Vision, 19--37. Springer

    2022

  12. [20]

    He, Z.; Dai, T.; Lu, J.; Jiang, Y.; and Xia, S.-T. 2020. Fakd: Feature-affinity based knowledge distillation for efficient image super-resolution. In 2020 IEEE International Conference on Image Processing (ICIP), 518--522. IEEE

    2020

  13. [21]

    Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531

    2015 arXiv

  14. [22]

    Hong, C.; Baik, S.; Kim, H.; Nah, S.; and Lee, K. M. 2022. Cadyq: Content-aware dynamic quantization for image super-resolution. In European Conference on Computer Vision, 367--383. Springer

    2022

  15. [23]

    Huang, T.; You, S.; Wang, F.; Qian, C.; and Xu, C. 2022. Knowledge distillation from a stronger teacher. Advances in Neural Information Processing Systems, 35: 33716--33727

    2022

  16. [24]

    Jiang, Y.; Feng, C.; Zhang, F.; and Bull, D. 2024. MTKD: Multi-Teacher Knowledge Distillation for Image Super-Resolution. arXiv preprint arXiv:2404.09571

    2024 arXiv

  17. [25]

    Kim, H.; Baik, S.; Choi, M.; Choi, J.; and Lee, K. M. 2021. Searching for controllable image restoration networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 14234--14243

    2021

  18. [26]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  19. [27]

    Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25

    2012

  20. [28]

    Lee, W.; Lee, J.; Kim, D.; and Ham, B. 2020. Learning with privileged information for efficient image super-resolution. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXIV 16, 465--482. Springer

    2020

  21. [29]

    Lefkimmiatis, S. 2017. Non-local color image denoising with convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3587--3596

    2017

  22. [30]

    Li, C.; Guo, J.; Porikli, F.; Fu, H.; and Pang, Y. 2018. A cascaded convolutional neural network for single image dehazing. IEEE Access, 6: 24877--24887

    2018

  23. [31]

    Li, S.; Zhang, Y.; Li, W.; Chen, H.; Wang, W.; Jing, B.; Lin, S.; and Hu, J. 2024. Knowledge Distillation with Multi-granularity Mixture of Priors for Image Super-Resolution. arXiv preprint arXiv:2404.02573

    2024 arXiv

  24. [32]

    T.; Guo, X.; Lu, J.; and Brown, M

    Li, Y.; Tan, R. T.; Guo, X.; Lu, J.; and Brown, M. S. 2016. Rain streak removal using layer priors. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2736--2744

    2016

  25. [33]

    Liang, J.; Cao, J.; Sun, G.; Zhang, K.; Van Gool, L.; and Timofte, R. 2021. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision, 1833--1844

    2021

  26. [34]

    Lim, B.; Son, S.; Kim, H.; Nah, S.; and Mu Lee, K. 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 136--144

    2017

  27. [35]

    Lin, S.; Xie, H.; Wang, B.; Yu, K.; Chang, X.; Liang, X.; and Wang, G. 2022. Knowledge distillation via the target-aware transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10915--10924

    2022

  28. [36]

    Loshchilov, I.; and Hutter, F. 2016. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983

    2016 arXiv

  29. [37]

    Luo, Y.; Huang, Q.; Ling, J.; Lin, K.; and Zhou, T. 2023. Local and global knowledge distillation with direction-enhanced contrastive learning for single-image deraining. Knowledge-Based Systems, 268: 110480

    2023

  30. [38]

    Nah, S.; Hyun Kim, T.; and Mu Lee, K. 2017 a . Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3883--3891

    2017

  31. [39]

    Nah, S.; Hyun Kim, T.; and Mu Lee, K. 2017 b . Deep multi-scale convolutional neural network for dynamic scene deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3883--3891

    2017

  32. [40]

    Oh, J.; Kim, H.; Nah, S.; Hong, C.; Choi, J.; and Lee, K. M. 2022. Attentive fine-grained structured sparsity for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 17673--17682

    2022

  33. [41]

    U.; Kim, J.; and Chun, S

    Park, D.; Kang, D. U.; Kim, J.; and Chun, S. Y. 2020. Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In European Conference on Computer Vision, 327--343. Springer

    2020

  34. [42]

    Park, W.; Kim, D.; Lu, Y.; and Cho, M. 2019. Relational knowledge distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 3967--3976

    2019

  35. [43]

    Ren, D.; Zuo, W.; Hu, Q.; Zhu, P.; and Meng, D. 2019. Progressive image deraining networks: A better and simpler baseline. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 3937--3946

    2019

  36. [44]

    Sanh, V.; Debut, L.; Chaumond, J.; and Wolf, T. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108

    2019 arXiv

  37. [45]

    Sun, L.; Dong, J.; Tang, J.; and Pan, J. 2023. Spatially-adaptive feature modulation for efficient image super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 13190--13199

    2023

  38. [46]

    Tian, Y.; Krishnan, D.; and Isola, P. 2019. Contrastive representation distillation. arXiv preprint arXiv:1910.10699

    2019 arXiv

  39. [47]

    Timofte, R.; Agustsson, E.; Van Gool, L.; Yang, M.-H.; and Zhang, L. 2017. Ntire 2017 challenge on single image super-resolution: Methods and results. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 114--125

    2017

  40. [48]

    Touvron, H.; Cord, M.; Douze, M.; Massa, F.; Sablayrolles, A.; and J \'e gou, H. 2021. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, 10347--10357. PMLR

    2021

  41. [49]

    Wang, L.; Dong, X.; Wang, Y.; Ying, X.; Lin, Z.; An, W.; and Guo, Y. 2021 a . Exploring sparsity in image super-resolution for efficient inference. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4917--4926

    2021

  42. [50]

    Wang, L.; Huang, J.; Li, Y.; Xu, K.; Yang, Z.; and Yu, D. 2021 b . Improving weakly supervised visual grounding by contrastive knowledge distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14090--14100

    2021

  43. [51]

    Wang, X.; Xie, L.; Dong, C.; and Shan, Y. 2021 c . Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In Proceedings of the IEEE/CVF international conference on computer vision, 1905--1914

    2021

  44. [52]

    Wang, Y.; Lin, S.; Qu, Y.; Wu, H.; Zhang, Z.; Xie, Y.; and Yao, A. 2021 d . Towards compact single image super-resolution via contrastive self-distillation. arXiv preprint arXiv:2105.11683

    2021 arXiv

  45. [53]

    C.; Sheikh, H

    Wang, Z.; Bovik, A. C.; Sheikh, H. R.; and Simoncelli, E. P. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4): 600--612

    2004

  46. [54]

    Wang, Z.; Cun, X.; Bao, J.; Zhou, W.; Liu, J.; and Li, H. 2022. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 17683--17693

    2022

  47. [55]

    Wu, G.; Jiang, J.; Jiang, K.; and Liu, X. 2024. Learning from history: Task-agnostic model contrastive learning for image restoration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 5976--5984

    2024

  48. [56]

    Xu, G.; Liu, Z.; Li, X.; and Loy, C. C. 2020. Knowledge distillation meets self-supervision. In European conference on computer vision, 588--604. Springer

    2020

  49. [57]

    Yang, C.; An, Z.; Zhou, H.; Zhuang, F.; Xu, Y.; and Zhang, Q. 2023. Online knowledge distillation via mutual contrastive learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8): 10212--10227

    2023

  50. [58]

    T.; Feng, J.; Liu, J.; Guo, Z.; and Yan, S

    Yang, W.; Tan, R. T.; Feng, J.; Liu, J.; Guo, Z.; and Yan, S. 2017. Deep joint rain detection and removal from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1357--1366

    2017

  51. [59]

    W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F

    Zamir, S. W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F. S.; and Yang, M.-H. 2022. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5728--5739

    2022

  52. [60]

    W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F

    Zamir, S. W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F. S.; Yang, M.-H.; and Shao, L. 2021. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 14821--14831

    2021

  53. [61]

    Zhang, H.; and Patel, V. M. 2018. Density-aware single image de-raining using a multi-stream dense network. In Proceedings of the IEEE conference on computer vision and pattern recognition, 695--704

    2018

  54. [62]

    Zhang, H.; Sindagi, V.; and Patel, V. M. 2019. Image de-raining using a conditional generative adversarial network. IEEE transactions on circuits and systems for video technology, 30(11): 3943--3956

    2019

  55. [63]

    Zhang, K.; Zuo, W.; Chen, Y.; Meng, D.; and Zhang, L. 2017. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26(7): 3142--3155

    2017

  56. [64]

    Zhang, Q.; Liu, X.; Li, W.; Chen, H.; Liu, J.; Hu, J.; Xiong, Z.; Yuan, C.; and Wang, Y. 2024. Distilling Semantic Priors from SAM to Efficient Image Restoration Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 25409--25419

    2024

  57. [65]

    Zhang, X.; Zeng, H.; Guo, S.; and Zhang, L. 2022. Efficient long-range attention network for image super-resolution. In European conference on computer vision, 649--667. Springer

    2022

  58. [66]

    Zhang, Y.; Li, K.; Li, K.; Wang, L.; Zhong, B.; and Fu, Y. 2018. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European conference on computer vision (ECCV), 286--301

    2018

  59. [67]

    Zhang, Y.; Li, W.; Li, S.; Hu, J.; Chen, H.; Wang, H.; Tu, Z.; Wang, W.; Jing, B.; and Wang, Y. 2023. Data upcycling knowledge distillation for image super-resolution. arXiv preprint arXiv:2309.14162

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.