Pith. sign in

REVIEW 3 major objections 5 minor 3 cited by

Diffusion-based Perceptual Neural Video Compression with Temporal Diffusion Information Reuse

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims a diffusion-based video codec can beat GAN-based and traditional codecs on perceptual quality while cutting diffusion inference time by 47% through temporal reuse.

desk verdict First solid diffusion-in-the-loop video codec with genuinely useful TDIR and QPP, but the 47% speedup claim rests on a long-GOP test that is missing. read the letter →

arxiv 2501.13528 v1 pith:UW33W77T submitted 2025-01-23 cs.CV cs.LGeess.IV

Wenzhuo Ma , Zhenzhong Chen This is my paper · ORCID
classification cs.CVcs.LGeess.IV
keywords neuralvideocompressiondiffusionmodelsperceptualtemporalinformationreusequantizationparameterpromptingvariablebitrateDISTSconditionalcoding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

DiffVC is a neural video codec that bolts a pre-trained foundational diffusion model onto the standard conditional-coding pipeline, so the decoder generates perceptually rich frames instead of just minimizing pixel error. To make the slow iterative diffusion process practical, the paper introduces Temporal Diffusion Information Reuse (TDIR): for each P-frame it skips the expensive noise-estimation network in the first half of the diffusion schedule and recycles the cleaned-up latent states stored from the previous frame. A separate Quantization Parameter-based Prompting (QPP) mechanism feeds the codec's quantization setting into the diffusion model through cross-attention, letting one trained model operate across several bitrates. The reported experiments claim best-reported DISTS scores on all three test sets and visual reconstructions with more texture detail than GAN-based or traditional codecs at similar bitrates.

What carries the argument

The load-bearing mechanism is the Temporal Diffusion Information Reuse (TDIR) strategy. During independent diffusion at step $n$, the U-Net predicts noise $\epsilon^n_\theta$, from which the model computes the predicted noise-free latent $\ddot{y}^n_t$; these $\ddot{y}^n_t$ values are stored in a Diffusion Buffer. For the next P-frame, the first $DS-D$ steps retrieve $\ddot{y}^n_{t-1}$ from the buffer and perform only posterior sampling to move to timestep $n-1$, bypassing the U-Net entirely; the remaining $D$ steps run independent diffusion to restore frame-specific detail. The paper couples this with the QP-based Prompting mechanism, which channel-averages the encoder/decoder quantization ratio, encodes it with a pre-trained text encoder, and injects the resulting tokens into the diffusion U-Net's cross-attention layers, so the same model adapts to different bitrates.

What would settle it

Encode a long sequence with thousands of consecutive P-frames (a very large intra-period) and track DISTS, LPIPS, and the cosine similarity of reused latents as functions of P-frame distance; if perceptual error grows steadily with distance from the last I-frame or similarity drops well below the reported 0.86, the TDIR reuse assumption fails for real streams.

Watch

Extended reading notes

Core claim

The central claim is that the iterative denoising cost of a diffusion-based decoder need not be paid independently for every video frame. Because consecutive frames are highly correlated, the predicted noise-free latents at corresponding diffusion timesteps are similar across adjacent frames; DiffVC stores these latents in a diffusion buffer during one frame's decoding and reuses them as the starting estimate for the next frame's first half of the denoising steps. This halves the number of U-Net evaluations and reduces per-frame decoding time by 47% while requiring only about 1.96% more bitrate to reach the same average perceptual quality. The same framework, with quantization-parameter prompting, keeps a single diffusion model effective across four bitrate points, and the measured DISTS improvements over prior codecs are claimed on HEVC, MCL-JCV, and UVG.

Load-bearing premise

The 47% speedup rests on the assumption that two neighboring frames at the same diffusion stage have nearly identical cleaned-up internal states, so reusing one frame's state for the next introduces little error; the experiments only demonstrate this over short runs of consecutive P-frames, leaving long video streams untested.

Editorial extensions

If this is right

  • A single diffusion-based codec can serve as a perceptual codec with variable bitrate support, removing the need to train one diffusion model per rate point.
  • Diffusion decoding cost in video no longer scales as the full number of diffusion steps per frame, so real-time perceptual decoding becomes more plausible.
  • The TDIR design keeps the total number of sampling steps unchanged, so its speedup composes with other diffusion accelerators such as faster samplers or better starting latents.
  • Because the reuse buffer stores only predicted noise-free latents, the same strategy transfers to other diffusion-based video tasks such as generation, not just compression.
  • The perceptual gains are concentrated in structure-texture metrics like DISTS, indicating the codec is trading pixel fidelity for human-perceptual quality at low bitrates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test the paper does not run is a long-duration stream: with thousands of consecutive P-frames, the reuse buffer could accumulate drift, so an adaptive policy that shortens the reuse phase as frame distance grows would likely be needed in practice.
  • The QPP mechanism encodes only the global quantization ratio, so a promising extension is a content-adaptive prompt that also describes local texture or motion, which could sharpen reconstructions scene by scene.
  • Since independent diffusion steps dominate decoding time, the 47% figure is a lower bound on speedup for longer videos where many frames benefit from reuse; conversely, frequent scene cuts or I-frame refreshes would reduce the achievable speedup.
  • The reported DISTS and LPIPS results suggest that perceptual quality metrics, not PSNR, are the right yardstick for this class of codec; a user study comparing DiffVC and GAN codecs on perceived realism would test whether the metric gains translate to human preference.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DiffVC, a diffusion-based perceptual neural video codec that combines a DCVC-DC-style conditional coding backbone with a pre-trained Stable Diffusion V2.1 generative decoder. Motion and contextual modules compress motion vectors and latent representations, while a ControlNet-guided U-Net denoises a noised version of the reconstructed latent, conditioned on temporal context from the previous decoded frame. To reduce the latency of iterative diffusion inference, the paper introduces Temporal Diffusion Information Reuse (TDIR), which buffers the predicted noise-free latents of the previous frame and reuses them for the first half of the diffusion steps of the current P-frame. The paper also proposes Quantization Parameter-based Prompting (QPP), which feeds the ratio of encoder/decoder quantization parameters through a CLIP text encoder to modulate U-Net cross-attention and thus support variable bitrates with a single model. Experiments are conducted on HEVC, MCL-JCV, and UVG using the first 96 frames with an intra-period of 32, with BD-rates anchored to VTM-17.0. The paper reports state-of-the-art DISTS performance on all three datasets, a 47% reduction in P-frame decoding time, and a 1.96% average perceptual BD-rate loss for TDIR, with ablations isolating the contributions of the diffusion model, TDIR, and QPP.

Significance. If the reported results hold, DiffVC is a meaningful advance: it is one of the first diffusion-based video codecs that integrates a foundational diffusion model into a conditional coding paradigm, supports variable bitrates with a single model, and explicitly addresses the inference-latency bottleneck of diffusion-based decoding. The paper's strengths include a clean and well-motivated architecture, a multi-stage training recipe, and ablations that separately quantify the effect of the diffusion model, TDIR, and QPP. The per-mode speed measurement in Table 6 and the cosine-similarity analysis in Fig. 8 are useful supporting evidence. The main risks are that the central speed/quality trade-off claim for TDIR is validated only over short P-frame runs of at most 31 frames, that the key hyperparameter D is selected on the test benchmark, and that the headline BD-rate numbers are point estimates without error bars or per-sequence variability. These issues are fixable with additional experiments and analysis and do not, on the current evidence, invalidate the overall approach, but they are load-bearing for the paper's main claims.

major comments (3)
  1. [Sec. 4.1, Sec. 4.3.2, Fig. 9b] The TDIR speedup claim is established only for very short P-frame runs. All main results use the first 96 frames with intra-period 32 (Sec. 4.1), so the longest P-frame run after an I-frame is 31 frames. The paper's own Fig. 9b states that when D is too small, 'excessive reuse of prior frame diffusion information leads to significant error accumulation,' which confirms that reuse errors are a real mechanism rather than a hypothetical failure mode. With D=25 and DS=50, each P-frame chains 25 reuse steps plus 25 independent steps, so a 1000-frame all-P sequence would involve roughly 25,000 reuse operations; Fig. 8's adjacent-frame cosine similarity does not establish whether the error saturates or grows over a long run. The authors should report experiments with substantially longer P-runs (e.g., at least 96 consecutive P-frames without intra refresh, and preferably longer), including the perceptual metrics as a function of frame index for DiffVC versus the no-TDIR ablation (Method D in Table 5). Without such evidence, the 47% speedup with only 1.96% perceptual loss cannot be claimed for realistic GOP lengths.
  2. [Sec. 4.3.4, Fig. 9b] The value D=25 is selected by inspecting perceptual BD-rate on HEVC Class C, which is the same test benchmark used for the headline results in Table 3 and Table 5. No separate validation set or cross-validation procedure is described. Because D is a hyperparameter of the proposed method, selecting it on the test set and then reporting results on that test set inflates the reported performance. Please either select D on a held-out set (e.g., a Vimeo-90k validation split or a different dataset) or report the D-sweep results on all three test datasets with per-sequence variability so that the sensitivity and the degree of selection bias are transparent. This is particularly important because the TDIR comparison in Table 5 reports a 1.96% average difference that may be within the noise of the BD-rate estimation.
  3. [Table 3, Table 5] The paper reports BD-rate and BD-metric values as point estimates without confidence intervals, per-sequence breakdowns, or statistical significance tests. Several entries in Table 3 are N/A or non-monotonic (for example, DiffVC's PSNR/MS-SSIM/NIQE rows on HEVC, and PLVC's LPIPS/NIQE rows on MCL-JCV), and Table 5's decoding-time values are also single measurements. The 'state-of-the-art across all datasets for the DISTS metric' claim therefore rests on a single number with no measure of dispersion. Please report per-sequence Bjøntegaard deltas (or bootstrap/standard-error intervals) for the main comparisons, and explicitly discuss how the N/A entries affect the fairness and completeness of the comparisons. At minimum, the key TDIR comparison (Method D versus DiffVC in Table 5) needs an indication of variability across sequences, since the claimed 1.96% loss may be within run-to-run or sequence-to-sequence noise.
minor comments (5)
  1. [Eq. (6), Fig. 3] It is not specified how the scalar CAP(qenc/qdec) is fed to the pre-trained CLIP Text Encoder. CLIP's text encoder normally operates on tokenized text, not raw scalars; the authors should clarify whether the ratio is converted to a string, discretized and embedded, or passed through an additional learned projection. This detail is needed for reproducibility.
  2. [Throughout] There are repeated typographical issues, including 'roboustly' in the abstract/conclusion, 'Tainable' in Table 2, and inconsistent spacing in 'Di ffusion' and 'Di ffVC'. These should be corrected in a final polish pass.
  3. [Sec. 3.2, Eq. (1)] The notation n is described as ranging over [0, N], but the standard DDPM formulation uses n=1,...,N and the definition of alpha_bar_n for n=0 is ambiguous. Please clarify the ranges and the definition of alpha_bar_0.
  4. [Sec. 4.3.4, Fig. 9b] The sentence 'when D < 15, perceptual performance drops sharply' is followed by the explanation that excessive reuse leads to error accumulation; this is an important admission but it is stated only in the discussion. It would be helpful to show the per-frame error accumulation curve (e.g., frame index versus LPIPS or DISTS) for D=25 over the full test window to directly demonstrate that the error has reached a steady state by frame 31.
  5. [Table 5] The 'Average' column in Table 5 should state explicitly how the average is computed when the NIQE BD-rate is unavailable (e.g., 'average over the remaining metrics'), and the decoding-time comparison should clarify whether the reported time includes all P-frame decoding operations (contextual decoding plus diffusion) or only the diffusion module.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DiffVC's perceptual quality gains, TDIR speedup, and QPP variable-bitrate support are all empirically measured against external baselines and are not derived from their own assumptions.

full rationale

The paper's central claims are empirical rather than derivational. DiffVC is evaluated against traditional codecs (JM/HM/VTM), distortion-oriented learned codecs (DCVC series), GAN-based codecs (DVC-P, PLVC), and another diffusion codec (EVC-PDM) on HEVC, MCL-JCV, and UVG, with BD-rates anchored to VTM-17.0 (Table 3). These are externally falsifiable benchmarks, not quantities reconstructed from the method's own definitions. The TDIR speedup claim is supported by an ablation (Table 5, Method D vs. DiffVC) and by measured cosine similarity between adjacent-frame predicted noise-free latents (Fig. 8); the reuse operation is a concrete algorithmic step, not a parameter fitted to the target metric. The QPP mechanism is justified by the ablation in Section 4.3.3 (Method C vs. DiffVC), and the prompt-to-bitrate correspondence is learned through mixed-bitrate fine-tuning of the diffusion model, not assumed by construction. No load-bearing self-citation appears: the authors' prior works (refs [34], [39]) are related-work citations, and the core modules are adapted from externally published DCVC-DC and Stable Diffusion. No uniqueness theorem is imported from the authors, and no known result is renamed. The most substantive concerns are benchmark-integrity and generalization, not circularity: D=25 and DS=50 were chosen with the test benchmark in view, and the TDIR long-GOP behavior is untested because experiments use only 96 frames with intra-period 32 while Fig. 9b itself admits that excessive reuse causes significant error accumulation. These concerns affect external validity but do not make any prediction equivalent to an input or fit.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper rests on two main assumptions: the temporal similarity of diffusion predictions (TDIR) and the informativeness of a quantization-parameter ratio as a prompt (QPP). Both are empirically motivated, but neither is derived. The diffusion process itself is standard DDPM. No new physical entities are introduced.

free parameters (5)
  • Independent diffusion steps D = 25
    Selected via ablation on HEVC Class C (Fig. 9b); the paper reports performance remains stable for D>=25 and drops sharply for D<15. This value is then used for all final evaluations, including on HEVC, making it a hand-tuned hyperparameter rather than a derived quantity.
  • Total diffusion steps DS = 50
    Chosen by hand as a compromise between quality and speed; Fig. 9a shows perceptual quality degrades as DS decreases, but no formal derivation is given.
  • Perceptual loss weight wp = 0.025
    Set manually in the training strategy (Section 3.4); no sensitivity analysis is provided.
  • Bitrate balancing lambdas = [16, 48, 128, 384]
    Four discrete rate points chosen by hand to enable variable bitrate training; these are not optimized per test video.
  • Periodic frame weights wt = (0.5, 1.2, 0.5, 0.9)
    Follows DCVC-DC [4] to mitigate error propagation; not analyzed in this paper.
assumptions (5)
  • standard math DDPM noise schedule and posterior sampling (Eq. 1-5) describe the diffusion process.
    The method relies on the standard DDPM formulation.
  • domain assumption Adjacent video frames have sufficiently similar predicted noise-free latents at corresponding diffusion timesteps.
    This is the core assumption behind TDIR. The paper provides empirical evidence via cosine similarity in Fig. 8, but it is a heuristic, not a theorem.
  • domain assumption The ratio qenc/qdec after channel averaging correlates with bitrate and distortion level.
    The QPP mechanism assumes this scalar ratio is a useful conditioning signal. The paper shows a curve for one sequence in Fig. 3 and an ablation, but no theoretical justification.
  • domain assumption CLIP text encoder can embed a scalar ratio into tokens that the U-Net can use for cross-attention.
    The paper treats Stable Diffusion's text conditioning as a general-purpose prompt interface; this is a practical assumption validated only by the QPP ablation.
  • domain assumption Pre-trained Stable Diffusion V2.1 provides a strong image prior that improves perceptual quality when fine-tuned on compression latents.
    The whole method rests on this prior being transferable to the video latent distribution; the ablation of Method A vs B tests this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion-based Perceptual Neural Video Compression with Temporal Diffusion Information Reuse." pith.science (2026). https://pith.science/paper/UW33W77T

@misc{pith2026250113528,
  author       = {Pith},
  title        = {Pith review of: Diffusion-based Perceptual Neural Video Compression with Temporal Diffusion Information Reuse},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UW33W77T}},
  note         = {Machine review of arXiv:2501.13528}
}
read the original abstract

Recently, foundational diffusion models have attracted considerable attention in image compression tasks, whereas their application to video compression remains largely unexplored. In this article, we introduce DiffVC, a diffusion-based perceptual neural video compression framework that effectively integrates foundational diffusion model with the video conditional coding paradigm. This framework uses temporal context from previously decoded frame and the reconstructed latent representation of the current frame to guide the diffusion model in generating high-quality results. To accelerate the iterative inference process of diffusion model, we propose the Temporal Diffusion Information Reuse (TDIR) strategy, which significantly enhances inference efficiency with minimal performance loss by reusing the diffusion information from previous frames. Additionally, to address the challenges posed by distortion differences across various bitrates, we propose the Quantization Parameter-based Prompting (QPP) mechanism, which utilizes quantization parameters as prompts fed into the foundational diffusion model to explicitly modulate intermediate features, thereby enabling a robust variable bitrate diffusion-based neural compression framework. Experimental results demonstrate that our proposed solution delivers excellent performance in both perception metrics and visual quality.

Figures

Figures reproduced from arXiv: 2501.13528 by the authors.

Figure 1
Figure 1. The framework of DiffVC. DiffVC consists of three main components: Motion Modules, Contextual Modules, and Diffusion Modules. The Motion Modules (green) manage motion vector estimation and compression. The Contextual Modules (red) focus on extracting temporal context and compressing conditional residues. Finally, the Diffusion Modules (blue) apply multiple diffusion steps to generate high perceptual-quality reconstr… view at source ↗
Figure 2
Figure 2. Temporal Diffusion Information Reuse Strategy. The left panel illustrates the inference process of TDIR, where the vertical axis represents video frames and the horizontal axis represents diffusion timesteps. The first P frame undergoes independent diffusion for DS steps, while subsequent P frames reuse diffuse for DS − D steps before undergoing independent diffusion for the remaining D steps. The right panel provid… view at source ↗
Figure 3
Figure 3. QP-based Prompting Mechanism. The ratio of the quantization parameters, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The rate-perception/distortion curves of our proposed DiffVC and other video compression methods on the HEVC dataset. Solid lines with dots represent traditional codecs, solid lines with triangles denote distortion-oriented neural video compression methods, dashed line…
Figure 5
Figure 5. Figure 5: The rate-perception/distortion curves of our proposed DiffVC and other video compression methods on the MCL-JCV dataset. Solid lines with dots represent traditional codecs, solid lines with triangles denote distortion-oriented neural video compression methods, dashed l…
Figure 6
Figure 6. Figure 6: The rate-perception/distortion curves of our proposed DiffVC and other video compression methods on the UVG dataset. Solid lines with dots represent traditional codecs, solid lines with triangles denote distortion-oriented neural video compression methods, dashed lines…
Figure 7
Figure 7. Figure 7: Visual results of traditional codec VTM-17.0, distortion-oriented codec DCVC-FM, GAN-based codec PLVC and [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: The average cosine similarity of predicted noise-free latent [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Influence of DS and D. Subfigure (a) illustrates the variation in perceptual BD-rate across different total diffusion steps (DS ), with DiffVC as the anchor. Subfigure (b) shows the changes in perceptual BD-rate under different independent diffusion steps (D) while mai…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative Video Compression with Adaptive Score Distillation

    eess.IV 2026-07 conditional novelty 6.0 of 10

    A from-scratch, pixel-space video diffusion codec with a ground-truth-aligned gate on DMD gradients achieves one-step decoding and ~62–71% bitrate savings at matched LPIPS/FID over GLVC.

  2. DiffVC-OSD: One-Step Diffusion-based Perceptual Neural Video Compression Framework

    eess.IV 2025-08 conditional novelty 6.0 of 10

    DiffVC-OSD compresses video with a one-step diffusion model, a temporal context adapter, and end-to-end finetuning, reporting top perceptual quality on three test sets with about 20x faster decoding than multi-step di...

  3. Semantic-Aware Adaptive Video Streaming Using Latent Diffusion Models for Wireless Networks

    cs.MM 2025-02 reject novelty 4.0 of 10

    The paper proposes LD-ABS, an adaptive bitrate streaming framework that compresses I-frames with a latent diffusion model and reconstructs P and B frames at the receiver, claiming better QoE than existing ABR algorithms.

Reference graph

Works this paper leans on

60 extracted references · 43 canonical work pages · cited by 3 Pith papers

  1. [1]

    Sullivan, Gisle Bjøntegaard, and Ajay Luthra

    Thomas Wiegand, Gary J. Sullivan, Gisle Bjøntegaard, and Ajay Luthra. Overview of the H.264 /A VC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology, 13(7):560–576, 2003

    work page 2003

  2. [2]

    Sullivan, Jens-Rainer Ohm, Woojin Han, and Thomas Wiegand

    Gary J. Sullivan, Jens-Rainer Ohm, Woojin Han, and Thomas Wiegand. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology , 22(12):1649–1668, 2012

    2012

  3. [3]

    Sullivan, and Jens-Rainer Ohm

    Benjamin Bross, Ye-Kui Wang, Yan Ye, Shan Liu, Jianle Chen, Gary J. Sullivan, and Jens-Rainer Ohm. Overview of the versatile video coding (VVC) standard and its appli- cations. IEEE Transactions on Circuits and Systems for Video Technology, 31(10):3736–3764, 2021

    work page 2021

  4. [4]

    Neural video compression with diverse contexts

    Jiahao Li, Bin Li, and Yan Lu. Neural video compression with diverse contexts. In IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 22616–22626, 2023

    2023

  5. [5]

    Neural video compression with feature modulation

    Jiahao Li, Bin Li, and Yan Lu. Neural video compression with feature modulation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26099– 26108, 2024

    2024

  6. [6]

    Rethinking lossy com- pression: The rate-distortion-perception tradeoff

    Yochai Blau and Tomer Michaeli. Rethinking lossy com- pression: The rate-distortion-perception tradeoff. In Pro- ceedings of International Conference on Machine Learn- ing, volume 97, pages 675–685, 2019

    work page 2019

  7. [7]

    IBVC: interpolation-driven b-frame video compres- sion

    Chenming Xu, Meiqin Liu, Chao Yao, Weisi Lin, and Yao Zhao. IBVC: interpolation-driven b-frame video compres- sion. Pattern Recognition, 153:110465, 2024

    work page 2024

  8. [8]

    Efros, Eli Shecht- man, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 586–595, 2018

    2018

Show all 60 references
  1. [9]

    Neural video compression using gans for detail synthesis and propaga- tion

    Fabian Mentzer, Eirikur Agustsson, Johannes Ballé, David Minnen, Nick Johnston, and George Toderici. Neural video compression using gans for detail synthesis and propaga- tion. In European Conference on Computer Vision, volume 13686, pages 562–578, 2022

    2022

  2. [10]

    DVC-P: deep video compression with perceptual optimizations

    Saiping Zhang, Marta Mrak, Luis Herranz, Marc Górriz Blanch, Shuai Wan, and Fuzheng Yang. DVC-P: deep video compression with perceptual optimizations. In Inter- national Conference on Visual Communications and Image Processing, pages 1–5. IEEE, 2021

    2021

  3. [11]

    Percep- tual learned video compression with recurrent conditional GAN

    Ren Yang, Radu Timofte, and Luc Van Gool. Percep- tual learned video compression with recurrent conditional GAN. In Proceedings of International Joint Conference on Artificial Intelligence, pages 1537–1544, 2022

    2022

  4. [12]

    High visual-fidelity learned video compression

    Meng Li, Yibo Shi, Jing Wang, and Yunqi Huang. High visual-fidelity learned video compression. In Proceedings of ACM International Conference on Multimedia, pages 8057–8066, 2023

    2023

  5. [13]

    CGVC-T: contextual generative video compression with transformers

    Pengli Du, Ying Liu, and Nam Ling. CGVC-T: contextual generative video compression with transformers. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 14(2):209–223, 2024

    2024

  6. [14]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems , pages 2672–2680, 2014

    2014

  7. [15]

    High-resolution im- age synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution im- age synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10674–10685, 2022

    2022

  8. [16]

    Extreme genera- tive image compression by learning text embedding from diffusion models

    Zhihong Pan, Xin Zhou, and Hao Tian. Extreme genera- tive image compression by learning text embedding from diffusion models. arXiv, abs/2211.07793, 2022

    2022 arXiv

  9. [17]

    Lossy image compres- sion with conditional di ffusion models

    Ruihan Yang and Stephan Mandt. Lossy image compres- sion with conditional di ffusion models. In Advances in Neural Information Processing Systems, 2023

    2023

  10. [18]

    Text + sketch: Image compression at ultra low rates

    Eric Lei, Yigit Berkay Uslu, Hamed Hassani, and Shirin Saeedi Bidokhti. Text + sketch: Image compression at ultra low rates. arXiv, abs/2307.01944, 2023

    2023 arXiv

  11. [19]

    Muckley, Jakob Verbeek, and Stéphane Lathuilière

    Marlène Careil, Matthew J. Muckley, Jakob Verbeek, and Stéphane Lathuilière. Towards image compression with perfect realism at ultra-low bitrates. In International Con- ference on Learning Representations, 2024

    2024

  12. [20]

    Lossy image compression with foundation diffusion models

    Lucas Relic, Roberto Azevedo, Markus Gross, and Christo- pher Schroers. Lossy image compression with foundation diffusion models. arXiv, abs/2404.08580, 2024

    2024 arXiv

  13. [21]

    Di ffusion-based extreme image com- pression with compressed feature initialization

    Zhiyuan Li, Yanhui Zhou, Hao Wei, Chenyang Ge, and Ajmal Mian. Di ffusion-based extreme image com- pression with compressed feature initialization. arXiv, abs/2410.02640, 2024

    2024 arXiv

  14. [22]

    Simoncelli

    Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. Image quality assessment: Unifying structure and texture similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(5):2567–2581, 2022

    2022

  15. [23]

    Simoncelli

    Johannes Ballé, Valero Laparra, and Eero P. Simoncelli. End-to-end optimized image compression. InInternational Conference on Learning Representations, 2017

    2017

  16. [24]

    Variational image compression14 with a scale hyperprior

    Johannes Ballé, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compression14 with a scale hyperprior. In International Conference on Learning Representations, 2018

    2018

  17. [25]

    Joint autoregressive and hierarchical priors for learned image compression

    David Minnen, Johannes Ballé, and George Toderici. Joint autoregressive and hierarchical priors for learned image compression. In Advances in Neural Information Process- ing Systems, pages 10794–10803, 2018

    2018

  18. [26]

    Learned image compression with discretized gaussian mixture likelihoods and attention modules

    Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7936–7945, 2020

    2020

  19. [27]

    Checkerboard context model for e fficient learned image compression

    Dailan He, Yaoyan Zheng, Baocheng Sun, Yan Wang, and Hongwei Qin. Checkerboard context model for e fficient learned image compression. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14771– 14780, 2021

    2021

  20. [28]

    ELIC: efficient learned image com- pression with unevenly grouped space-channel contextual adaptive coding

    Dailan He, Ziming Yang, Weikun Peng, Rui Ma, Hongwei Qin, and Yan Wang. ELIC: efficient learned image com- pression with unevenly grouped space-channel contextual adaptive coding. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5708–5717, 2022

    2022

  21. [29]

    Learned image compression with mixed transformer-cnn architectures

    Jinming Liu, Heming Sun, and Jiro Katto. Learned image compression with mixed transformer-cnn architectures. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14388–14397, 2023

    2023

  22. [30]

    A universal optimization framework for learning-based image codec

    Jing Zhao, Bin Li, Jiahao Li, Ruiqin Xiong, and Yan Lu. A universal optimization framework for learning-based image codec. ACM Transactions on Multimedia Comput- ing, Communications, and Applications, 20(1):16:1–16:19, 2024

    2024

  23. [31]

    DVC: an end-to-end deep video compression framework

    Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chun- lei Cai, and Zhiyong Gao. DVC: an end-to-end deep video compression framework. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11006– 11015, 2019

    2019

  24. [32]

    FVC: A new frame- work towards deep video compression in feature space

    Zhihao Hu, Guo Lu, and Dong Xu. FVC: A new frame- work towards deep video compression in feature space. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1502–1511, 2021

    2021

  25. [33]

    An end-to-end learning framework for video compression

    Guo Lu, Xiaoyun Zhang, Wanli Ouyang, Li Chen, Zhiyong Gao, and Dong Xu. An end-to-end learning framework for video compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(10):3292–3308, 2021

    2021

  26. [34]

    Learned video compression via heterogeneous deformable compensation network

    Huairui Wang, Zhenzhong Chen, and Chang Wen Chen. Learned video compression via heterogeneous deformable compensation network. IEEE Transactions on Multimedia, 26:1855–1866, 2024

    2024

  27. [35]

    Learned video compression with adap- tive temporal prior and decoded motion-aided quality en- hancement

    Jiayu Yang, Chunhui Yang, Fei Xiong, Yongqi Zhai, and Ronggang Wang. Learned video compression with adap- tive temporal prior and decoded motion-aided quality en- hancement. ACM Transactions on Multimedia Computing, Communications, and Applications, 20(8):238:1–238:21, 2024

    2024

  28. [36]

    Tomczak, and Taco Cohen

    AmirHossein Habibian, Ties van Rozendaal, Jakub M. Tomczak, and Taco Cohen. Video compression with rate- distortion autoencoders. In IEEE/CVF International Con- ference on Computer Vision, pages 7032–7041, 2019

    2019

  29. [37]

    High-quality single- model deep video compression with frame-conv3d and multi-frame differential modulation

    Wenyu Sun, Chen Tang, Weigui Li, Zhuqing Yuan, Huazhong Yang, and Yongpan Liu. High-quality single- model deep video compression with frame-conv3d and multi-frame differential modulation. In European Confer- ence on Computer Vision, volume 12375, pages 239–254, 2020

    2020

  30. [38]

    Adversarial distortion for learned video compression

    Vijay Veerabadran, Reza Pourreza, AmirHossein Habib- ian, and Taco Cohen. Adversarial distortion for learned video compression. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 640–644, 2020

    2020

  31. [39]

    Exploring long- and short-range temporal information for learned video com- pression

    Huairui Wang and Zhenzhong Chen. Exploring long- and short-range temporal information for learned video com- pression. IEEE Transactions on Image Processing , 33: 780–792, 2024

    2024

  32. [40]

    Deep contextual video com- pression

    Jiahao Li, Bin Li, and Yan Lu. Deep contextual video com- pression. In Advances in Neural Information Processing Systems, pages 18114–18125, 2021

    2021

  33. [41]

    Hybrid spatial-temporal en- tropy modelling for neural video compression

    Jiahao Li, Bin Li, and Yan Lu. Hybrid spatial-temporal en- tropy modelling for neural video compression. In ACM In- ternational Conference on Multimedia, pages 1503–1511, 2022

    2022

  34. [42]

    Temporal context mining for learned video compres- sion

    Xihua Sheng, Jiahao Li, Bin Li, Li Li, Dong Liu, and Yan Lu. Temporal context mining for learned video compres- sion. IEEE Transactions on Multimedia, 25:7311–7322, 2023

    2023

  35. [43]

    Extreme video compression with pre- trained diffusion models

    Bohan Li, Yiming Liu, Xueyan Niu, Bo Bai, Lei Deng, and Deniz Gündüz. Extreme video compression with pre- trained diffusion models. arXiv, abs/2402.08934, 2024

    2024 arXiv

  36. [44]

    I 2vc: A unified framework for intra- & inter-frame video compression

    Meiqin Liu, Chenming Xu, Yukai Gu, Chao Yao, and Yao Zhao. I 2vc: A unified framework for intra- & inter-frame video compression. arXiv, abs/2405.14336, 2024

    2024 arXiv

  37. [45]

    Anurag Ranjan and Michael J. Black. Optical flow esti- mation using a spatial pyramid network. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2720–2729, 2017

    2017

  38. [46]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In IEEE/CVF International Conference on Computer Vision, pages 3813–3824, 2023

    2023

  39. [47]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing System, 2020

    2020

  40. [48]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In International Conference on Learning Representations, 2021

    2021

  41. [49]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of In...

    2021

  42. [50]

    Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T. Freeman. Video enhancement with task- oriented flow. International Journal of Computer Vision, 127(8):1106–1125, 2019.15

    2019

  43. [51]

    UVG dataset: 50/120fps 4k sequences for video codec analysis and development

    Alexandre Mercat, Marko Viitanen, and Jarno Vanne. UVG dataset: 50/120fps 4k sequences for video codec analysis and development. In Proceedings of ACM Multimedia Systems Conference, pages 297–302, 2020

    2020

  44. [52]

    Haiqiang Wang, Weihao Gan, Sudeng Hu, Joe Yuchieh Lin, Lina Jin, Longguang Song, Ping Wang, Ioannis Kat- savounidis, Anne Aaron, and C.-C. Jay Kuo. MCL-JCV: A jnd-based H.264/A VC video quality assessment dataset. In IEEE International Conference on Image Processing, pages 150...

    2016

  45. [53]

    High-fidelity generative image com- pression

    Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. High-fidelity generative image com- pression. In Advances in Neural Information Processing Systems, 2020

    2020

  46. [54]

    JVET. JM-19.0. http://iphome.hhi.de/suehring/,

  47. [55]

    HM-16.25

    JVET. HM-16.25. https://vcgit.hhi.fraunhofer. de/jvet/HM/, 2022. Accessed: 2022-11-02

    2022

  48. [56]

    VTM-17.0

    JVET. VTM-17.0. https://vcgit.hhi.fraunhofer. de/jvet/VVCSoftware_VTM/, 2022. Accessed: 2022- 11-02

    2022

  49. [57]

    Mul- tiscale structural similarity for image quality assessment

    Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Mul- tiscale structural similarity for image quality assessment. In Asilomar Conference on Signals, Systems & Computers, volume 2, pages 1398–1402, 2003

    2003

  50. [58]

    Sutherland, Michael Arbel, and Arthur Gretton

    Mikolaj Binkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD gans. In Interna- tional Conference on Learning Representations, 2018

    2018

  51. [59]

    completely blind

    Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. Making a "completely blind" image quality analyzer.IEEE Signal Processing Letters, 20(3):209–212, 2013. 16

    2013

  52. [2022]

    Accessed: 2022-03-02

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.