Pith. sign in

REVIEW 4 major objections 5 minor 57 references

Continuous Video Process: Modeling Videos as Continuous Multi-Dimensional Processes for Video Prediction

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The Continuous Video Process treats video as a continuous interpolation between frames and reports state-of-the-art prediction with 75% fewer sampling steps.

desk verdict This is a postprint of the authors' own CVPR 2024 paper, and the variational bound derivation has a load-bearing inconsistency; the empirical results are strong but the theory doesn't hold as written. read the letter →

arxiv 2412.04929 v2 pith:4JVIGFYD submitted 2024-12-06 cs.CV cs.AIcs.LGstat.ML

Gaurav Shrivastava , Abhinav Shrivastava This is my paper · ORCID
classification cs.CVcs.AIcs.LGstat.ML
keywords videopredictioncontinuousprocessdiffusionmodelsvariationalboundnoisescheduleMarkovchainframeinterpolationsamplingefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most diffusion-based video predictors treat a video as a stack of independent frames and bolt on temporal attention to keep them consistent. This paper argues that the right object is the transition between two consecutive frames, modeled as a continuous interpolation with a noise schedule that vanishes at both endpoints. It derives a variational bound whose training objective is simply to predict the next frame from a noised interpolation, and uses a Markov-chain reverse process to sample. The authors report that this Continuous Video Process (CVP) reaches the best FVD scores among the baselines they compare on KTH, BAIR, Human3.6M, and UCF101, while needing only 25 sampling steps per frame rather than 100 for the MCVD baseline. If correct, video prediction becomes both more accurate and substantially cheaper at inference time, without temporal attention.

What carries the argument

The central object is the continuous interpolation $\mathbf{x}_t = (1-t)\mathbf{x} + t\mathbf{y} - \frac{t\log t}{\sqrt{2}}\mathbf{z}$ between two consecutive frames, together with the noise schedule $g(t) = -t\log t$, which is zero at both endpoints so that $p(\mathbf{x}_t)$ exists everywhere. The reverse process is a Markov chain with learned Gaussian transitions, and the loss reduces to $L_{\mathrm{simple}}(\theta) = \mathbb{E}_{t,\mathbf{x}_t}\left[\frac{1}{2g^2(t)}\lVert \mathbf{y} - \mathbf{y}_\theta(\mathbf{x}_t, t)\rVert^2\right]$. Sampling iterates $\mathbf{x}_{t+1} = \mathbf{x}_t + (\hat{\mathbf{y}}(\mathbf{x}_t,t)-\mathbf{x})d - t\log t\,\mathbf{z}$, which lets the model generate the next frame with far fewer steps than a full denoising diffusion process.

What would settle it

Re-run the BAIR 1-to-15 setting with VDM and CVP under the same conditioning, sampling steps, and FVD computation; the paper's Table 2 lists VDM at 66.9 and CVP at 70.1, so if CVP cannot match or beat VDM in a controlled comparison, the claim that CVP is state-of-the-art on BAIR collapses.

Watch

Extended reading notes

Core claim

The paper's central claim is that video prediction should be reframed as learning a continuous process between two consecutive frames, rather than generating each frame independently. Given consecutive frames $\mathbf{x}$ and $\mathbf{y}$, CVP defines intermediate states by $\mathbf{x}_t = (1-t)\mathbf{x} + t\mathbf{y} - \frac{t\log t}{\sqrt{2}}\mathbf{z}$, with $g(t) = -t\log t$ so that the noise is zero at both endpoints. The forward process is a fixed Markov chain, and the reverse process is a learned Markov chain with Gaussian transitions, trained by minimizing a variational bound that simplifies to predicting $\mathbf{y}$ from a noised interpolation. The paper reports that this model sets new state-of-the-art FVD scores on KTH, BAIR, Human3.6M, and UCF101 while using 75% fewer sampling steps per frame than the MCVD baseline.

Load-bearing premise

The argument assumes that the transition between two consecutive frames is a first-order Markov process and that a few context frames (as few as four on KTH) carry enough information to predict the next frame; if real video dynamics need longer memory, the learned reverse process degrades, as the paper's Limitation section acknowledges.

Editorial extensions

If this is right

  • Per-frame sampling drops from 100 steps for MCVD to 25 steps for CVP, a 75% reduction, making video prediction substantially cheaper at inference time.
  • Training requires only a short context block plus one future frame (e.g., 4 context frames plus 1 future frame on KTH), rather than long blocks of future frames.
  • Temporal coherence emerges from the continuous interpolation itself, so the model does not require temporal attention blocks or other external constraints.
  • The 4-context-frame predictor can autoregressively produce 30 or 40 future frames on KTH by feeding its own output back as context.
  • Because CVP predicts the next frame rather than a whole block, the same approach extends to arbitrary prediction horizons and to other endpoint-to-endpoint tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not develop is using the same two-endpoint interpolation for video frame interpolation, where the endpoint is a future frame rather than the immediate next frame.
  • The paper's reliance on a short context window is a bottleneck; a testable extension would condition the reverse process on a learned summary of a longer history, which should help videos with longer-range dependencies.
  • The same variational bound and zero-endpoint noise schedule could be applied to image restoration by setting one endpoint to a corrupted image and the other to a clean image, an application the paper mentions only briefly.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Continuous Video Process (CVP), a video prediction method that models the transition between two consecutive frames as a continuous diffusion-like process along an interpolation path with a noise schedule that vanishes at the endpoints. The authors derive a variational bound, simplify it to a denoising loss, and report experiments on KTH, BAIR, Human3.6M, and UCF101, claiming state-of-the-art performance and a 75% reduction in sampling steps relative to MCVD.

Significance. If correct, the approach would offer an efficient and conceptually clean alternative to diffusion-based video prediction, avoiding external temporal-attention constraints. The paper is clearly written and the empirical study covers standard benchmarks. However, the central theoretical derivation is internally inconsistent, the claimed state-of-the-art result is contradicted by the paper's own Table 2, and the novelty of the submission is obscured by self-citation of the authors' prior CVPR 2024 paper. These issues are load-bearing rather than cosmetic, so the result as presented cannot be accepted.

major comments (4)
  1. [Section 3, Eqs. (1)-(2) and Appendix B] The forward process is mathematically inconsistent. Eq. (1) defines the entire path x_t with a single noise draw z, so given x, y, and z, the whole trajectory is deterministic. Eq. (2) instead uses an independent noise draw at each step. The posterior q(x_t | x_{t-1}, x, y) = N(x_t; x_{t-1} + (y-x), g^2(t) I) in Eq. (9) does not follow from Eq. (1): conditioning on x_{t-1} determines z, leaving zero variance. If, alternatively, Eq. (2) is taken as the definition with independent noises, then the marginal at t=1 has accumulated nonzero variance and does not satisfy the endpoint condition x_1 = y required by Eq. (1). Appendix B's derivation of Eq. (2) from Eq. (1) treats z_{t+\Delta t} - z_t as an independent Gaussian with variance 2, but under Eq. (1) these are the same draw, not independent. This invalidates the posterior used throughout the paper.
  2. [Appendix A, Eq. (8)] Because the forward posterior q(x_t | x_{t-1}, x, y) is not the correct posterior for the process defined by Eq. (1), the variational bound derivation in Appendix A does not produce a valid ELBO for p_theta(y | x). In addition, the step 'Both x0 and xT are observed variable hence, we ignore the first term in the RHS' drops the term -log p(x0) + log p(x0|xT); this term is constant with respect to theta only because the forward process has no learnable parameters, which the paper does not state. More importantly, the simplification to a sum of KL divergences relies entirely on the erroneous Gaussian posterior. Thus Eq. (13) may be a plausible denoising objective, but it is not derived from the claimed variational bound.
  3. [Table 2 and Section 5 (BAIR)] The abstract and Section 5 claim state-of-the-art performance across all four datasets, but Table 2 shows that on BAIR with p=1, k=1, #pred=15, VDM achieves FVD 66.9 while CVP achieves 70.1 (lower is better). VDM therefore outperforms CVP on this benchmark. This directly contradicts the paper's central empirical claim that CVP establishes state-of-the-art results.
  4. [Introduction, Section 1 and Contributions] The paper presents the method as a novel contribution, but Section 1 states 'Our proposed method [40]', where reference [40] is the authors' own CVPR 2024 paper 'Video prediction by modeling videos as continuous multi-dimensional processes'. The contributions list claims the introduction of a novel model class and a novel variational bound, but the manuscript does not state what is new relative to [40]. If this submission is an extension of the prior paper, the novelty claim must be clarified explicitly; as written, the paper appears to be a re-presentation of already published work.
minor comments (5)
  1. [Eq. (9)] There is a notation mismatch in Eq. (9): the mean is written as \tilde{\mu}(x_{t-1}, x, y) but the definition below it gives \tilde{\mu}(x_t, x, y) = x_t + (y-x). The argument should be consistent.
  2. [Section 5, KTH setup] The text says CVP uses only the last 4 of the 10 context frames, while Table 1's header says all models condition on 10 past frames. This discrepancy should be clarified so the evaluation protocol is unambiguous.
  3. [Table 3] The Grid keypoint baseline uses 8 context frames whereas CVP uses 5; the comparison is not controlled for context length, which should be noted or adjusted.
  4. [Algorithm 1 and Table 6] Algorithm 1 samples t from Uniform({1,...,T}) but Table 6 reports the best result with t sampled from sqrt(U[0,1]); the relation between the discrete timesteps in the algorithm and the continuous distribution in the ablations should be explained.
  5. [Table 6 and Eq. (13)] The loss in Eq. (13) divides by g^2(t) where g(t) = -t log(t). Since g(t) vanishes at t=0 and t=1, the weighting can diverge near the endpoints; the paper should state how the discrete sampling avoids this, for example by excluding the endpoints.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the empirical benchmarks are external and the method is re-derived in Section 3; the only self-citation is transparent and not load-bearing for the measured results.

full rationale

The paper's empirical claims (FVD/PSNR/SSIM on KTH, BAIR, Human3.6M, UCF101) are measured against external baselines and are not derived from any fitted parameter of the model, so they are not circular. The training objective Eq. (13) is a standard denoising loss constructed to predict the target frame y from an interpolated noisy frame; this is the intended supervised objective, not a hidden equivalent of an input. The only self-reference is the sentence 'Our proposed method [40] defines this continuous process' (Section 1), which points to the authors' own CVPR 2024 paper. That citation is transparent and is not used to justify the measured FVD scores; the method equations are restated in Section 3 and the appendix. A separate mathematical concern exists: the forward-process posterior Eq. (9) is not actually implied by the single-noise interpolation Eq. (1), because the appendix replaces z_{t+Δt} - z_t with an independent sqrt(2) z, which is incompatible with the single z in Eq. (1). This is an internal consistency/correctness issue, not a circular reduction of a prediction to its input, so it does not raise the circularity score beyond 2.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a Markov-interpolation model of video dynamics, an empirically chosen noise schedule and time-step distribution, and a standard but sketchy variational-bound derivation. No code is released, so the empirical claims rest on the reported tables alone.

free parameters (5)
  • Noise schedule g(t) = g(t) = -t log(t)
    Selected empirically via ablations on KTH (Table 6) among sin(pi t), t sin(pi t), sqrt(t(1-t)), and -t log(t).
  • Timestep sampling distribution = t ~ sqrt(U[0,1])
    Chosen empirically over U[0,1]; improves FVD from 190.4 to 140.6 on KTH (Table 6).
  • Number of sampling steps = 25
    Ablations show 5, 10, 25, 50 steps; 25 chosen as the efficiency/quality trade-off (Table 6).
  • Number of context frames = 4 for KTH, 5 for Human3.6M/UCF101, 1 or 2 for BAIR
    Set per dataset; KTH uses 4 while the baseline table claims 10 past frames.
  • Training hyperparameters = batch 64, 500k iterations, LR 5e-5, 2 A6000 GPUs
    Reported in Appendix C; these choices affect convergence and are not derived from first principles.
assumptions (5)
  • domain assumption The forward process x_t = (1-t)x + t y + g(t) z with g(t) vanishing at endpoints is a valid generative interpolation for video frames.
    Eq. (1); no proof that this interpolation captures video dynamics; only empirical validation.
  • domain assumption Transitions between consecutive video frames are first-order Markov: the current state depends only on the previous state.
    Section 3, paragraph after Eq. (2); used to derive the reverse process and the variational bound.
  • domain assumption The reverse process can be Gaussian with untrained variance g^2(t)I and mean parameterized as x_t + (y_theta - x).
    Section 3, after Eq. (9); the capacity of the network is assumed sufficient.
  • standard math The simplification of the variational bound in Appendix A is valid, including dropping terms that depend only on observed endpoints.
    Appendix A; the derivation is sketched and contains unjustified cancellations.
  • domain assumption FVD evaluated with I3D embeddings trained on Kinetics-400 is a reliable metric for comparing video prediction models.
    Section 4.2; the paper uses FVD as the primary metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Continuous Video Process: Modeling Videos as Continuous Multi-Dimensional Processes for Video Prediction." pith.science (2026). https://pith.science/paper/4JVIGFYD

@misc{pith2026241204929,
  author       = {Pith},
  title        = {Pith review of: Continuous Video Process: Modeling Videos as Continuous Multi-Dimensional Processes for Video Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4JVIGFYD}},
  note         = {Machine review of arXiv:2412.04929}
}
read the original abstract

Diffusion models have made significant strides in image generation, mastering tasks such as unconditional image synthesis, text-image translation, and image-to-image conversions. However, their capability falls short in the realm of video prediction, mainly because they treat videos as a collection of independent images, relying on external constraints such as temporal attention mechanisms to enforce temporal coherence. In our paper, we introduce a novel model class, that treats video as a continuous multi-dimensional process rather than a series of discrete frames. We also report a reduction of 75\% sampling steps required to sample a new frame thus making our framework more efficient during the inference time. Through extensive experimentation, we establish state-of-the-art performance in video prediction, validated on benchmark datasets including KTH, BAIR, Human3.6M, and UCF101. Navigate to the project page https://www.cs.umd.edu/~gauravsh/cvp/supp/website.html for video results.

Figures

Figures reproduced from arXiv: 2412.04929 by the authors.

Figure 1
Figure 1. The figure is divided into two parts. The top portion [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Fig. (a) demonstrates the methodology for estimating [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure represents qualitative results of our CVP model on [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Figure represents qualitative results of our CVP model on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Figure represents qualitative results of our CVP model on the Human3.6M dataset. The number of context frames used in the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Figure represents qualitative results of our CVP model on the UCF dataset. The number of context frames used in the above setting [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 42 canonical work pages

  1. [40]

    Video predic- tion by modeling videos as continuous multi-dimensional processes

    Gaurav Shrivastava and Abhinav Shrivastava. Video predic- tion by modeling videos as continuous multi-dimensional processes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7236–7245,

    work page

  2. [1]

    Slamp: Stochastic latent appearance and motion prediction

    Adil Kaan Akan, Erkut Erdem, Aykut Erdem, and Fatma Güney. Slamp: Stochastic latent appearance and motion prediction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14728–14737, 2021. 5

    work page 2021

  3. [2]

    Fitvid: Overfitting in pixel-level video prediction

    Mohammad Babaeizadeh, Mohammad Taghi Saffar, Suraj Nair, Sergey Levine, Chelsea Finn, and Dumitru Erhan. Fitvid: Overfitting in pixel-level video prediction. arXiv preprint arXiv:2106.13195, 2021. 6

    arXiv 2021

  4. [3]

    Cold diffusion: Inverting arbitrary image transforms without noise

    Arpit Bansal, Eitan Borgnia, Hong-Min Chu, Jie S Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Cold diffusion: Inverting arbitrary image transforms without noise. arXiv preprint arXiv:2208.09392,

    arXiv

  5. [4]

    Disentangling multiple features in video sequences using gaussian processes in variational autoencoders, 2020

    Sarthak Bhagat, Shagun Uppal, Zhuyun Yin, and Nengli Lim. Disentangling multiple features in video sequences using gaussian processes in variational autoencoders, 2020. 2

    work page 2020

  6. [5]

    Hierarchical video prediction using re- lational layouts for human-object interactions

    Navaneeth Bodla, Gaurav Shrivastava, Rama Chellappa, and Abhinav Shrivastava. Hierarchical video prediction using re- lational layouts for human-object interactions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12146–12155, 2021. 1, 2

    work page 2021

  7. [6]

    Deep video generation, prediction and completion of human action sequences

    Haoye Cai, Chunyan Bai, Yu-Wing Tai, and Chi-Keung Tang. Deep video generation, prediction and completion of human action sequences. Lecture Notes in Computer Science, page 374–390, 2018. 2

    work page 2018

  8. [7]

    Im- proved conditional vrnns for video prediction

    Lluis Castrejon, Nicolas Ballas, and Aaron Courville. Im- proved conditional vrnns for video prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7608–7617, 2019. 2, 6

    work page 2019

Show all 57 references
  1. [8]

    Courville

    Lluís Castrejón, Nicolas Ballas, and Aaron C. Courville. Improved conditional vrnns for video prediction. CoRR, abs/1904.12165, 2019. 2

    1904 arXiv

  2. [9]

    Latent structured models for human pose estimation

    Cristian Sminchisescu Catalin Ionescu, Fuxin Li. Latent structured models for human pose estimation. InInternational Conference on Computer Vision, 2011. 2, 4

    2011

  3. [10]

    Adver- sarial video generation on complex datasets

    Aidan Clark, Jeff Donahue, and Karen Simonyan. Adver- sarial video generation on complex datasets. arXiv preprint arXiv:1907.06571, 2019. 2, 6

    1907 arXiv

  4. [11]

    Video ladder networks

    Francesco Cricri, Xingyang Ni, Mikko Honkala, Emre Aksu, and Moncef Gabbouj. Video ladder networks. CoRR, abs/1612.01756, 2016. 2

    2016 arXiv

  5. [12]

    Efficient video prediction via sparsely conditioned flow matching

    Aram Davtyan, Sepehr Sameni, and Paolo Favaro. Efficient video prediction via sparsely conditioned flow matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 23263–23274, 2023. 2, 5, 6

    2023

  6. [13]

    Inversion by direct iteration: An alternative to denoising diffusion for image restoration

    Mauricio Delbracio and Peyman Milanfar. Inversion by direct iteration: An alternative to denoising diffusion for image restoration. arXiv preprint arXiv:2303.11435, 2023. 2

    2023 arXiv

  7. [14]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009. 2

    2009

  8. [15]

    Stochastic video generation with a learned prior, 2018

    Emily Denton and Rob Fergus. Stochastic video generation with a learned prior, 2018. 2, 5, 6, 7

    2018

  9. [16]

    Diffusion models beat gans on image synthesis.Advances in Neural Information Processing Systems, 34, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.Advances in Neural Information Processing Systems, 34, 2021. 1

    2021

  10. [17]

    Lee, and Sergey Levine

    Frederik Ebert, Chelsea Finn, Alex X. Lee, and Sergey Levine. Self-supervised visual planning with temporal skip connec- tions, 2017. 2, 4

    2017

  11. [18]

    Elsayed, A

    N. Elsayed, A. S. Maida, and M. Bayoumi. Reduced-gate convolutional lstm architecture for next-frame video predic- tion using predictive coding. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1–9, 2019. 2

    2019

  12. [19]

    Geodict: an integrated gazetteer

    Jacques Fize, Gaurav Shrivastava, and Pierre André Ménard. Geodict: an integrated gazetteer. In Proceedings of Language, Ontology, Terminology and Knowledge Structures Workshop (LOTKS 2017), 2017. 2

    2017

  13. [20]

    Stochastic latent residual video prediction

    Jean-Yves Franceschi, Edouard Delasalles, Mickaël Chen, Sylvain Lamprier, and Patrick Gallinari. Stochastic latent residual video prediction. In International Conference on Machine Learning, pages 3233–3246. PMLR, 2020. 5, 6

    2020

  14. [21]

    Accurate grid keypoint learning for efficient video prediction

    Xiaojie Gao, Yueming Jin, Qi Dou, Chi-Wing Fu, and Pheng- Ann Heng. Accurate grid keypoint learning for efficient video prediction. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages 5908–5915. IEEE, 2021. 5, 6

    2021

  15. [22]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014. 2

    2014

  16. [23]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. Advances in Neural Information Processing Systems, 2020. 1, 3

    2020

  17. [24]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models. 2022. 2, 6

    2022

  18. [25]

    Diffusion models for video prediction and infilling, 2022

    Tobias Höppe, Arash Mehrjou, Stefan Bauer, Didrik Nielsen, and Andrea Dittadi. Diffusion models for video prediction and infilling, 2022. 2, 6, 7

    2022

  19. [26]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding varia- tional bayes. CoRR, abs/1312.6114, 2013. 2

    2013 arXiv

  20. [27]

    Ccvs: context-aware controllable video synthesis

    Guillaume Le Moing, Jean Ponce, and Cordelia Schmid. Ccvs: context-aware controllable video synthesis. Advances in Neu- ral Information Processing Systems, 34:14042–14055, 2021. 6, 7

    2021

  21. [28]

    Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, and Sergey Levine

    Alex X. Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, and Sergey Levine. Stochastic adversarial video prediction. arXiv preprint arXiv:1804.01523, 2018. 2, 5, 6

    2018 arXiv

  22. [29]

    Nuwa-infinity: Autoregressive over autoregressive generation for infinite visual synthesis

    Jian Liang, Chenfei Wu, Xiaowei Hu, Zhe Gan, Jianfeng Wang, Lijuan Wang, Zicheng Liu, Yuejian Fang, and Nan Duan. Nuwa-infinity: Autoregressive over autoregressive generation for infinite visual synthesis. Advances in Neural Information Processing Systems, 35:15420–15432, 2022. 6

    2022

  23. [30]

    Transformation-based adversarial video prediction on large- scale data

    Pauline Luc, Aidan Clark, Sander Dieleman, Diego de Las Casas, Yotam Doron, Albin Cassirer, and Karen Simonyan. Transformation-based adversarial video prediction on large- scale data. arXiv preprint arXiv:2003.04035, 2020. 2, 6

    2003 arXiv

  24. [31]

    Unsupervised learning of object structure and dynamics from videos

    Matthias Minderer, Chen Sun, Ruben Villegas, Forrester Cole, Kevin P Murphy, and Honglak Lee. Unsupervised learning of object structure and dynamics from videos. Advances in Neural Information Processing Systems, 32, 2019. 5, 6

    2019

  25. [32]

    Folded re- current neural networks for future video prediction

    Marc Oliu, Javier Selva, and Sergio Escalera. Folded re- current neural networks for future video prediction. CoRR, abs/1712.00311, 2017. 2

    2017 arXiv

  26. [33]

    Latent video transformer

    Ruslan Rakhimov, Denis V olkhonskiy, Alexey Artemov, De- nis Zorin, and Evgeny Burnaev. Latent video transformer. arXiv preprint arXiv:2006.10704, 2020. 6

    2006 arXiv

  27. [34]

    Valorcarn-tetis: Terms extracted with biotex

    Mathieu Roche, Maguelonne Teisseire, and Gaurav Shrivas- tava. Valorcarn-tetis: Terms extracted with biotex. 2017. 2

    2017

  28. [35]

    Recognizing actions using object states

    Nirat Saini, Bo He, Gaurav Shrivastava, Sai Saketh Ramb- hatla, and Abhinav Shrivastava. Recognizing actions using object states. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality, 2022. 1

    2022

  29. [36]

    Schuldt, I

    C. Schuldt, I. Laptev, and B. Caputo. Recognizing human actions: a local svm approach. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., pages 32–36 V ol.3, 2004. 2, 4

    2004

  30. [37]

    Diverse Video Generation

    Gaurav Shrivastava. Diverse Video Generation. PhD thesis, University of Maryland, College Park, 2021. 1

    2021

  31. [38]

    Advanced video modeling techniques for generation and enhancement tasks

    Gaurav Shrivastava. Advanced video modeling techniques for generation and enhancement tasks. PhD thesis, University of Maryland, College Park, 2024

    2024

  32. [39]

    Diverse video generation using a gaussian process trigger

    Gaurav Shrivastava and Abhinav Shrivastava. Diverse video generation using a gaussian process trigger. arXiv preprint arXiv:2107.04619, 2021. 1, 2, 6

    2021 arXiv

  33. [41]

    Video dynamics prior: An internal learning approach for robust video enhancements

    Gaurav Shrivastava, Ser-Nam Lim, and Abhinav Shrivastava. Video dynamics prior: An internal learning approach for robust video enhancements. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 1

    2023

  34. [42]

    Video decomposition prior: Editing videos layer by layer

    Gaurav Shrivastava, Ser-Nam Lim, and Abhinav Shrivastava. Video decomposition prior: Editing videos layer by layer. In The Twelfth International Conference on Learning Represen- tations, 2024. 1

    2024

  35. [43]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. International Conference on Learning Representations, 2020. 1

    2020

  36. [44]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference on Learning Representations, 2021. 1

    2021

  37. [45]

    Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild, 2012. 2, 4

    2012

  38. [46]

    Unsupervised learning of video representations using lstms

    Nitish Srivastava, Elman Mansimov, and Ruslan Salakhutdi- nov. Unsupervised learning of video representations using lstms. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37, pages 843–852. JMLR.org, 2015. 2

    2015

  39. [47]

    To- wards accurate generative models of video: A new metric and challenges, 2018

    Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric and challenges, 2018. 4

    2018

  40. [48]

    Decomposing motion and content for natural video sequence prediction

    Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, and Honglak Lee. Decomposing motion and content for natural video sequence prediction. CoRR, abs/1706.08033,

    arXiv

  41. [49]

    Learning to generate long-term future via hierarchical prediction, 2017

    Ruben Villegas, Jimei Yang, Yuliang Zou, Sungryull Sohn, Xunyu Lin, and Honglak Lee. Learning to generate long-term future via hierarchical prediction, 2017. 2

    2017

  42. [50]

    Le, and Honglak Lee

    Ruben Villegas, Arkanath Pathak, Harini Kannan, Dumitru Erhan, Quoc V . Le, and Honglak Lee. High fidelity video prediction with large stochastic recurrent neural networks,

  43. [51]

    Mcvd-masked conditional video diffusion for prediction, gen- eration, and interpolation

    Vikram V oleti, Alexia Jolicoeur-Martineau, and Chris Pal. Mcvd-masked conditional video diffusion for prediction, gen- eration, and interpolation. Advances in Neural Information Processing Systems, 35:23371–23385, 2022. 2, 5, 6, 7

    2022

  44. [52]

    Patch to the future: Unsupervised visual prediction

    Jacob Walker, Abhinav Gupta, and Martial Hebert. Patch to the future: Unsupervised visual prediction. In Proceed- ings of the IEEE conference on Computer Vision and Pattern Recognition, pages 3302–3309, 2014. 2

    2014

  45. [53]

    The pose knows: Video forecasting by generating pose futures

    Jacob Walker, Kenneth Marino, Abhinav Gupta, and Martial Hebert. The pose knows: Video forecasting by generating pose futures. In International Conference on Computer Vision,

  46. [54]

    Eidetic 3d LSTM: A model for video prediction and beyond

    Yunbo Wang, Lu Jiang, Ming-Hsuan Yang, Li-Jia Li, Ming- sheng Long, and Li Fei-Fei. Eidetic 3d LSTM: A model for video prediction and beyond. In International Conference on Learning Representations, 2019. 2

    2019

  47. [55]

    Hierarchical long-term video prediction without super- vision, 2018

    Nevan Wichers, Ruben Villegas, Dumitru Erhan, and Honglak Lee. Hierarchical long-term video prediction without super- vision, 2018. 2

    2018

  48. [56]

    Videogpt: Video generation using vq-vae and transform- ers

    Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srini- vas. Videogpt: Video generation using vq-vae and transform- ers. arXiv preprint arXiv:2104.10157, 2021. 6

    2021 arXiv

  49. [57]

    A data-driven approach for event prediction

    Jenny Yuen and Antonio Torralba. A data-driven approach for event prediction. In European Conference on Computer Vision, pages 707–720. Springer, 2010. 2 Continuous Video Process: Modeling Videos as Continuous Multi-Dimensional Processes for Video Prediction Supplementary Mate...

    2010

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.