Pith. sign in

REVIEW 5 major objections 5 minor 45 references

Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The MTID model synthesizes intermediate visual features by interpolating start and goal latents, and uses them to supervise a diffusion planner, improving procedure planning on CrossTask, COIN, and NIV.

desk verdict Solid incremental work whose own ablations undercut the central interpolation claim; the method still works, but the story needs to change. read the letter →

arxiv 2507.03393 v1 pith:4N35VNEO submitted 2025-07-04 cs.CV

Yufan Zhou , Zhaobo Qi , Lingshuai Lin , Junqi Jing , Tingting Chai , Beichen Zhang , Shuhui Wang , Weigang Zhang This is my paper
classification cs.CV
keywords procedureplanninginstructionalvideosdiffusionmodelstemporalinterpolationlatentspaceactionsequencegenerationmaskedprojectionvideoprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MTID, a diffusion-based model for procedure planning in instructional videos that predicts the action sequence taking a scene from a start observation to a goal observation. Its central idea is that the missing intermediate visual states, which no previous method supervises at the visual level, can be synthesized as latent-space interpolations between the encoded start and goal frames. These interpolated features are injected into the denoising U-Net through cross-attention, so the diffusion process receives mid-state visual guidance while training end-to-end. The paper argues this visual-level temporal supervision is richer than the text-level supervision used by prior work and shows gains over strong baselines on CrossTask, COIN, and NIV on most metrics. The importance, if the claim holds, is that expensive intermediate-frame labels are not needed to give a planner a sense of what lies between the two observed states.

What carries the argument

The load-bearing object is the latent space temporal interpolation module: a learnable interpolation matrix $\phi$ that produces $M$ latent mid-state features as convex combinations of the encoded start and goal latents, followed by transformer refinement and cross-attention injection into each residual temporal block of the U-Net. It carries the argument by supplying the only visual-level supervision of intermediate states; the masks and proximity loss are auxiliary mechanisms that constrain and reweight the diffusion output.

What would settle it

Train MTID on CrossTask with the interpolation matrix fixed to a uniform linear schedule $\phi_j = j/(M+1)$ instead of the learned one; if the success rate at $T=3$ does not fall materially below 40.45, the learnable interpolation is not the source of the gain. A second check is the paper's own Table 15, where replacing interpolated features with real intermediate frame features raises CrossTask SR from 40.45 to 49.05 at $T=3$, so a reader could test whether the interpolated features behave more like a goal-feature copy than like real mid-states.

Watch

Extended reading notes

Core claim

MTID decomposes procedure planning into task classification, intermediate-feature reconstruction, and action generation: $p(a_{1:T} \mid V_s, V_g)$ is written as an integral over task class $c$ and $M$ synthesized intermediate features $\upsilon_{1:M}$. The intermediate features come from a latent space temporal interpolation module: an observation encoder maps $V_s$ and $V_g$ to latents $L_s$ and $L_g$, a learnable interpolation matrix $\phi \in \mathbb{R}^{M \times O}$ with entries in $[0,1]$ produces $I_j = (1-\phi_j)L_s + \phi_j L_g$, and transformer blocks refine the sequence before it is fed as key and value into cross-attention layers of the U-Net's residual temporal blocks. An action-aware mask projection restricts the initial Gaussian noise to actions belonging to the predicted task, and a task-adaptive masked proximity loss up-weights steps near the observed endpoints while using the interpolated features for continuous supervision. On CrossTask, COIN, and NIV the model reports higher success rate, mean accuracy, and mean IoU than previous diffusion, knowledge-graph, and LLM-based planners on most horizons, with additional gains in diversity-oriented uncertainty metrics on CrossTask.

Load-bearing premise

The method assumes that the true intermediate visual states lie on or near the straight line in latent space between the encoded start and goal frames, so each mid-state can be written as $(1-\phi_j)L_s + \phi_j L_g$.

Editorial extensions

If this is right

  • Procedure planners can be trained with visual mid-state supervision without collecting or annotating intermediate frames, since the supervision is synthesized in latent space.
  • Injecting interpolated features via cross-attention makes the diffusion U-Net temporally coherent, leading to higher success rate and mean accuracy at horizons $T=3$ to $T=6$ on CrossTask.
  • An action-aware mask on the initial noise confines generation to the action set of the predicted task, filtering out task-irrelevant actions.
  • A gradient proximity loss that down-weights middle steps while using synthesized mid-state features improves over the both-ends weighting used by the PDPP baseline.
  • Under the KEPP evaluation setting, MTID matches or exceeds the strongest text- and LLM-supervised baselines on COIN and NIV for most metrics, showing that visual-level supervision can substitute for text-level state-change descriptions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable consequence of the ablation showing interpolation weights converge near 1 is that a simpler goal-latent injection with per-step learnable offsets should reproduce most of the gain on COIN, where the paper observes interpolated features even outperform real features.
  • The gap between interpolated and real mid-state features (SR 40.45 vs 49.05 on CrossTask at $T=3$) suggests the current linear interpolation is a performance ceiling; a learned mid-state predictor could close much of that gap while keeping the no-annotation advantage.
  • The use of $M$ interpolated features matched to U-Net depth rather than to the $T$ planning steps implies the number of supervision points matters more than one-to-one alignment with action steps, a principle that could transfer to other conditional generation tasks with synthesized intermediate guidance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Masked Temporal Interpolation Diffusion (MTID), a diffusion-based model for procedure planning in instructional videos. Given a start observation Vs and a goal observation Vg, the model predicts an action sequence a1:T. The method decomposes the posterior in Eq. (1) into three sub-problems: a transformer task classifier, a latent-space temporal interpolation module that generates M intermediate latent features using a learnable interpolation matrix (Eq. 6), and a U-Net denoiser with cross-attention injection (Eq. 8). It also introduces a masked projection for action initialization and a task-adaptive masked proximity loss. Experiments are reported on CrossTask, COIN, and NIV, with ablations and uncertainty modeling.

Significance. If the central mechanism worked as claimed, replacing unlabeled mid-state supervision by latent interpolation would be a practically valuable contribution to diffusion-based procedure planning, and the reported gains on CrossTask and COIN would be of interest. The paper is clearly written, releases code, and contains a broad ablation suite (Tables 4-16) and uncertainty evaluation (Tables 7-8). However, the core claim that the interpolation module provides genuine mid-state visual supervision is weakened by the paper's own ablations, which show that the learned interpolation ratio collapses to the goal latent, and by Table 15, where real intermediate features substantially outperform the interpolated ones on CrossTask. The comparison protocol for mIoU also differs from prior work, making cross-method mIoU claims unreliable.

major comments (5)
  1. [§3.2.2, Eq. (6); §4.3, Fig. 4a] The paper states in §4.3 that the best initialization is φ≈1 and that φ converges close to 1. For a matrix φ of shape M×O, Eq. (6) then yields Ij≈Lg for every j, so the module does not produce an ordered sequence of distinct interpolated mid-state features. This contradicts the claimed mid-state visual supervision and makes the p(υ1:M | Vs, Vg) term in Eq. (1) a goal-copying operation rather than an interpolation. Please report the learned φ distribution (e.g., mean and standard deviation per row) or otherwise demonstrate that distinct mid-state features are produced, or re-frame the contribution as an adaptive goal-conditioning mechanism.
  2. [Table 15 and §D] Under the same training pipeline, replacing the interpolated features with real intermediate video features improves CrossTask SR from 40.45 to 49.05 at T=3 and from 24.76 to 36.55 at T=4, while on COIN the real features are worse and on NIV the results are mixed. This shows that the interpolation is not a reliable substitute for actual mid-state features and that its benefit is dataset-dependent. The current discussion attributes the gap to 'simple memory vs hard temporal relationships', but this does not address the failure of the linear interpolation assumption in Eq. (6) on the largest/long-sequence benchmark. Please provide a concrete analysis of where interpolation fails or temper the 'richer mid-state details' claim substantially.
  3. [Table 1 note and Appendix A.3] The mIoU scores in Table 1 are computed per-sequence, whereas previous works compute mIoU over mini-batches, and Appendix A.3 admits this can lower scores. Since the baseline mIoU numbers in Table 1 (e.g., SCHEMA 79.82 vs MTID 69.17 at T=3) are reported under different protocols, the cross-method mIoU comparisons in Tables 1 and 3 are not valid as presented. The authors should either re-run all baselines under the per-sequence protocol or omit mIoU from the headline comparison.
  4. [Table 16 and Appendix D] Under the PDPP protocol, MTID is below PDPP on NIV by 0.57 SR at T=3 and 0.91 SR at T=4, and it is also lower on mAcc and mIoU. Since the abstract and conclusion claim promising performance across three benchmark datasets, this negative result should be prominently disclosed in the main text rather than only in an appendix, and its implications for the generality of the interpolation mechanism should be discussed.
  5. [Abstract and §3.3] The abstract claims 'end-to-end training tailored to task-specific requirements', but §3.3 trains the task classifier in a first stage with cross-entropy and then trains the diffusion model in a second stage; there is no gradient flow from the diffusion loss into the classifier. Please either implement joint training or replace 'end-to-end' with a two-stage description.
minor comments (5)
  1. [Appendix A.5] The first sentence says the uncertainty modeling is on CrossTask and COIN, while §4.4 says CrossTask and NIV; please reconcile the dataset names.
  2. [Figure 4b] The symbol 'copy(Lt)' in the caption is not defined; from the text it appears to mean returning Ls for the first half and Lg for the second half. Use copy(Ls/Lg) consistently.
  3. [Eq. (8)] The cross-attention formula is written as a single expression without explicit Q/K/V projections. Please specify the shapes of the input matrix, the interpolated features, and the projection layers so that the implementation is reproducible.
  4. [Table 9] The CrossTask-128 row is identical to the COIN-128 row, which appears to be a copy-paste error; please correct it.
  5. [§3.2.2] The phrase 'requires no parameter tuning' is misleading because W, k, and τ in Eq. (6) are learned; say 'no manual hyper-parameter tuning' or similar.

Circularity Check

2 steps flagged · score 5.0 of 10

The claimed mid-state supervision reduces by construction: with the fitted interpolation matrix converging close to 1, Eq. (6) makes every interpolated feature a copy of the goal latent; the benchmark results themselves remain genuine empirical measurements.

  1. self definitional [Section 3.2.2, Eq. (6); Section 4.3, Figure 4(a) ablation]
    "Unlike fixed linear interpolation, our method dynamically adjusts the interpolation through the learnable interpolation matrix ϕ ∈ R^{M×O} ... Ij = (1 − ϕj) · Ls + ϕj · Lg, (6). ... The highest score occurs when ϕ is initialized to 1, which proves to be the most stable and achieves the best overall performance, indicating that ϕ converges close to 1."

    The paper's core contribution is that the interpolation module yields intermediate features providing 'richer mid-state details' for supervision. But Eq. (6) defines every I_j solely from the endpoint latents L_s and L_g, and the §4.3 ablation reports the fitted ϕ converges close to 1. With ϕ_j ≈ 1 for all j, I_j ≈ L_g: the 'intermediate' features equal one of the two inputs, so the claimed mid-state supervision contains no information beyond the goal feature. The prediction of intermediate states reduces by construction (Eq. (6) plus the fitted matrix) to copying the goal latent; the factor p(υ_{1:M} | V_s, V_g) in Eq. (1) therefore introduces no new mid-state information.

  2. renaming known result [Section 4.3, Figure 4(b); Appendix D, Table 15]
    "The results show that directly returning Lg performs well, suggesting that Vg may play a critical role in action sequence inference. ... CrossTask, characterized by large size and long sequences, exhibits a significant performance gap favoring real features (Interpolated 40.45 vs Real 49.05 SR at T=3)."

    The fitted interpolation matrix ϕ is the mechanism claimed to supply mid-state visual supervision. The paper's own experiments show the learned interpolation degenerates to returning L_g (ϕ ≈ 1) and that 'directly returning Lg performs well'; meanwhile real intermediate features outperform the interpolated ones by a large margin (Table 15). The contribution attributed to latent space temporal interpolation therefore reduces, on the paper's own data, to re-weighted goal conditioning plus cross-attention/transformer refinement — a known conditioning mechanism relabeled as mid-state prediction. The interpolated 'prediction' of mid-states is not an independent estimate; it is a fitted convex combination that the paper's own ablation shows collapses to the goal.

full rationale

The paper is evaluated against external benchmarks (CrossTask, COIN, NIV) with ground-truth action labels, and the reported SR/mAcc/mIoU numbers are genuine test-set measurements, not fitted values; there are no load-bearing self-citations and no imported uniqueness theorems. However, the central mechanistic claim — that the latent space temporal interpolation module 'leverag[es] a learnable interpolation matrix to generate intermediate latent features, thereby augmenting visual supervision with richer mid-state details' — reduces, on the paper's own equations and ablations, to re-weighted goal conditioning. Eq. (6) defines every interpolated feature I_j as a convex combination of the start and goal latents, and Section 4.3 reports that the fitted ϕ converges close to 1, so I_j ≈ L_g for all j: the 'intermediate' supervision is, by construction, a copy of one of the two inputs. The paper's own Figure 4b confirms that directly returning L_g performs well, and Table 15 shows real intermediate features outperform the interpolated ones by a wide margin (CrossTask SR 49.05 vs 40.45 at T=3), confirming the interpolated features carry little genuine mid-state content. The partial circularity is that the claimed new supervision signal is defined solely from the inputs and learned to collapse to the goal, so describing the module as providing 'mid-state details' is a renaming of goal-conditioning. That said, the empirical comparison is honest and externally grounded — the paper discloses the collapse, reports the real-feature upper bound, and does not hide the limitation — so the score is 5 rather than higher.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The method rests on standard deep learning assumptions and a specific ad hoc assumption that latent linear interpolation models intermediate states. Key hand-set hyperparameters (w0, rho, tau init) are not reported in the text, and the interpolated features are an internal construct with no external validation. The paper's own Table 15 provides the most direct evidence about how strong that assumption is.

free parameters (4)
  • w0 = Not reported in paper
    Initial weight in the gradient-weighted proximity loss (Eq. 11). Chosen by hand; no value or ablation given in the text.
  • rho = Not reported in paper
    Scaling coefficient in the mask matrix m_{t,d} (Eq. 12). Chosen by hand; no value or sensitivity analysis provided.
  • tau initialization value = 1 (best in ablation)
    The learnable matrix tau in Eq. 6 is initialized to a constant; the ablation in Figure 4a shows 1 is optimal. This is a design choice, not derived.
  • M (number of interpolated features) = Equal to number of residual temporal blocks in U-Net
    Set by architecture, not tuned; the ablations compare T vs M features and conclude M is better, but M is not independently justified.
assumptions (4)
  • domain assumption HowTo100M pre-trained visual features are sufficient representations of video observations.
    All experiments use pre-extracted HowTo100M features (Appendix A.2). The method may not transfer to raw video input or other feature extractors.
  • domain assumption Task-conditional active action set Task(c) derived from training annotations covers test-time actions.
    The masked projection (Eq. 9) uses Task(c) to zero out actions. If test-time actions fall outside the training-derived set, the model cannot predict them.
  • ad hoc to paper Convex combination in latent space between start and goal features approximates intermediate visual states.
    Eq. 6 defines interpolated features as I_j = (1 - phi_j)L_s + phi_j L_g. No theoretical justification is given, and Table 15 shows a substantial gap between interpolated and real features on CrossTask.
  • domain assumption Ground-truth action labels and task labels in the datasets are accurate.
    Training and evaluation rely on the correctness of the action and task annotations in CrossTask, COIN, and NIV.
invented entities (2)
  • Interpolated latent features F_1..F_M
    purpose: Provide mid-state visual supervision to guide the diffusion denoiser through cross-attention.
    These features are trained only against action labels, not validated against actual mid-state observations. Table 15 shows they underperform real intermediate features on CrossTask.
  • Learnable interpolation matrix phi
    purpose: Weight the start and goal latent features to generate intermediate features.
    The matrix is an internal mechanism. Ablations show the learned weights converge near 1, meaning it largely copies the goal feature, so its role as a 'temporal interpolation' is weak.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos." pith.science (2026). https://pith.science/paper/4N35VNEO

@misc{pith2026250703393,
  author       = {Pith},
  title        = {Pith review of: Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4N35VNEO}},
  note         = {Machine review of arXiv:2507.03393}
}
read the original abstract

In this paper, we address the challenge of procedure planning in instructional videos, aiming to generate coherent and task-aligned action sequences from start and end visual observations. Previous work has mainly relied on text-level supervision to bridge the gap between observed states and unobserved actions, but it struggles with capturing intricate temporal relationships among actions. Building on these efforts, we propose the Masked Temporal Interpolation Diffusion (MTID) model that introduces a latent space temporal interpolation module within the diffusion model. This module leverages a learnable interpolation matrix to generate intermediate latent features, thereby augmenting visual supervision with richer mid-state details. By integrating this enriched supervision into the model, we enable end-to-end training tailored to task-specific requirements, significantly enhancing the model's capacity to predict temporally coherent action sequences. Additionally, we introduce an action-aware mask projection mechanism to restrict the action generation space, combined with a task-adaptive masked proximity loss to prioritize more accurate reasoning results close to the given start and end states over those in intermediate steps. Simultaneously, it filters out task-irrelevant action predictions, leading to contextually aware action sequences. Experimental results across three widely used benchmark datasets demonstrate that our MTID achieves promising action planning performance on most metrics. The code is available at https://github.com/WiserZhou/MTID.

Figures

Figures reproduced from arXiv: 2507.03393 by the authors.

Figure 1
Figure 1. The core idea to solve procedure planning with previous methods and ours. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our Masked Temporal Interpolation Diffusion (prediction horizon [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Module in Figure 3a generates temporally coherent latent features from observations, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Ablation Studies for Interpolation Strategy. Figure 4b illustrates the features generated by [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Ablation studies for interpolation strategy. Note: “always 1” indicates that [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Combined ablation studies of different coefficients. Note: “t1” refers to the number of [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Visualization of diverse plans produced by our model with different horizons. Note: each [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 28 canonical work pages

  1. [1]

    Uncertainty-aware anticipation of activities

    Yazan Abu Farha and Juergen Gall. Uncertainty-aware anticipation of activities. In ICCV Workshops, 2019

    work page 2019

  2. [2]

    Unsupervised learning from narrated instruction videos

    Jean-Baptiste Alayrac, Piotr Bojanowski, Nishant Agrawal, Josef Sivic, Ivan Laptev, and Simon Lacoste-Julien. Unsupervised learning from narrated instruction videos. In CVPR, 2016

    work page 2016

  3. [3]

    Trajectory prediction for robot navigation using flow-guided markov neural operator

    Rashmi Bhaskara, Hrishikesh Viswanath, and Aniket Bera. Trajectory prediction for robot navigation using flow-guided markov neural operator. In ICRA, 2024

    work page 2024

  4. [4]

    Procedure planning in instructional videos via contextual modeling and model-based policy learning

    Jing Bi, Jiebo Luo, and Chenliang Xu. Procedure planning in instructional videos via contextual modeling and model-based policy learning. In ICCV, 2021

    work page 2021

  5. [5]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In CVPR, 2017

    2017

  6. [6]

    Procedure planning in instructional videos

    Chien-Yi Chang, De-An Huang, Danfei Xu, Ehsan Adeli, Li Fei-Fei, and Juan Carlos Niebles. Procedure planning in instructional videos. In ECCV, 2020

    work page 2020

  7. [7]

    Diffusion models in vision: A survey

    Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. Diffusion models in vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45 0 (9): 0 10850--10869, 2023

    work page 2023

  8. [8]

    Who let the dogs out? modeling dog behavior from visual data

    Kiana Ehsani, Hessam Bagherinezhad, Joseph Redmon, Roozbeh Mottaghi, and Ali Farhadi. Who let the dogs out? modeling dog behavior from visual data. In CVPR, 2018

    work page 2018

Show all 45 references
  1. [9]

    Masked diffusion transformer is a strong image synthesizer

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. In ICCV, 2023

    2023

  2. [10]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016

    2016

  3. [11]

    Streamingt2v: Consistent, dynamic, and extendable long video generation from text

    Roberto Henschel, Levon Khachatryan, Daniil Hayrapetyan, Hayk Poghosyan, Vahram Tadevosyan, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Streamingt2v: Consistent, dynamic, and extendable long video generation from text. arXiv preprint arXiv:2403.14773, 2024

    2024 arXiv

  4. [12]

    Cnn architectures for large-scale audio classification

    Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. Cnn architectures for large-scale audio classification. In ICASSP, 2017

    2017

  5. [13]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020

    2020

  6. [14]

    Videobooth: Diffusion-based video generation with image prompts

    Yuming Jiang, Tianxing Wu, Shuai Yang, Chenyang Si, Dahua Lin, Yu Qiao, Chen Change Loy, and Ziwei Liu. Videobooth: Diffusion-based video generation with image prompts. In CVPR, 2024

    2024

  7. [15]

    Text2video-zero: Text-to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tadevosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to-image diffusion models are zero-shot video generators. In ICCV, 2023

    2023

  8. [16]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  9. [17]

    Skip-plan: Procedure planning in instructional videos via condensed action space learning

    Zhiheng Li, Wenjia Geng, Muheng Li, Lei Chen, Yansong Tang, Jiwen Lu, and Jie Zhou. Skip-plan: Procedure planning in instructional videos via condensed action space learning. In ICCV, 2023

    2023

  10. [18]

    Bat: Behavior-aware human-like trajectory prediction for autonomous driving

    Haicheng Liao, Zhenning Li, Huanming Shen, Wenxuan Zeng, Dongping Liao, Guofa Li, and Chengzhong Xu. Bat: Behavior-aware human-like trajectory prediction for autonomous driving. In AAAI, 2024

    2024

  11. [19]

    Howto100m: Learning a text-video embedding by watching hundred million narrated video clips

    Antoine Miech, Dimitri Zhukov, Jean-Baptiste Alayrac, Makarand Tapaswi, Ivan Laptev, and Josef Sivic. Howto100m: Learning a text-video embedding by watching hundred million narrated video clips. In ICCV, 2019

    2019

  12. [20]

    Mish: A self regularized non-monotonic activation function

    Diganta Misra. Mish: A self regularized non-monotonic activation function. arXiv preprint arXiv:1908.08681, 2019

    1908 arXiv

  13. [21]

    Why not use your textbook? knowledge-enhanced procedure planning of instructional videos

    Kumaranage Ravindu Yasas Nagasinghe, Honglu Zhou, Malitha Gunawardhana, Martin Renqiang Min, Daniel Harari, and Muhammad Haris Khan. Why not use your textbook? knowledge-enhanced procedure planning of instructional videos. In CVPR, 2024

    2024

  14. [22]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021

    2021

  15. [23]

    SCHEMA : State CH anges MA tter for procedure planning in instructional videos

    Yulei Niu, Wenliang Guo, Long Chen, Xudong Lin, and Shih-Fu Chang. SCHEMA : State CH anges MA tter for procedure planning in instructional videos. In ICLR, 2024

    2024

  16. [24]

    Self-regulated learning for egocentric video activity anticipation

    Zhaobo Qi, Shuhui Wang, Chi Su, Li Su, Qingming Huang, and Qi Tian. Self-regulated learning for egocentric video activity anticipation. IEEE transactions on pattern analysis and machine intelligence, 45 0 (6): 0 6715--6730, 2021

    2021

  17. [25]

    Uncertainty-boosted robust video activity anticipation

    Zhaobo Qi, Shuhui Wang, Weigang Zhang, and Qingming Huang. Uncertainty-boosted robust video activity anticipation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  18. [26]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022

    2022

  19. [27]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015

    2015

  20. [28]

    Robovqa: Multimodal long-horizon reasoning for robotics

    Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil J Joshi, et al. Robovqa: Multimodal long-horizon reasoning for robotics. In ICRA, 2024

    2024

  21. [29]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021

    2021

  22. [30]

    Universal planning networks: Learning generalizable representations for visuomotor control

    Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Universal planning networks: Learning generalizable representations for visuomotor control. In ICML, 2018

    2018

  23. [31]

    Plate: Visually-grounded planning with transformers in procedural tasks

    Jiankai Sun, De-An Huang, Bo Lu, Yun-Hui Liu, Bolei Zhou, and Animesh Garg. Plate: Visually-grounded planning with transformers in procedural tasks. IEEE Robotics and Automation Letters, 7 0 (2): 0 4924--4930, 2022

    2022

  24. [32]

    Coin: A large-scale dataset for comprehensive instructional video analysis

    Yansong Tang, Dajun Ding, Yongming Rao, Yu Zheng, Danyang Zhang, Lili Zhao, Jiwen Lu, and Jie Zhou. Coin: A large-scale dataset for comprehensive instructional video analysis. In CVPR, 2019

    2019

  25. [33]

    Event-guided procedure planning from instructional videos with text supervision

    An-Lan Wang, Kun-Yu Lin, Jia-Run Du, Jingke Meng, and Wei-Shi Zheng. Event-guided procedure planning from instructional videos with text supervision. In ICCV, 2023 a

    2023

  26. [34]

    Pdpp: Projected diffusion for procedure planning in instructional videos

    Hanlin Wang, Yilu Wu, Sheng Guo, and Limin Wang. Pdpp: Projected diffusion for procedure planning in instructional videos. In CVPR, 2023 b

    2023

  27. [35]

    Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving

    Yuqi Wang, Jiawei He, Lue Fan, Hongxin Li, Yuntao Chen, and Zhaoxiang Zhang. Driving into the future: Multiview visual forecasting and planning with world model for autonomous driving. In CVPR, 2024

    2024

  28. [36]

    Art-v: Auto-regressive text-to-video generation with diffusion models

    Wenming Weng, Ruoyu Feng, Yanhui Wang, Qi Dai, Chunyu Wang, Dacheng Yin, Zhiyuan Zhao, Kai Qiu, Jianmin Bao, Yuhui Yuan, et al. Art-v: Auto-regressive text-to-video generation with diffusion models. In CVPR, 2024

    2024

  29. [37]

    Group normalization

    Yuxin Wu and Kaiming He. Group normalization. In ECCV, 2018

    2018

  30. [38]

    P3iv: Probabilistic procedure planning from instructional videos with weak supervision

    He Zhao, Isma Hadji, Nikita Dvornik, Konstantinos G Derpanis, Richard P Wildes, and Allan D Jepson. P3iv: Probabilistic procedure planning from instructional videos with weak supervision. In CVPR, 2022

    2022

  31. [39]

    Upscale-a-video: Temporal-consistent diffusion model for real-world video super-resolution

    Shangchen Zhou, Peiqing Yang, Jianyi Wang, Yihang Luo, and Chen Change Loy. Upscale-a-video: Temporal-consistent diffusion model for real-world video super-resolution. In CVPR, 2024 a

    2024

  32. [40]

    Storydiffusion: Consistent self-attention for long-range image and video generation

    Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. Storydiffusion: Consistent self-attention for long-range image and video generation. arXiv preprint arXiv:2405.01434, 2024 b

    2024 arXiv

  33. [41]

    Cross-task weakly supervised learning from instructional videos

    Dimitri Zhukov, Jean-Baptiste Alayrac, Ramazan Gokberk Cinbis, David Fouhey, Ivan Laptev, and Josef Sivic. Cross-task weakly supervised learning from instructional videos. In CVPR, 2019

    2019

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  35. [43]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  36. [44]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  37. [45]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.