Pith. sign in

REVIEW 4 major objections 6 minor 144 references

The paper introduces Cross-Video Scene Procedure Planning (CVSPP) and OSEF, which fuses all candidate evidence softly and improves exact-video-and-plan success by up to 10.7 points on same-task COIN and CrossTask cells.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 07:44 UTC pith:FQHUGGCM

load-bearing objection A genuinely new benchmark and a clean interface ablation, but the window-less headline metric and the under-specified leakage audit keep me from trusting the absolute numbers as much as the relative claim. the 4 major comments →

arxiv 2607.29401 v1 pith:FQHUGGCM submitted 2026-07-31 cs.CV

OSEF: One-Step Evidence Fusion for Cross-Video Scene Procedure Planning

Zhentong Ye , Lei Zhang , Sijia Zhou , Yingda Yu , Yuehan Shi , Jiaqi Xuan , Shuaiwu Dong , Guanchao Tong
show 2 more authors
Meimei Zhang Bin Li
This is my paper
classification cs.CV
keywords CVSPPprocedure planningvideo retrievaltemporal localizationevidence fusionsoft latticeinstructional videosretrieval-to-plan error propagation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a new task, Cross-Video Scene Procedure Planning (CVSPP): given a start–goal query that has been redacted so it cannot leak the answer, and a small set of candidate videos, a model must find the right video, locate the right segment, and output the exact action sequence. The authors argue that two coupled failures—same-task evidence ambiguity and retrieval-to-plan error propagation—make retrieve-then-plan cascades brittle. Their method, One-Step Evidence Fusion (OSEF), scores every cell of every candidate against the query and feeds the whole soft lattice to the planner through a token-global adapter, never cropping a window first. On the benchmark's six method-rankable cells OSEF ranks first, and on four matched same-task cells it improves exact-video-and-plan success by 2.9–10.7 percentage points over the strongest hard-selection baseline. A component study attributes the largest single gain to the token-global interface, not to soft pooling.

Core claim

The central discovery is that hard evidence selection—choosing one video and one window before planning—is the main bottleneck in cross-video procedure planning, and that keeping evidence soft through the planner is a better strategy. OSEF computes a query-conditioned lattice of cell scores, lifts them to valid intervals via a symmetric formula, and passes the entire lattice into a goal-conditioned Transformer planner through a token-global adapter. The paper reports that this one-step fusion path improves FULL-SR (exact video and exact action sequence) by 2.9–10.7 points over an enhanced hard-selection SOTA on four matched COIN/CrossTask cells, and that the token-global interface—rather tha

What carries the argument

The token-global Evidence-to-Plan (E2P) adapter: a one-layer Transformer that contextualizes the flattened K×M cell-feature lattice (with a learned score projection injecting evidence logits), followed by 2T learned endpoint queries that cross-attend to all cells to produce T start–goal state pairs. This is what lets the planner retain every candidate's evidence; the query-conditioned cell-and-span lattice Hθ(q,k,i,j) = ½(Lθ(q,k,i)+Lθ(q,k,j)) supplies the evidence scores, and a joint-cascade loss couples plan error to candidate probabilities.

Load-bearing premise

The redacted start–goal query must be genuinely uninformative about the hidden video, window, and action sequence; if the fail-closed audit misses a leak, every retrieval and planning score could be inflated without the model solving the intended task.

What would settle it

A direct audit: insert a detectable cue into the legal query (e.g., the first action verb) and re-run the full benchmark; if FULL-SR does not collapse, the redaction gate is leaking. Alternatively, compute FULL-SR when the evidence branch is given the true video id as an additional input; if OSEF's margin over hard selection shrinks to zero, the gain is entirely in retrieval, not in plan fusion.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • On the six benchmark cells that support method ranking, OSEF is first, and on the four matched same-task COIN/CrossTask cells it beats the enhanced hard-selection SOTA by 2.9–10.7 FULL-SR points.
  • The token-global interface, not soft pooling or the coupled losses, is the main source of the gain (Table 4).
  • Fusion never falls below the hard baseline on any row, and the full stack lifts COIN-T3 PLAN-SR from 46.61% to 55.34%.
  • Distinct-task retrieval is nearly saturated on native sources (NIV-T3 R@1 99.20%), leaving exact plan composition as the bottleneck; T=4 lowers FULL-SR by 17.81–22.70 points while R@1 changes by less than a point.
  • Localization remains weak: CT-T3 FULL-SR 58.09% drops to 6.21% FULL@IoU.3, so a correct plan does not certify the window.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: If the token-global adapter is the key, then a pure soft-pooling or hard-crop variant should lose the same 3–6 points on any new same-task benchmark, not just COIN/CT.
  • Inference: The fail-closed leakage gate is a strong protocol innovation; other retrieval-planning benchmarks could adopt the same redaction audit to prevent answer leakage.
  • Inference: The near-zero FULL@IoU.5 numbers suggest that OSEF's headline FULL-SR metric may overstate practical usability for applications that need the exact segment; a window-aware variant would be the next logical benchmark cell.
  • Inference: Because K=4 and M=16 are fixed, the paper's K-degradation curve (R@1 29.48→8.12 as K goes 4→16) indicates that same-task retrieval, not planning, will be the scalability bottleneck for larger candidate sets.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Cross-Video Scene Procedure Planning (CVSPP), a task in which a model receives an answer-redacted start–goal text query and a set of K candidate videos, and must retrieve the supporting video, localize the relevant temporal window, and predict the intervening closed-set action sequence. The authors construct an 11-source benchmark with typed negative roles, a fail-closed query-observability audit, and separate Evidence- and Plan-axis metrics. They adapt nine existing VSPP planners and propose OSEF, which scores a query-conditioned cell-and-span lattice over all candidates and feeds the full soft lattice to a planner through a token-global adapter, avoiding an early hard video/window selection. OSEF is reported to rank first on the six native source–horizon cells and to improve exact-video-and-plan success by 2.9–10.7 points on four matched same-task COIN/CrossTask cells over an enhanced hard-selection SOTA. The paper also reports diagnostics on converted sources, stress tests on candidate count and feature noise, case analyses, and a supplementary code/constructors package.

Significance. If the empirical claims hold, CVSPP is a useful new benchmark that couples video corpus moment retrieval with procedure planning, and OSEF's delayed-commitment, token-global fusion design is a plausible architectural contribution. The release of model constructors and evaluation code, the typed-negative-role construction, and the attempted fail-closed audit are concrete strengths that go beyond many benchmark papers. However, two load-bearing issues temper the significance: the headline metric omits the localization component that the task definition itself requires, and the leakage audit is under-specified for a setting where queries and hidden answers are derived from the same annotation sources. The validation-selected reporting and the mixed-regime presentation of the hard-SOTA row also need addressing before the stated SOTA/rank claims can be taken at face value.

major comments (4)
  1. [Sec. 4, Eq. (1); Sec. 6.3; Limitations] The task definition in Sec. 4 requires a prediction triple (k, i, j) — supporting video, start/end cell, and action sequence — and the Introduction says CVSPP 'must identify the supporting video, localize its window, and predict the intervening action sequence.' However, FULL-SR in Eq. (1) is defined as Pr[khat=k*, Ahat=A*] with no window term. The paper's own Limitations acknowledge that 'FULL-SR can be correct with a temporally wrong window,' and Sec. 6.3 reports CT-T3 FULL@IoU.3/IoU.5 of only 6.21%/0.54% while FULL-SR is 58.09%. The headline metric therefore does not measure the localization component that the paper advertises. This is not merely a naming issue: the claim 'first benchmark to score all three outputs together' (Sec. 1) is not supported by the stated metric. Please either add a window condition to FULL-SR (or a jointly thresholded variant) or explicitly rename the headli
  2. [Sec. 3, 'Legal query and negative roles'; Sec. 6.6] The fail-closed leakage audit is under-specified. Sec. 3 states only that a normalized phrase audit compares every query against its target action labels, with matches human-adjudicated or quarantined. The legal queries are generated by deterministic templates from start–goal state descriptions, and those state descriptions and the hidden action sequences can come from the same annotation pipeline. An exact/substring phrase audit may not detect near-paraphrase leakage, so retrieval and planning scores in Tables 3–4 could in principle be inflated. The Sec. 6.6 controls (query-only FULL-SR 1.12%, task-name/wrong-transition R@1 changes of 1.81/0.66 points) are encouraging, but they cover only one hard same-task COIN cohort. Please provide a detailed audit protocol — including paraphrase/n-gram checks against captions, state descriptions, and action labels — and per-source query-only baselin
  3. [Sec. 6.1, 'Reporting protocol'] Tables 3–4 report validation-selected performance, not locked-test evaluation: each metric comes from the earliest epoch with maximum validation FULL-SR. The paper acknowledges that the original run weights were not retained and that the tables therefore report validation rather than test numbers. A supplementary locked-split retrain is claimed to reproduce the four matched cells within 2.0 points, but this is not shown in the main paper. Since the central contribution is a ranking/SOTA comparison, the main paper should report locked-test numbers with retained weights, or at least include the locked-split table with per-cell values. Without that, the reported margins and ranks are not final.
  4. [Table 3 and Sec. 6.2] The hard-selection SOTA row in Table 3 uses same-task candidate caches, while every other row (including OSEF) uses distinct-task candidate sets. Sec. 6.2 explicitly states this and says the row is 'not comparable' with the others, yet it is printed in the same table without visual separation, and it includes NIV entries even though the abstract's 2.9–10.7-point improvement is computed only on the four matched CT/COIN same-task cells in Table 4. As printed, the Table 3 layout invites the reader to compare, e.g., CT-T3 18.23 (hard SOTA, same-task) with 58.09 (OSEF, distinct-task), which are different regimes. Please move the hard-SOTA row into a clearly labeled same-task table (or remove it from Table 3), and state in the caption that Table 3's non-hard rows use distinct-task candidates.
minor comments (6)
  1. [Figure 1] The figure contains a typo: 'Ture✓' should be 'True✓'.
  2. [Eq. (3)] The symbol 'CET' is used without definition. Presumably it is a stepwise cross-entropy loss over the T action steps; please define it.
  3. [Table 3 footnote] The footnote says 'Cells are seed 0/three-seed means' — this is ambiguous. Specify which cells are single-seed and which are three-seed means, and give the seed SDs in the main table or a companion table.
  4. [Sec. 4, metric contract] FULL@IoU.3 and FULL@IoU.5 are used in Sec. 6.3 without formal definitions in the metric contract. Please state the exact conjunction (video = k*, plan = A*, and window IoU above the threshold) in Sec. 4.
  5. [Sec. 6.3] The statement that the best adapted route exceeds the majority floor on ANet and CEgo 'by only 3 and 6 strict successes' is hard to interpret without per-cell sample sizes. Please report the numbers in the form 'X/Y examples above the floor' or provide the exact counts in the per-cell supplement.
  6. [Sec. 3, source list] The text says 'Eight more sources' are converted/diagnostic, but Table 2 lists nine non-native entries (YC2, TACoS, HiR, GS, ANet, CEgo, GUIDE, ELearn, plus diagnostic EEL). Clarify whether EEL is included in the 'eight' or counted separately.

Circularity Check

0 steps flagged

No significant circularity: OSEF's predictions are scored against external dataset labels, its legal-query audit is a construction safeguard rather than a fitted target, and its component studies are ablation-based rather than definitional.

full rationale

The paper's central claims are empirical: OSEF is trained and evaluated on external instructional-video datasets (COIN, CrossTask, NIV, and converted sources) with ground-truth action labels and candidate videos that are not generated by OSEF. The headline metric FULL-SR (Eq. 1) is a joint exact-match criterion on retrieved video and predicted action sequence, and it is not used to define any model parameter. The legal-query construction in Sec. 3 is a benchmark-design detail, not a fitted input: queries are audited fail-closed, and the paper's own controls (Sec. 6.6: query-only FULL-SR 1.12 ± 0.16%, wrong-transition R@1 movement 0.66 points; task-name control R@1 movement 1.81 points) empirically address the possibility that the redacted query text leaks the answer. The component study (Table 4) compares alternative interfaces and loss combinations, and the reported gains are relative to hard-selection and pooling baselines, not to quantities derived from OSEF itself. Eq. 2 is an explicit parameter-free construction of interval scores from cell scores, and Eq. 3 is supervised plan loss against external action labels. The one possibly self-authored citation (Li et al. 2023a, ICASSP) is used only as related-work context for the adaptation protocol and is not load-bearing in the derivation or evaluation. The paper also discloses validation-based selection and a locked-split retrain, further reducing the concern that reported numbers are definitionally forced. No equation, fitted parameter, or self-citation chain reduces the claimed predictions to their own inputs.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced. The central claim rests on benchmark-provenance assumptions (leakage audit, label correctness, negative-role validity) and on hand-set training/architecture hyperparameters rather than on fitted theoretical constants.

free parameters (4)
  • Loss coefficients in the released objective = 2, 0.1, 1, 0.5, 1
    L = 2L_ret + 0.1L_state + L_evloc + 0.5L_span + L_JC (Sec. 5). These weights are hand-set and affect the balance between retrieval, localization, and planning gradients.
  • Candidate count K and cell count M = K=4, M=16
    The main protocol fixes K=4 candidates and M=16 cells per video. These are experimental choices that shape difficulty, not learned values, but they are chosen by the authors.
  • Text encoder selection = BERT-base default; DeBERTa-v3-base on NIV
    The choice is justified by a 'matched legal-cache control' at NIV-T4 but is a model-selection decision that affects results.
  • Adapter/planner depth and endpoint query count = 1-layer adapter; 3-layer planner; 2T endpoint queries
    Architecture hyperparameters chosen by hand in Sec. 5; the endpoint-head control is mentioned but no grid is shown in the main text.
axioms (4)
  • domain assumption The fail-closed legal-query audit guarantees no leakage of video identity, timestamps, or action sequence into the query.
    Sec. 3 states a normalized phrase audit compares every query against target action labels and quarantines matches. All benchmark numbers depend on this audit being correct.
  • domain assumption Frozen S3D features pretrained on HowTo100M and BERT/DeBERTa text features are sufficient inputs for evidence binding and planning.
    Sec. 6.1 uses these frozen features throughout; if they are too weak, the reported fusion gains could be an artifact of the feature backbone.
  • domain assumption Validation-selected FULL-SR is a reliable proxy for generalization.
    Sec. 6.1 reports that Tables 3–4 are validation performance rather than locked-test; a prospective retrain on four cells reproduced within 2.0 points, but not all cells were locked-test checked.
  • domain assumption Same-task candidate sets with K=4 and M=16 expose the two stated obstacles (same-task ambiguity, retrieval-to-plan error propagation).
    The benchmark's negative roles and same-task caches are constructed by the authors; their validity as a faithful operationalization of the claimed obstacles is assumed.

pith-pipeline@v1.3.0-daily-deepseek · 15687 in / 14807 out tokens · 166256 ms · 2026-08-03T07:44:16.072124+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of OSEF: One-Step Evidence Fusion for Cross-Video Scene Procedure Planning." pith.science (2026). https://pith.science/paper/FQHUGGCM

@misc{pith2026260729401,
  author       = {Pith},
  title        = {Pith review of: OSEF: One-Step Evidence Fusion for Cross-Video Scene Procedure Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQHUGGCM}},
  note         = {Machine review of arXiv:2607.29401}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Video Scene Procedure Planning (VSPP) supplies the target start-goal observations in advance, leaving open how a planner should act when the evidence must itself be retrieved. We introduce Cross-Video Scene Procedure Planning (CVSPP): given an answer-redacted start-goal query and K candidate videos, a model must retrieve the supporting video, localize the relevant window, and predict the action sequence. Two obstacles couple here. Same-task demonstrations share stages and windows, and an early hard selection passes the wrong scene chain to the planner. We build an eleven-source benchmark with typed negative roles, a fail-closed answer-leakage gate, and separate Evidence- and Plan-axis metrics. On its 14 source-horizon cells we adapt nine planner families against a majority-sequence floor. We then present One-Step Evidence Fusion (OSEF), which scores a query-conditioned cell-and-span lattice over all candidates and feeds the full soft lattice to the planner through a token-global adapter, cropping no window beforehand. OSEF ranks first on all six cells the benchmark certifies as method-rankable. On four matched same-task COIN and CrossTask cells it improves exact-video-and-plan success by 2.9-10.7 points over an enhanced hard-selection SOTA, and a component study assigns the largest single increment to the token-global interface. Five converted-source cells sit at or near the majority-sequence floor, the benchmark's remaining headroom. The supplementary package includes model constructors and evaluation code.

Figures

Figures reproduced from arXiv: 2607.29401 by Bin Li, Guanchao Tong, Jiaqi Xuan, Lei Zhang, Meimei Zhang, Shuaiwu Dong, Sijia Zhou, Yingda Yu, Yuehan Shi, Zhentong Ye.

Figure 1
Figure 1. Figure 1: Task motivation. The three prior paradigms of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: CVSPP construction and source composition. (a) A lineage enters [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: OSEF framework. The Evidence branch scores a query-conditioned K × M cell lattice; the token-global E2P adapter reads all cells into the planner without first cropping a span. 5 Method: One-Step Evidence Fusion Design principle. OSEF delays irreversible evi￾dence commitment: it scores every candidate cell and predicts the plan from the complete lattice in one differentiable path. “One-step” refers to this … view at source ↗
Figure 4
Figure 4. Figure 4: End-to-end hard same-task COIN-T3 cases. Sample 1496 is the maximum-span-IoU full success; Sample 742 predicts the same window but binds the wrong video. Query q scores candidates through pk; Lθ(q, k, m) selects [ˆı,ȷˆ], and the 2T end￾point queries decode Aˆ. the planner fuses. The noise probe perturbs cached features at evaluation only, so it measures feature￾space robustness, not raw-video quality. Fini… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

144 extracted references · 8 canonical work pages

  1. [1]

    Advances in Neural Information Processing Systems (Datasets and Benchmarks Track) , volume =

    Ego4D Goal-Step: Toward Hierarchical Understanding of Procedural Activities , author =. Advances in Neural Information Processing Systems (Datasets and Benchmarks Track) , volume =

  2. [2]

    , howpublished =

    Lee, Seohyun and Choi, Seoung and Ko, Dohwan and Kim, Jongha and Kim, Hyunwoo J. , howpublished =

  3. [3]

    International Conference on Machine Learning , year =

    Revisiting Uncertainty: On Evidential Learning for Partially Relevant Video Retrieval , author =. International Conference on Machine Learning , year =

  4. [4]

    Chrono: A Simple Blueprint for Representing Time in Multimodal Large Language Models , author =

  5. [5]

    Open-Event Procedure Planning in Instructional Videos , author =

  6. [6]

    Advances in Neural Information Processing Systems , volume =

    Evidential Deep Learning to Quantify Classification Uncertainty , author =. Advances in Neural Information Processing Systems , volume =

  7. [7]

    Advances in Neural Information Processing Systems , volume =

    Attention is All you Need , author =. Advances in Neural Information Processing Systems , volume =

  8. [8]

    Categorical Reparameterization with

    Jang, Eric and Gu, Shixiang and Poole, Ben , booktitle =. Categorical Reparameterization with. 2017 , url =

    2017

  9. [9]

    Stochastic Beams and Where To Find Them: The

    Kool, Wouter and van Hoof, Herke and Welling, Max , booktitle =. Stochastic Beams and Where To Find Them: The

  10. [10]

    Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence , pages =

    Reparameterizable Subset Sampling via Continuous Relaxations , author =. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence , pages =. doi:10.24963/ijcai.2019/544 , year =

    work page doi:10.24963/ijcai.2019/544 2019

  11. [11]

    Proceedings of the 34th International Conference on Machine Learning , volume =

    On Calibration of Modern Neural Networks , author =. Proceedings of the 34th International Conference on Machine Learning , volume =

  12. [12]

    doi:10.1109/WACV61041.2025.00854 , year =

    Shi, Lei and B\"urkner, Paul-Christian and Bulling, Andreas , booktitle =. doi:10.1109/WACV61041.2025.00854 , year =

    arXiv 2025

  13. [13]

    doi:10.18653/v1/2025.emnlp-main.1793 , year =

    Gao, Guangze and Li, Zixuan and Yuan, Chunfeng and Li, Jiawei and Wu, Jianzhuo and Zhang, Yuehao and Jin, Xiaolong and Li, Bing and Hu, Weiming , booktitle =. doi:10.18653/v1/2025.emnlp-main.1793 , year =

    doi:10.18653/v1/2025.emnlp-main.1793 2025

  14. [14]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention , author =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. doi:10.18653/v1/2025.acl-long.1126 , year =

    doi:10.18653/v1/2025.acl-long.1126 2025

  15. [15]

    arXiv preprint arXiv:2603.02227 , year =

    Routing Absorption in Sparse Attention: Why Random Gates Are Hard to Beat , author =. arXiv preprint arXiv:2603.02227 , year =

    arXiv

  16. [16]

    Zhang, Jie and Ye, Qilang and Zhou, Hao and Liang, Haochen and Luo, Fei , booktitle =

  17. [17]

    Yu, Rongyi and Duan, Chenyuan and Zhang, Wentao , journal =

  18. [18]

    Ben-Ami, Dan and Serussi, Gabriele and Cohen, Kobi and Baskin, Chaim , booktitle =

  19. [19]

    Propose, Assess, Search: Harnessing

    Islam, Md Mohaiminul and Nagarajan, Tushar and Wang, Huiyu and Chu, Fu-Jen and Kitani, Kris and Bertasius, Gedas and Yang, Xitong , booktitle =. Propose, Assess, Search: Harnessing

  20. [20]

    European Conference on Computer Vision , pages =

    Procedure Planning in Instructional Videos , author =. European Conference on Computer Vision , pages =

  21. [21]

    and Wildes, Richard P

    Zhao, He and Hadji, Isma and Dvornik, Nikita and Derpanis, Konstantinos G. and Wildes, Richard P. and Jepson, Allan D. , booktitle =

  22. [22]

    doi:10.1609/aaai.v39i9.32992 , year =

    Yang, Dejie and Zhao, Zijing and Liu, Yang , booktitle =. doi:10.1609/aaai.v39i9.32992 , year =

    work page doi:10.1609/aaai.v39i9.32992

  23. [23]

    IEEE/CVF International Conference on Computer Vision , pages =

    Procedure Planning in Instructional Videos via Contextual Modeling and Model-Based Policy Learning , author =. IEEE/CVF International Conference on Computer Vision , pages =

  24. [24]

    Wang, Hanlin and Wu, Yilu and Guo, Sheng and Wang, Limin , booktitle =

  25. [25]

    Niu, Yulei and Guo, Wenliang and Chen, Long and Lin, Xudong and Chang, Shih-Fu , booktitle =

  26. [26]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Why Not Use Your Textbook? Knowledge-Enhanced Procedure Planning of Instructional Videos , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  27. [27]

    Shi, Lei and Aregbede, Victor and Persson, Andreas and L\"angkvist, Martin and Loutfi, Amy and Lowry, Stephanie , howpublished =

  28. [28]

    Shi, Lei and Bulling, Andreas , howpublished =

  29. [29]

    International Conference on Learning Representations , url =

    Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos , author =. International Conference on Learning Representations , url =

  30. [30]

    Seminara, Luigi and Moltisanti, Davide and Furnari, Antonino , booktitle =

  31. [31]

    Choi, Woo Suk and Jang, Youwon and Lee, Minsu and Zhang, Byoung-Tak , booktitle =

  32. [32]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Gumbel Reranking: Differentiable End-to-End Reranker Optimization , author =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. doi:10.18653/v1/2025.acl-long.354 , year =

    work page doi:10.18653/v1/2025.acl-long.354 2025

  33. [33]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    End-to-End Learning of Visual Representations from Uncurated Instructional Videos , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  34. [34]

    doi:10.1007/978-3-031-72980-5_24 , year =

    Zare, Ali and Niu, Yulei and Ayyubi, Hammad and Chang, Shih-Fu , booktitle =. doi:10.1007/978-3-031-72980-5_24 , year =

    work page doi:10.1007/978-3-031-72980-5_24

  35. [35]

    Li, Xinjie , howpublished =

  36. [36]

    doi:10.1145/3474085.3475281 , year =

    Hou, Zhijian and Ngo, Chong-Wah and Chan, Wing Kwong , booktitle =. doi:10.1145/3474085.3475281 , year =

    arXiv

  37. [37]

    doi:10.1109/TPAMI.2025.3614169 , year =

    Chen, Xianke and Liu, Daizong and Yang, Xun and Li, Xirong and Dong, Jianfeng and Wang, Meng and Wang, Xun , journal =. doi:10.1109/TPAMI.2025.3614169 , year =

    arXiv 2025

  38. [38]

    IEEE Transactions on Multimedia , pages =

    Towards Efficient Partially Relevant Video Retrieval with Active Moment Discovering , author =. IEEE Transactions on Multimedia , pages =. doi:10.1109/TMM.2025.3590937 , year =

    arXiv 2025

  39. [39]

    International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =

    Video Corpus Moment Retrieval with Contrastive Learning , author =. International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =

  40. [40]

    ACM International Conference on Multimedia Retrieval , pages =

    Improving Video Corpus Moment Retrieval with Partial Relevance Enhancement , author =. ACM International Conference on Multimedia Retrieval , pages =. doi:10.1145/3652583.3658088 , year =

    arXiv

  41. [41]

    European Conference on Information Retrieval , series =

    Event-Aware Video Corpus Moment Retrieval , author =. European Conference on Information Retrieval , series =. doi:10.1007/978-3-032-21289-4_1 , year =

    work page doi:10.1007/978-3-032-21289-4_1

  42. [42]

    TimeLoc: A Unified End-to-End Framework for Precise Timestamp Localization in Long Videos , author =

  43. [43]

    CAST: Modeling Visual State Transitions for Consistent Video Retrieval , author =

  44. [44]

    International Joint Conference on Artificial Intelligence , year =

    Denoise-then-Retrieve: Text-Conditioned Video Denoising for Video Moment Retrieval , author =. International Joint Conference on Artificial Intelligence , year =

  45. [45]

    Measure Twice, Cut Once: A Semantic-Oriented Approach to Video Temporal Localization with Video LLMs , author =

  46. [46]

    Advances in Neural Information Processing Systems , year =

    MAGNET: A Multi-Agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks , author =. Advances in Neural Information Processing Systems , year =

  47. [47]

    Conference on Empirical Methods in Natural Language Processing , pages =

    Show and Guide: Instructional-Plan Grounded Vision and Language Model , author =. Conference on Empirical Methods in Natural Language Processing , pages =. doi:10.18653/v1/2024.emnlp-main.1191 , year =

    doi:10.18653/v1/2024.emnlp-main.1191 2024

  48. [48]

    doi:10.52202/079017-1278 , year =

    Chen, Houlun and Wang, Xin and Chen, Hong and Zhang, Zeyang and Feng, Wei and Huang, Bin and Jia, Jia and Zhu, Wenwu , booktitle =. doi:10.52202/079017-1278 , year =

    work page doi:10.52202/079017-1278

  49. [49]

    IEEE International Conference on Acoustics, Speech and Signal Processing , pages =

    Learning to Locate Visual Answer in Video Corpus Using Question , author =. IEEE International Conference on Acoustics, Speech and Signal Processing , pages =. doi:10.1109/ICASSP49357.2023.10096391 , year =

    arXiv 2023

  50. [50]

    Zhang, Chen-Lin and Wu, Jianxin and Li, Yin , booktitle =

  51. [51]

    Shi, Dingfeng and Zhong, Yujie and Cao, Qiong and Ma, Lin and Li, Jia and Tao, Dacheng , booktitle =

  52. [52]

    Lin, Tianwei and Liu, Xiao and Li, Xin and Ding, Errui and Wen, Shilei , booktitle =

  53. [53]

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , booktitle =

  54. [54]

    Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90\ author =

  55. [55]

    IEEE Conference on Computer Vision and Pattern Recognition , pages =

    Unsupervised Learning from Narrated Instruction Videos , author =. IEEE Conference on Computer Vision and Pattern Recognition , pages =

  56. [56]

    Tang, Yansong and Ding, Dajun and Rao, Yongming and Zheng, Yu and Zhang, Danyang and Zhao, Lili and Lu, Jiwen and Zhou, Jie , booktitle =

  57. [57]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Cross-Task Weakly Supervised Learning from Instructional Videos , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  58. [58]

    AAAI Conference on Artificial Intelligence , year =

    Towards Automatic Learning of Procedures from Web Instructional Videos , author =. AAAI Conference on Artificial Intelligence , year =

  59. [59]

    Transactions of the Association for Computational Linguistics , volume =

    Grounding Action Descriptions in Videos , author =. Transactions of the Association for Computational Linguistics , volume =

  60. [60]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Hierarchical Video-Moment Retrieval and Step-Captioning , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  61. [61]

    Grauman, Kristen and Westbury, Andrew and Byrne, Eugene and Chavis, Zachary and Furnari, Antonino and Girdhar, Rohit and Hamburger, Jackson and Jiang, Hao and Liu, Miao and Liu, Xingyu and others , booktitle =

  62. [62]

    IEEE International Conference on Computer Vision , pages =

    Dense-Captioning Events in Videos , author =. IEEE International Conference on Computer Vision , pages =

  63. [63]

    International Conference on Machine Learning , pages =

    Learning Transferable Visual Models from Natural Language Supervision , author =. International Conference on Machine Learning , pages =

  64. [64]

    Wang, Yi and Li, Kunchang and Li, Yizhuo and He, Yinan and Huang, Bingkun and Zhao, Zhiyu and Zhang, Hongjie and Xu, Jilan and Liu, Yi and Wang, Zun and others , howpublished =

  65. [65]

    Mur-Labadia, Lorenzo and Muckley, Matthew and Bar, Amir and Assran, Mido and Sinha, Koustuv and Rabbat, Mike and LeCun, Yann and Ballas, Nicolas and Bardes, Adrien , howpublished =

  66. [66]

    Li, Jingyao and Wang, Jingyun and Tan, Molin and Wang, Haochen and Yan, Cilin and Shi, Likun and Cai, Jiayin and Jiang, Xiaolong and Hu, Yao , booktitle =

  67. [67]

    Seminara, Luigi and Furnari, Antonino and Torresani, Lorenzo , howpublished =

  68. [68]

    Yan, Pengyu and Gorugantu, Akhil and Bhosale, Mahesh and Wasi, Abdul and Trivedi, Vishvesh and Doermann, David , howpublished =

  69. [69]

    Ge, Mingji and Chen, Qirui and Li, Zeqian and Xie, Weidi , howpublished =

  70. [70]

    Yuan, Huaying and Liu, Zheng and Zhou, Junjie and Qian, Hongjin and Shu, Yan and Sebe, Nicu and Wen, Ji-Rong and Dou, Zhicheng , howpublished =

  71. [71]

    Enhancing Visual Planning with Auxiliary Tasks and Multi-token Prediction , author =

  72. [72]

    Planning with Reasoning using Vision Language World Model , author =

  73. [73]

    Tian, Shulin and Wang, Ruiqi and Guo, Hongming and Wu, Penghao and Dong, Yuhao and Wang, Xiuying and Yang, Jingkang and Zhang, Hao and Zhu, Hongyuan and Liu, Ziwei , howpublished =

  74. [75]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages=

    Skip-Plan: Procedure Planning in Instructional Videos via Condensed Action Space Learning , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. doi:10.1109/ICCV51070.2023.00945 , year=

    arXiv 2023

  75. [76]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    A Language-First Approach for Procedure Planning , author=. Findings of the Association for Computational Linguistics: ACL 2023 , pages=. doi:10.18653/v1/2023.findings-acl.122 , year=

    work page doi:10.18653/v1/2023.findings-acl.122 2023

  76. [77]

    Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages=

    GUIDE: A Guideline-Guided Dataset for Instructional Video Comprehension , author=. Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages=. doi:10.24963/ijcai.2024/118 , year=

    work page doi:10.24963/ijcai.2024/118 2024

  77. [79]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. doi:10.1109/CVPR52733.2024.02084 , year=

    arXiv 2024

  78. [80]

    Scientific Data , volume=

    A Dataset for Medical Instructional Video Classification and Question Answering , author=. Scientific Data , volume=. doi:10.1038/s41597-023-02036-y , year=

    doi:10.1038/s41597-023-02036-y

  79. [81]

    IEEE Conference on Computer Vision and Pattern Recognition , pages =

    The Language of Actions: Recovering the Syntax and Semantics of Goal-Directed Human Activities , author =. IEEE Conference on Computer Vision and Pattern Recognition , pages =

  80. [82]

    Rescaling Egocentric Vision: Collection, Pipeline and Challenges for

    Damen, Dima and Doughty, Hazel and Farinella, Giovanni Maria and Furnari, Antonino and Kazakos, Evangelos and Ma, Jian and Moltisanti, Davide and Munro, Jonathan and Perrett, Toby and Price, Will and Wray, Michael , journal =. Rescaling Egocentric Vision: Collection, Pipeline and Challenges for. doi:10.1007/s11263-021-01531-2 , year =

    doi:10.1007/s11263-021-01531-2

Showing first 80 references.