Pith. sign in

REVIEW 3 major objections 4 minor 36 references

ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual-Language Models through Procedural Plans

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that frontier vision-language models score near chance on a new benchmark, ISO-Bench, for deciding whether a visual task step must happen before or after a textual plan step.

desk verdict ISO-Bench fills a real gap and the dataset is new, but its DEP/NONDEP labels encode step-in-snippet membership rather than causal dependence, so the causal-reasoning claim collapses into image-text grounding. read the letter →

arxiv 2507.23135 v1 pith:U4QCKXJF submitted 2025-07-30 cs.CL

Ananya Sadana , Yash Kumar Lal , Jiawei Zhou This is my paper · ORCID
classification cs.CL
keywords ISO-Benchmultimodalcausalreasoningproceduralplansvision-languagemodelstemporaldependencieschain-of-thoughtinstructionalvideosbenchmarkconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that current vision-language models cannot reliably infer causal dependencies between what an image shows and what a procedural text says: given a frame of a task step and a snippet from a plan, the best open and proprietary models sit at or near chance (best zero-shot F1 0.57, best chain-of-thought F1 0.62) while humans reach 0.98. To make the point, the authors construct ISO-Bench, a manually vetted set of 764 image-snippet pairs drawn from two instructional video datasets, with binary before/after dependency labels. The result matters because real-world plan-following, video understanding, and embodied agents all require grounding a visual state in the causal structure of a plan; if the finding holds, this ability is a distinct deficit of current multimodal models, and explanation prompting does not close it.

What carries the argument

The load-bearing object is ISO-Bench itself. For each video, text snippets are the first or last k steps (k in {2,3,4}); an image is sampled from a different step; the question is whether the image step must occur before or after the snippet. Gold labels come from two pools: 'positive' frames taken just outside the snippet (within three steps) are labeled dependent, and 'negative' frames taken from inside the snippet are labeled non-dependent. A human vetting step prunes ambiguous pairs, and the explain-then-answer evaluation adds a chain-of-thought explanation before the binary answer, but the core machinery is the contrast between these two pools.

What would settle it

Run a retrieval-style control that answers 'yes' whenever the image shows a step listed in the snippet and 'no' otherwise; if that control reaches or exceeds the reported 0.57 zero-shot F1 on ISO-Bench, the gold labels are predictable from step identity, and the benchmark does not isolate causal reasoning.

Watch

Extended reading notes

Core claim

ISO-Bench presents a model with an image of one step from an instructional video, a text snippet from the plan (the first or last two to four steps), and a binary question: must the step in the image come before or after the snippet? A 'yes' answer means the image step is causally dependent on the snippet's steps, while 'no' means independent. Built from YouCook2 and CrossTask and human-vetted, the benchmark contains 764 examples with roughly equal dependent and non-dependent instances. Across ten evaluated models, the strongest zero-shot F1 is 0.57 and the best explain-then-answer F1 is 0.62, while humans reach 0.98; error analysis attributes most failures to missing causal reasoning rather than to perception.

Load-bearing premise

The ground-truth labels are assigned by temporal position — a frame just outside the text snippet counts as causally dependent, and a frame from inside the snippet counts as not — so the benchmark measures causal reasoning only if those positional categories are perfect proxies for causal dependence.

Editorial extensions

If this is right

  • No evaluated vision-language model reliably answers ISO-Bench questions; even the best proprietary model is closer to chance than to human performance.
  • Explanation-then-answer prompting changes F1 by at most about 0.06 for the stronger models and hurts some open-source models, indicating the deficit is not caused by answer formatting.
  • Humans at 0.98 F1 establish that the task is solvable from the same image and text, placing the gap in the models rather than in the benchmark's premise.
  • Error analysis finds 62% of incorrect answers come from missing precondition/effect reasoning, so improving causal dependency inference is the priority over improving perception.
  • The best-performing models tend to over-predict dependence, showing a systematic bias rather than random failure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A model that simply recognizes which plan step an image shows could reproduce the gold labels without causal reasoning, because labels are assigned by temporal position (outside vs inside the snippet) rather than by explicit causal relations; a stricter test would annotate preconditions and effects directly.
  • A natural extension is to fine-tune models on procedural dependency graphs or multi-frame progress signals; the paper's own error categories suggest such training would mainly target the 62% causal-reasoning failures.
  • If the same construction were applied to non-English plans or additional domains, a persistent gap would support the claim that cross-modal causal reasoning is a general VLM weakness rather than an artifact of English cooking videos.
  • A more stringent benchmark variant would require the model to state the actual precondition or effect linking the image to the snippet, not merely a binary before/after judgment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. ISO-Bench is a proposed benchmark for evaluating whether vision-language models can infer causal dependencies between an image showing a single task step and a text snippet from a procedural plan. The paper constructs 764 examples from YouCook2 and CrossTask, assigns each example a binary DEP/NONDEP label, evaluates ten VLMs under answer-only and explain-then-answer prompting, and reports near-chance F1 scores for most models, modest gains from chain-of-thought, and a 0.98 human F1 baseline. It also analyzes performance by step distance and temporal direction and provides a qualitative error taxonomy. The central claim is that these results demonstrate a specific deficit in cross-modal causal reasoning.

Significance. If the benchmark measured what it claims, the finding that frontier VLMs perform only slightly above chance on cross-modal causal dependency in procedural plans while humans achieve 0.98 F1 would be an important result for multimodal reasoning, embodied AI, and plan understanding. The paper has several strengths: it evaluates a broad set of ten open and proprietary models, provides a human baseline, reports per-class and macro metrics, and includes a structured error analysis. However, the central construct is not operationalized by the dataset construction: the gold labels are determined by whether the image step lies outside or inside the text snippet, not by causal precondition/effect relations. This invalidates the headline claim and makes the reported scores ambiguous, because the task can be shortcut through image-text grounding and temporal position matching. The paper's contribution is therefore better described as a study of visual grounding and ordering of procedural steps, not of causal reasoning.

major comments (3)
  1. [§3 and Appendix B] The core construct is not operationalized. Section 3 states that the binary label indicates 'whether there is a causal dependence between the image step and any step in the text,' but Appendix B constructs the two pools by temporal position: the positive pool contains 'frames whose step comes just outside the snippet, within three steps,' and the negative pool contains 'frames taken inside the snippet itself.' A step that lies just before or after the snippet need not be a causal precondition or effect of any snippet step, and a step inside the snippet can be a precondition for another step inside the same snippet. The gold answer is therefore fully determined by pool membership, so a model can solve the task by grounding the image to one of the steps in the text snippet and checking whether that step is inside the snippet, without any inference about preconditions or effects.
  2. [Appendix B and Appendix E] The annotation procedure verifies pool membership, not causal dependence. The Streamlit app asks annotators to accept a POS if the frame 'truly matches' and a NEG if it 'truly conflicts,' i.e., it checks whether the frame agrees with the requested temporal relation. The human baseline in Appendix E inherits these labels; the 0.98 F1 therefore shows that humans can verify whether a frame belongs to the prescribed side of the snippet, not that independent causal judgments agree with the gold labels. No second annotation pass, adjudication, or independent validation step establishes that the labels correspond to genuine causal precondition/effect relations.
  3. [§3 and Appendix H] The question template 'Must the step in the image happen before/after the snippet?' conflates temporal necessity with causal dependence. Because every positive-pool frame is constructed to be on the requested side of the snippet, the answer 'yes' is correct for all positive examples by construction, regardless of whether the step is causally related to any snippet step. The empirical finding that models score near chance is therefore ambiguous: it may indicate a failure to determine the step's position in the plan or a failure to ground the image to the correct step, rather than a failure of causal reasoning. The analysis in §5.3 does not resolve this ambiguity because the error categories assume the gold labels are causal.
minor comments (4)
  1. [Appendix H] The QUESTION placeholder reads '<before|after> the <first|last> step in the plan,' whereas Section 3 and Appendix B describe comparison to the text snippet. This inconsistency makes it unclear what question the models actually answered.
  2. [Appendix B] 'For each such video it randomly picked an integer k between 2 and 4' has a subject-verb agreement issue, and the random selection of k is not described in the main text, leaving the reader to infer the snippet-size distribution.
  3. [Appendix F] There are typographical errors such as 'shown to demonstrate enhanced performance,' 'high-resoultion,' and 'seperately' that should be corrected.
  4. [Table 1] The table caption says 'weighted macro average,' but the table reports unweighted macro average; the metric definition in Section 4.1 should be aligned with what is actually reported.

Circularity Check

1 steps flagged · score 6.0 of 10

ISO-Bench's gold labels are generated from temporal pool membership, so the benchmark's causal-reasoning claim is self-definitional rather than independently tested.

  1. self definitional [Section 3 (Benchmark Construction); Appendix B (Processing YouCook2); Appendix C (Processing CrossTask).]
    "We define two candidate frame groups: (1) Positive pool – frames whose step comes just outside the snippet, within three steps ... These frames genuinely belong before or after the snippet, so pairing them with the snippet should be logically correct. (2) Negative pool – frames taken inside the snippet itself ... showing them separately violates the required temporal relation and therefore constitutes an incorrect match. ... The app shows the snippet, the frame, and whether the sample is tagged POS ... or NEG ..."

    By construction, the gold label is read off pool membership: an image from a step inside the text snippet is NEG/NONDEP, and an image from a step just outside the snippet (within three positions) is POS/DEP. The 'causal dependence' label is therefore defined by positional containment or adjacency, not by any precondition/effect relation between the image step and the snippet steps. A model can answer correctly by grounding the image to a step inside versus outside the text span; no inference about causes, preconditions, or effects is needed. The abstract's claim that ISO-Bench tests whether models 'infer causal dependencies between visual observations and procedural text' is thus baked into the label generator.

full rationale

The paper does not fit parameters or derive predictions from equations, so there is no fitted-input circularity and no load-bearing self-citation. The circularity is at the level of construct definition: Appendix B defines DEP/NONDEP gold labels by whether the image frame comes from a step just outside the snippet (Positive pool) or inside the snippet (Negative pool), and the annotator's job is to verify that pool membership ('truly matches' / 'truly conflicts'). The causal-dependency claim from the abstract and Section 3 is therefore not an independent property established for each pair; it is assigned by the same positional rule that generates the question. Consequently, the reported failure of VLMs (best F1 0.57; 0.62 with CoT) and the human 0.98 baseline are measures of whether the model can tell that an image belongs to a step inside versus adjacent to the text span, not of cross-modal causal inference about preconditions and effects. This makes the benchmark's central evaluative claim self-definitional, though the raw benchmark construction and evaluation pipeline are otherwise transparent and reproducible.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the source datasets' step order being causal, on the midpoint frames representing steps, and on the assumption that POS/NEG temporal pools equal DEP/NONDEP causal labels. The third is the load-bearing assumption and appears false, which drives the low soundness score.

free parameters (2)
  • Text snippet length k = 2, 3, or 4 (randomly selected)
    The authors randomly pick k between 2 and 4 for each video, which controls how much plan context is shown. This is a design choice rather than a fitted value.
  • Step-distance threshold for close vs far analysis = 3
    Used in Section 5.1 to split questions into close and far steps; a hand-set analysis boundary, not fitted to data.
assumptions (3)
  • domain assumption The order of annotated steps in YouCook2 and CrossTask reflects the true causal or temporal order of the procedure.
    Section 3 builds labels directly from these step annotations; if the source plan order is wrong or non-causal, the benchmark labels inherit the error.
  • ad hoc to paper A frame taken from a step just outside the text snippet is causally dependent on the snippet, and a frame taken from inside the snippet is not.
    Appendix B defines positive and negative pools by temporal position relative to the snippet, but the paper claims the labels encode causal dependence. This assumption is load-bearing and appears false, since a step can be adjacent in time yet causally independent.
  • domain assumption The midpoint frame of a step adequately conveys the state of that step.
    Appendix B saves one JPEG at the midpoint of each step span; if the midpoint does not show the action, the image-text grounding fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual-Language Models through Procedural Plans." pith.science (2026). https://pith.science/paper/U4QCKXJF

@misc{pith2026250723135,
  author       = {Pith},
  title        = {Pith review of: ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual-Language Models through Procedural Plans},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U4QCKXJF}},
  note         = {Machine review of arXiv:2507.23135}
}
read the original abstract

Understanding causal relationships across modalities is a core challenge for multimodal models operating in real-world environments. We introduce ISO-Bench, a benchmark for evaluating whether models can infer causal dependencies between visual observations and procedural text. Each example presents an image of a task step and a text snippet from a plan, with the goal of deciding whether the visual step occurs before or after the referenced text step. Evaluation results on ten frontier vision-language models show underwhelming performance: the best zero-shot F1 is only 0.57, and chain-of-thought reasoning yields only modest gains (up to 0.62 F1), largely behind humans (0.98 F1). Our analysis further highlights concrete directions for improving causal understanding in multimodal models.

Figures

Figures reproduced from arXiv: 2507.23135 by the authors.

Figure 1
Figure 1. An example plan, visual states, and their [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Difference in performance of models between [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Screenshot with instructions provided to annotators from whom we collected gold labels for [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 22 canonical work pages

  1. [1]

    Antoine Bosselut, Omer Levy, Ari Holtzman, Corin Ennis, Dieter Fox, and Yejin Choi. 2018. Simulating action dynamics with neural process networks. ICLR

    work page 2018

  2. [2]

    Oana-Maria Camburu, Tim Rockt\" a schel, Thomas Lukasiewicz, and Phil Blunsom. 2018. https://proceedings.neurips.cc/paper_files/paper/2018/file/4c7a167bb329bd92580a99ce422d6fa6-Paper.pdf e-snli: Natural language inference with natural language explanations . In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc

    work page 2018

  3. [3]

    Bhavana Dalvi, Lifu Huang, Niket Tandon, Wen-tau Yih, and Peter Clark. 2018. https://doi.org/10.18653/v1/N18-1144 Tracking state changes in procedural text: a challenge dataset and models for process paragraph comprehension . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Langua...

    doi:10.18653/v1/n18-1144 2018

  4. [4]

    Bhavana Dalvi, Niket Tandon, Antoine Bosselut, Wen-tau Yih, and Peter Clark. 2019. https://doi.org/10.18653/v1/D19-1457 Everything happens for a reason: Discovering the purpose of actions in procedural text . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lang...

    work page doi:10.18653/v1/d19-1457 2019

  5. [5]

    Aissatou Diallo, Antonis Bikakis, Luke Dickens, Anthony Hunter, and Rob Miller. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.728 P izza C ommon S ense: A dataset for commonsense reasoning about intermediate steps in cooking recipes . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12482--12496, Miami, Florida, USA....

    work page doi:10.18653/v1/2024.findings-emnlp.728 2024

  6. [6]

    Lucia Donatelli, Theresa Schmidt, Debanjali Biswas, Arne K \"o hn, Fangzhou Zhai, and Alexander Koller. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.554 Aligning actions across recipe graphs . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6930--6942, Online and Punta Cana, Dominican Republic. Associa...

    work page doi:10.18653/v1/2021.emnlp-main.554 2021

  7. [7]

    Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. 2024. Mmbench-video: A long-form multi-shot benchmark for holistic video understanding. Advances in Neural Information Processing Systems, 37:89098--89124

    work page 2024

  8. [8]

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. 2025. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 24108--24118

    work page 2025

Show all 36 references
  1. [9]

    Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. 2017. Tracking the world state with recurrent entity networks. ICLR

    2017

  2. [10]

    Chlo \'e Kiddon, Ganesa Thandavam Ponnuraj, Luke Zettlemoyer, and Yejin Choi. 2015. https://doi.org/10.18653/v1/D15-1114 Mise en place: Unsupervised interpretation of instructional recipes . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Process...

    2015 doi

  3. [11]

    Sawan Kumar and Partha Talukdar. 2020. https://doi.org/10.18653/v1/2020.acl-main.771 NILE : Natural language inference with faithful natural language explanations . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8730--8742, On...

    2020 doi

  4. [12]

    Saksham Lal, Mizuki Morita, Yingya Li, Roberto Rossi, and Peter Lane. 2024. http://arxiv.org/abs/2406.15823 CaT - Bench : Benchmarking language-model understanding of causal and temporal dependencies in plans . arXiv preprint, arXiv:2406.15823. To appear in EMNLP 2024

    2024 arXiv

  5. [13]

    Angela Lin, Sudha Rao, Asli Celikyilmaz, Elnaz Nouri, Chris Brockett, Debadeepta Dey, and Bill Dolan. 2020. https://doi.org/10.18653/v1/2020.acl-main.440 A recipe for creating multimodal aligned datasets for sequential tasks . In Proceedings of the 58th Annual Meeting of the A...

    2020 doi

  6. [14]

    Sebastian Seung

    Haotian Liu, Chunyuan Li, Qingyang Wu, Yin Li, Jason Baldridge, and H. Sebastian Seung. 2023. https://arxiv.org/abs/2310.03744 LLaVA - 1.5 : Visual instruction tuning for large language-and-vision assistants . In Advances in Neural Information Processing Systems ( NeurIPS )

    2023 arXiv

  7. [15]

    Koki Maeda, Tosho Hirasawa, Atsushi Hashimoto, Jun Harashima, Leszek Rybicki, Yusuke Fukasawa, and Yoshitaka Ushiku. 2024. Com kitchens: An unedited overhead-view video dataset as a vision-language benchmark. In Proceedings of the European Conference on Computer Vision

    2024

  8. [16]

    Dai Quoc Nguyen, Dat Quoc Nguyen, Cuong Xuan Chu, Stefan Thater, and Manfred Pinkal. 2017. https://aclanthology.org/I17-2007/ Sequence to sequence learning for event prediction . In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume...

    2017

  9. [17]

    Liang-Ming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, Yugang Jiang, and Tat-Seng Chua. 2020. https://doi.org/10.1145/3394171.3413765 Multi-modal cooking workflow construction for food recipes . In Proceedings of the 28th ACM International Confer...

    2020

  10. [18]

    Paolo Pareti, Benoit Testu, Ryutaro Ichise, Ewan Klein, and Adam Barker. 2014. Integrating know-how into the linked data cloud. In International Conference on Knowledge Engineering and Knowledge Management, pages 385--396. Springer

    2014

  11. [19]

    Jae Sung Park, Ilia Kulikov, Chandra Bhagavatula, and Yejin Choi. 2020. https://arxiv.org/abs/2003.12506 Visualcomet: Reasoning about the dynamic context of a still image . In European Conference on Computer Vision ( ECCV )

    2020 arXiv

  12. [20]

    Ziyao Shangguan, Jiarui Qiu, Andrei Barbu, David Fouhey, and Song - Chang Huang. 2024. http://arxiv.org/abs/2410.23266 TOMATO : Assessing visual temporal reasoning capabilities in multimodal foundation models . arXiv preprint, arXiv:2410.23266

    2024 arXiv

  13. [21]

    Karthik Valmeekam, Matthew Marquez, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2023 a . https://arxiv.org/abs/2206.10498 Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change . Preprint, arXiv:2206.10498

    2023 arXiv

  14. [22]

    Karthik Valmeekam, Kartik Talamadupula, and Sanjay Srivastava. 2023 b . https://openreview.net/forum?id=wUU-7XTL5XO PlanBench : An extensible benchmark for evaluating large language models on planning and reasoning about change . In NeurIPS Datasets & Benchmarks Track

    2023

  15. [23]

    Chi, Quoc V Le, and Denny Zhou

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022. https://openreview.net/forum?id=_VjQlMeSB_J Chain of thought prompting elicits reasoning in large language models . In Advances in Neural Information Proc...

    2022

  16. [24]

    Te-Lin Wu, Alex Spangher, Pegah Alipoormolabashi, Marjorie Freedman, Ralph Weischedel, and Nanyun Peng. 2022. https://doi.org/10.18653/v1/2022.acl-long.310 Understanding multimodal procedural knowledge by sequencing multimodal instructional manuals . In Proceedings of the 60th...

    2022 doi

  17. [25]

    Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler - Cinbis. 2018. https://arxiv.org/abs/1810.06553 RecipeQA : A challenge dataset for multimodal comprehension of cooking recipes . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Process...

    2018 arXiv

  18. [26]

    Jinyoung Yeo, Gyeongbok Lee, Gengyu Wang, Seungtaek Choi, Hyunsouk Cho, Reinald Kim Amplayo, and Seung-won Hwang. 2018. https://aclanthology.org/L18-1316/ Visual choice of plausible alternatives: An evaluation of image-based commonsense causal reasoning . In Proceedings of the...

    2018

  19. [27]

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. 2024. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conferenc...

    2024

  20. [28]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. https://doi.org/10.18653/v1/P19-1472 H ella S wag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791--4...

    2019 doi

  21. [29]

    Hongming Zhang, Muhao Chen, Haoyu Wang, Yangqiu Song, and Dan Roth. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.119 Analogous process structure induction for sub-event sequence prediction . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Pr...

    2020 doi

  22. [30]

    Li Zhang, Hainiu Xu, Yue Yang, Shuyan Zhou, Weiqiu You, Manni Arora, and Chris Callison-Burch. 2023. https://doi.org/10.18653/v1/2023.findings-eacl.31 Causal reasoning of entities and events in procedural texts . In Findings of the Association for Computational Linguistics: EA...

    2023 doi

  23. [31]

    Zheyuan Zhang, Fengyuan Hu, Jayjun Lee, Freda Shi, Parisa Kordjamshidi, Joyce Chai, and Ziqiao Ma. 2025. https://openreview.net/forum?id=84pDoCD4lH Do vision-language models represent space and how? evaluating spatial frame of reference under ambiguities . In The Thirteenth In...

    2025

  24. [32]

    Liu, Julian Michael, and Mohit Iyyer

    Huaixiu Steven Zheng, Neel Kant, Rui Wang, Peter J. Liu, Julian Michael, and Mohit Iyyer. 2024. http://arxiv.org/abs/2406.04520 NaturalPlan : Benchmarking LLMs on natural-language planning . arXiv preprint, arXiv:2406.04520

    2024 arXiv

  25. [33]

    Corso, Richard Socher, and Caiming Xiong

    Luowei Zhou, Chenliang Xu, Jason J. Corso, Richard Socher, and Caiming Xiong. 2018. https://openaccess.thecvf.com/content_cvpr_2018/papers/Zhou_Towards_Automatic_Learning_CVPR_2018_paper.pdf Towards automatic learning of procedures from web instructional videos . In Proceeding...

    2018

  26. [34]

    Dimitri Zhukov, Jean-Baptiste Alayrac, Ramazan Gokberk Cinbis, David Fouhey, Ivan Laptev, and Josef Sivic. 2019. Cross-task weakly supervised learning from instructional videos. In CVPR

    2019

  27. [35]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  28. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.