Pith. sign in

REVIEW 3 major objections 6 minor 45 references

Predicting Implicit Arguments in Procedural Video Instructions

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Procedural instructions are full of unstated ingredients, and this paper shows that a small fine-tuned model can infer them better than GPT-4o.

desk verdict A solid dataset paper with a plausible headline result, but the gold-label construction leaves a style-matching circularity that the authors should address before publication. read the letter →

arxiv 2505.21068 v1 pith:73QCGAUO submitted 2025-05-27 cs.CL cs.CV

Anil Batra , Laura Sevilla-Lara , Marcus Rohrbach , Frank Keller This is my paper · ORCID
classification cs.CLcs.CV
keywords semanticrolelabelingimplicitargumentsproceduralvideounderstandingmultimodallargelanguagemodelsentitytrackingcookingdatasetsnext-steppredictionsilver-standardannotations
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Cookbook instructions omit things constantly: "cook and cut it into cubes" means cook the oiled pita bread you just made, not some unspecified object. This paper argues that this kind of implicit-argument inference is a real, testable capability that current multimodal models lack, and that it can be learned. To show this, the authors build Implicit-VidSRL, a semantic-role-labeling dataset of 231 cooking videos annotated as {verb, what, where/with} frames that mark which ingredients must be carried over from earlier steps or clips. They then fine-tune a 7B model, iSRL-Qwen2-VL, on silver-standard labels generated by GPT-4o, and report that it outperforms GPT-4o by 17% relative F1 on implicit what-arguments and 14.7% on implicit where/with-arguments. If this holds, it means a small open-weights model can track entities through procedural video as well as or better than a much larger closed model, a step toward assistants that understand instructions the way a sous-chef would.

What carries the argument

The load-bearing object is the semantic frame {verb, what, where/with}, a deliberately small role set for procedural steps: the verb is given, "what" names the objects the action affects, and "where/with" names the location or companion ingredients. The key move is that these arguments can be implicit — an entity inherited from an earlier step, possibly changed by cooking, like "pork with spices" or "the dressing" decomposed into its ingredient list. On top of this representation sit two evaluation tasks: a cloze task that masks the what and where/with arguments and asks the model to fill them given the verb and the preceding multimodal context, and a next-step prediction task that must generate the future instruction plus its semantic frames. Training signal comes from a silver-standard dataset: GPT-4o, prompted with chain-of-thought instructions and five in-context examples, converts Tasty instructions into frames with implicit arguments, producing roughly 2.5K training videos and 18K next-step samples, on which the 7B model is LoRA fine-tuned.

What would settle it

A concrete check is to re-annotate the evaluation set from scratch with fresh human annotators who never see any GPT-4o output, then rerun iSRL-Qwen2-VL against GPT-4o on those labels; if the relative gains shrink or vanish, the silver-training hypothesis is the problem. A second check is to train the same model on silver labels generated by a different model family and see whether the improvements replicate.

Watch

Extended reading notes

Core claim

The paper's central claim is that procedural instructions are systematically elliptical, and that a model which can fill in the missing argument — the "implicit argument" — has genuinely understood the procedure. The authors encode each instruction as one or more semantic frames of the form {verb, what, where/with}, where arguments marked implicit are ingredients named in earlier steps or visible in earlier clips but absent from the current sentence. On their new benchmark, Implicit-VidSRL, built from YouCook2 and Tasty videos, they find that strong multimodal models including GPT-4o and Qwen2-VL guess explicit ingredients decently but lose track of implicit ones, especially in later steps of long recipes. The proposed remedy, iSRL-Qwen2-VL, is a 7B Qwen2-VL model fine-tuned with LoRA on GPT-4o-generated silver SRL annotations; it closes most of the gap to the larger models and, with video-plus-text input, beats GPT-4o on the implicit-argument metrics. The paper also claims that emitting semantic frames as an intermediate representation improves next-step prediction, adding about 2% METEOR over the base model.

Load-bearing premise

The load-bearing premise is that the automatically generated silver-standard labels are accurate enough to teach genuine entity tracking, and that recipe steps really do decompose into simple verb-what-where frames; if the labels carry GPT-4o's biases, or the frame decomposition is wrong for other procedures, the reported gains would not reflect real procedural understanding.

Editorial extensions

If this is right

  • If the reported gains are correct, a 7B open-weights model fine-tuned on silver-standard labels can match or beat GPT-4o at implicit-argument prediction, so access to a large proprietary model is not the binding constraint for this capability.
  • Emitting {verb, what, where/with} frames while predicting the next step improves future-step prediction over the same model without frames, so SRL acts as an effective intermediate representation, not just an evaluation target.
  • Multimodal input (video plus text) helps only when combined with SRL training; untrained models stay biased toward surface text, so visual dynamics must be learned rather than assumed.
  • Longer recipes degrade implicit-argument scores for all tested models, meaning entity tracking across state changes is the limiting skill, and benchmarks should report performance by step position.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to swap the label generator: train on SRLs produced by a different LLM and verify the F1 gains survive; if they do not, part of the reported advantage is overfitting to GPT-4o's annotation style rather than to the implicit-argument facts.
  • The same silver-label recipe could transfer to other procedural domains with comparable ellipsis, such as assembly, medical, or craft instructions, where manual annotation would be far costlier.
  • The finding that both total entities and implicit arguments grow at later semantic-frame positions suggests that entity tracking, not verb semantics, is the limiting difficulty, so a natural next step is to vary the carry-over distance — how many steps back an ingredient must be retrieved — and measure where each model starts dropping entities.
  • Used as a probe, the cloze task could serve as a cheap diagnostic for long-context video understanding, since it isolates whether a model can track an entity across frames without requiring full sentence generation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Implicit-VidSRL, a dataset of 231 cooking videos from YouCook2 and Tasty annotated with semantic role labels ({verb, what, where/with}) that include implicit arguments inferred from textual and visual context. The authors define two tasks: a cloze-style implicit argument prediction task and a next-step prediction task. They evaluate several proprietary and open-source multimodal LLMs on these tasks and propose iSRL-Qwen2-VL, a Qwen2-VL model fine-tuned on silver-standard SRL data automatically generated by GPT-4o from Tasty instructions. The paper reports that iSRL-Qwen2-VL achieves 17% relative F1 improvement for what-implicit and 14.7% for where/with-implicit over GPT-4o with multimodal input, as well as modest gains in next-step prediction.

Significance. If the benchmark is valid, this is a useful contribution: it targets an underexplored problem (implicit arguments in procedural video), provides a public dataset and code, includes extensive experiments across multiple models and modalities, and shows that fine-tuning on automatically generated SRL data can improve implicit argument prediction. The paper is generally well written and the experimental setup is clearly described. The main weakness is that the validity of the gold-standard labels is not yet established, which is load-bearing for the central claim.

major comments (3)
  1. [Section 3 (Data Annotation) and Section 5 (Silver-standard Dataset)] The gold-standard test labels are produced by GPT-4o-Mini pre-annotation followed by manual correction by a single annotator, while the silver-standard training data are generated by GPT-4o using a nearly identical chain-of-thought prompt (Figure 11 vs. Figure 8). This creates a concrete risk of circularity: the fine-tuned model may learn to reproduce GPT-4o's decisions about which entities to treat as implicit, how to decompose composite entities, and how to fill WHERE_OR_WITH, and the test set may have been nudged toward the same style through the GPT-4o-Mini pre-annotation that the annotator corrects. The reported gain of iSRL-Qwen2-VL over GPT-4o zero-shot in Table 2 could therefore be inflated by style-matching rather than genuine procedural understanding. The authors should provide evidence that the gold labels are not biased in this way, for example by releasing an independently human-annotated test subset (without model pre-annotation) and showing the improvement persists, or by analyzing the annotator's correction rate and the degree to which the fine-tuned model's predictions differ in style from GPT-4o's. The Limitations section acknowledges reliance on initial SRL quality, but the more specific circularity between training-label generation and test-label construction is not addressed.
  2. [Section A.2 (Annotator Recruitment and Payments)] The reliability of the core annotations is unmeasured. Stage 1 implicit-entity identification was performed by two annotators on disjoint halves of 700 samples with zero overlap, and no inter-annotator agreement is reported. Stage 3 relied on a single annotator to correct the GPT-4o-Mini SRLs. Given that the benchmark's defining contribution is implicit-argument annotation, the authors should report inter-annotator agreement (e.g., Cohen's kappa or a similar measure) on an overlapping subset for Stage 1, and ideally for Stage 3 as well. Without this, readers cannot assess whether the gold labels are stable and reproducible, which is a necessary condition for the benchmark to be reliable.
  3. [Section 7.1 (Table 2) and Section 6.2] The headline improvements over GPT-4o are not accompanied by any variance or significance analysis. Fine-tuning with LoRA can be sensitive to random seeds and data ordering, and the evaluation metric involves a non-standard word-overlap IoU for non-exact matches. The authors should report mean and standard deviation over at least three training runs, or a significance test (e.g., bootstrap or paired test), to establish that the 17% and 14.7% relative improvements are not due to noise. This is particularly important because the differences are relative to a single zero-shot GPT-4o run.
minor comments (6)
  1. [Prompt figures (Figure 9, 10, 11)] The text in the prompt figures contains the typo "wheather" instead of "whether" (e.g., "decide wheather you need to fill"). Since these prompts are part of the reproducible annotation pipeline, the typos should be corrected.
  2. [Table 1] The row "whatemptyset/total count" lacks a space; it should be "what empty set / total count".
  3. [Table 3] The GEPSAN row (Row 1) has empty values for #Params and Type. Either fill in the information or explain why it is omitted.
  4. [Figure 2] The caption says the masked arguments are highlighted with red boxes, but the red boxes are not visible in the figure. Consider increasing contrast or adding explicit annotations.
  5. [Section 6.2] The sliding-window evaluation for next-step prediction (matching against the next three gold steps) is described only briefly. Please specify how the window is handled near the end of a recipe and how ties are broken.
  6. [Section 2 (Related Work)] The citation "Assembly101 (Sener et al.)" lacks a year. Also, the paper claims "state-of-the-art performance" in Section 7.2, but the comparison set is limited; consider softening the claim or adding more baselines.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical fine-tuning comparison, and the shared GPT-4o annotation pipeline is a benchmark-validity caveat rather than a derivation-level circularity.

full rationale

The paper's headline result (17% relative F1 improvement for what-implicit and 14.7% for where/with-implicit over GPT-4o, Section 7.1, Table 2) is a measured experimental outcome, not a quantity derived from the training setup. The silver-standard training data are generated by GPT-4o from Tasty text instructions using the annotation prompt family (Section 5, Figure 11), while the test-set gold labels are produced by GPT-4o-Mini in Stage 2 and then corrected by a single PhD student in Stage 3 (Section 3). This shared annotation pipeline raises a legitimate benchmark-validity concern: if the human correction is minimal or style-anchored, the gold labels could partly reflect GPT-4o-Mini conventions, and the fine-tuned model could be rewarded for matching those conventions. However, this is not circularity in the sense used here: the fine-tuned model is a different architecture (Qwen2-VL), the gold labels are not a subset of the training labels, human correction intervenes, and the reported improvement is not forced by construction. The paper's own Limitations ('The proposed method relies heavily on the quality of the initial SRLs') explicitly acknowledges the dependence on automatically generated SRLs, which is a robustness and data-quality caveat rather than a logical equivalence between input and output. No load-bearing self-citation is present: the authors' prior works (e.g., Regneri et al. 2013, Lin et al. 2022) appear only as background references. No uniqueness theorem or ansatz is imported from the authors' own prior work. Therefore, while the benchmark deserves scrutiny for annotation reliability, the derivation chain is not circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper's central contribution is a dataset, not a theoretical derivation. It relies on domain assumptions about the suitability of SRL for procedural video and the quality of LLM-generated labels, rather than on fitted physical parameters.

assumptions (4)
  • domain assumption Procedural instructions can be decomposed into {verb, what, where/with} frames.
    Section 3 states this decomposition; the entire annotation and training rely on it.
  • ad hoc to paper Implicit arguments are limited to ingredients appearing in previous steps or video clips.
    Appendix A.1 defines this rule; it excludes broader world knowledge and tools, shaping what the benchmark measures.
  • ad hoc to paper GPT-4o-generated silver-standard SRLs are accurate enough to train on.
    Section 5.1 uses GPT-4o to generate training labels; the Limitations explicitly acknowledge this may introduce biases.
  • domain assumption The gold-standard annotations are reliable.
    Created by GPT-4o-Mini plus one human annotator; no inter-annotator agreement is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Predicting Implicit Arguments in Procedural Video Instructions." pith.science (2026). https://pith.science/paper/73QCGAUO

@misc{pith2026250521068,
  author       = {Pith},
  title        = {Pith review of: Predicting Implicit Arguments in Procedural Video Instructions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/73QCGAUO}},
  note         = {Machine review of arXiv:2505.21068}
}
read the original abstract

Procedural texts help AI enhance reasoning about context and action sequences. Transforming these into Semantic Role Labeling (SRL) improves understanding of individual steps by identifying predicate-argument structure like {verb,what,where/with}. Procedural instructions are highly elliptic, for instance, (i) add cucumber to the bowl and (ii) add sliced tomatoes, the second step's where argument is inferred from the context, referring to where the cucumber was placed. Prior SRL benchmarks often miss implicit arguments, leading to incomplete understanding. To address this, we introduce Implicit-VidSRL, a dataset that necessitates inferring implicit and explicit arguments from contextual information in multimodal cooking procedures. Our proposed dataset benchmarks multimodal models' contextual reasoning, requiring entity tracking through visual changes in recipes. We study recent multimodal LLMs and reveal that they struggle to predict implicit arguments of what and where/with from multi-modal procedural data given the verb. Lastly, we propose iSRL-Qwen2-VL, which achieves a 17% relative improvement in F1-score for what-implicit and a 14.7% for where/with-implicit semantic roles over GPT-4o.

Figures

Figures reproduced from arXiv: 2505.21068 by the authors.

Figure 1
Figure 1. Implicit-VidSRL: A new semantic role label￾ing (SRL) based dataset, to represent procedural videos using semantic frames ({verb,what,where/with}) with implict arguments. For instance step 2 is transformed into step 2(a) & step 2(b). While in step 5 the arguments are implicit and require both visual and textual context to infer from step 3 & 2. The implicit information is emphasized using a background color. cooking … view at source ↗
Figure 2
Figure 2. The Implicit Argument Prediction task involves providing the input sequence, which may be in the form of text or video or both, to a multimodal large model, alongside masked semantic frames: The arguments that are highlighted with red boxes in the output structure are not provided as part of the input and have to be predicted. to measure understanding and inference abilities. Rubin (1976) demonstrated that human acc… view at source ↗
Figure 3
Figure 3. Qualitative example using video-only predictions. The example is from TASTY (Sener et al., 2022) with ID-cider-pulled-pork. The examples highlight common errors in the predictions, i.e., a failure to track the mixture ingredients, as in step 3(a) pork is mixed with spices. Incorrect predictions are highlighted in red and missing ingredients are indicated using ‘??’. all, we achieve state-of-the-art performance acros… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison of GPT-4o and our iSRL-Qwen2- VL model for argument prediction across semantic frame positions in multi-modal procedural inputs (V+T). strong results at early semantic positions, but per￾formance decreases gradually as semantic position increases. In contras…
Figure 5
Figure 5. Figure 5: Annotation Tool. The images in (a) & (b) shows the tool interface to annotate the implicit entities during Stage 1. While the image in (c) shows the tool interface for semantic role labeling in Stage 3 (the video is omitted for clarity). the annotators involved in the …
Figure 6
Figure 6. Figure 6: Comparison of GPT-4o and ours iSRL-Qwen2-VL models for argument prediction across semantic frame positions [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Qualitative Examples from Implicit-VidSRL dataset [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Dataset Prompt [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: Prompt for Implicit Argument Prediction as Cloze task. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: Prompt for Next Step Prediction with Semantic Frame. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Prompt to generate Silver-standard dataset using GPT-4o ( [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 41 canonical work pages

  1. [1]

    Annotation Guidelines (Section A)

    work page

  2. [2]

    Additional Dataset sources and Implementa- tion details (Section B)

    work page

  3. [3]

    Prompt Details for Silver-standard dataset and Task inference (Section C)

    work page

  4. [4]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13853–13863

    Learning to recognize procedural activities with distant supervision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13853–13863. Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. 2024. TempCompass: Do video LLMs really understand videos? InFindings of the As...

    work page 2024

  5. [5]

    add flour and spices in a bowl and mix

    If the verb is transitive i.e. need one object. For example, (i) “add flour and spices in a bowl and mix”. Here “mix” verb requires an implicit object i.e. mix flour and spices. Stage 2 - GPT-4o-Mini Labels:The prompt to pre-annotate and split the multi-step instructions is shown in Figure 8. Stage 3 - Manual Refinement:Along with the Stage 1 guidelines, ...

    work page

  6. [13]

    The Explicit Argu- ment is an argument directly mentioned or visible in the current video clip

    Additional Analysis and Qualitative Results (Section D); A TheImplicit-VidSRLDataset - Annotations A.1 Annotation Guidelines The annotation tool is shown in Figure 5 Stage 1 - Implicit Entities:We define the Im- plicit Argument as an ingredient implied by earlier steps or video clips, not directly visible in the cur- rent clip or text, but it must appear ...

    work page

  7. [14]

    Focus only on cooking ingredients and ignore the tools or cooking utensils

    work page

  8. [15]

    Only use the previously mentioned entities for the implicit arguments

    work page

Show all 45 references
  1. [16]

    them” and “it

    Only add implicit entities for nouns and pro- nouns like “them” and “it”, and disregard all other cases

  2. [17]

    add spices and onions to the bowl

    If the verb is di-transitive i.e. need two objects such as {verb,what,where/with}. For exam- ple: in the sentence “add spices and onions to the bowl”, the verb “add” has two argu- ments “spices and onions” (i.e. items need to be added) and “the bowl” (i.e. where the items will...

  3. [19]

    add onions and tomatoes to the blender and blend them

    Use previously mentioned entities for im- plicit arguments. For instance, (i) “add onions and tomatoes to the blender and blend them.”, (ii) “add spices and garlic to the blender” the second instruction be- comes { verb:add, what:[spices, garlic], where/with:[onions, tomatoes]...

  4. [20]

    Ignore instructions withoutwhatand where/witharguments, such as “repeat”

  5. [21]

    turn heat to low

    Avoid instructions without visible ingredients like "turn heat to low" unless they are visi- ble in the video. However, instructions like "bake tomatoes" are considered even if they are implicit

  6. [22]

    add sausages on top of pota- toes

    Determinewhatbased on the verb. For example, “add sausages on top of pota- toes” becomes { verb:add, what:sausages, where/with:potatoes}, and “top potatoes with sausages” turns into { verb:top, what:potatoes,where/with:sausages}. A.2 Annotator Recruitment and Payments We hired...

    2018

  7. [23]

    srl_id":

    [{"srl_id":"STEP_ID-ACTION_ID", "verb": "verb_action", "what": "main_object", "where_or_with": "WITH or WHERE arguments"}] Implicit Argument: An ingredient implied from previous steps that isn't directly stated in the current instruction. Implicit arguments can be in either th...

  8. [24]

    1. add the flour and water in the bowl and mix

    Split, Track and Predict Implicit Ingredients: 1.1 Split: Split each numbered recipe step into individual actions based on verbs. Each action should be a single-action sentence. For example - "1. add the flour and water in the bowl and mix." should be split into [{"srl_id":"1-...

  9. [27]

    ??". Based on the concatenated video clips and its corresponding steps, your goal is to fill the arguments of each verbs mentioned by

    The response should be in the same format as the examples provided and semantic role labels should be numbered with same step number as the recipe steps. ----------------------------------------------------------------- **QUERY - START**: **Task**: <<task>> **Recipe Steps**: <...

  10. [28]

    You do not focus on the tools used to prepare the ingredients

    You majorly focus on the ingredients used in the recipe. You do not focus on the tools used to prepare the ingredients

  11. [29]

    Make sure to observe the blank frame that separates the video clips depicting the recipe steps

    Carefully watch each video clip and understand the recipe steps. Make sure to observe the blank frame that separates the video clips depicting the recipe steps

  12. [30]

    Carefully understand and track the ingredients along with composite entities representing the composition of ingredients

  13. [31]

    verb": "add

    Based on the verb, decide wheather you need to fill the WHAT or WHERE_OR_WITH or both arguments. 4.1. *BOTH ARGUMENTS*: The verb such as add, mix, chop, etc. is used, then you need to fill the both arguments. However, if argument contains a tool, then you need to ignore the to...

  14. [32]

    ??" in the

    Only fill in the missing arguments marked with "??" in the "Response Semantic Role Labels". ……….. ----------------------------------------------------------------- **THE FOLLOWING EXAMPLES ASSUME THE VIDEO CLIPS ARE SUMMARIZED INTO TEXTUAL RECIPE STEPS AND GOAL IS TO FILL THE ...

  15. [33]

    srl":[ {

    put chickpeas parsley chopped onion chili powder ground cumin in food processor. ……….. **Masked Semantic Role Labels**: { "srl":[ {"srl_id":"1-1", "verb": "put", "what": "??", "where_or_with":"??"}, ……….. ] } **Response Semantic Role Labels**: { "srl":[ {"srl_id":"1-1", "verb"...

  16. [34]

    It majorly focus on the ingredients used in the recipe and avoid the focus on the tools used to prepare the ingredients

  17. [35]

    verb": "add

    Based on the verb, the WHAT or WHERE_OR_WITH or both arguments are updated. Consider the following cases: 2.1. *BOTH ARGUMENTS*: The verb such as add, mix, chop, etc. is used, then the both arguments are mandatory. However, if argument contains a tool, then we ignore the tool ...

  18. [36]

    Also, observe their corresponding semantic role labels

    Read and Watch the given steps in text and video clips. Also, observe their corresponding semantic role labels. Note that for multi-step actions, the semantic role labels are splited into individual actions

  19. [37]

    Similar to the examples, first predict the single or multiple semantic role labels for the next step based on the given steps

  20. [38]

    Make sure to include the verb, what, where_or_with, and tools in the semantic role labels

  21. [39]

    **Don'ts**

    Later, predict the next instruction based on the semantic role label. **Don'ts**

  22. [40]

    Do not generate any additional examples

    Do not include any additional information in the response. Do not generate any additional examples

  23. [41]

    Semantic Role Labels

    Do not change or remove the **SRL_ID** in the "Semantic Role Labels"

  24. [43]

    ----------------------------------------------------------------- Please note that the examples do not include video clips, as the examples are for illustration purposes only

    Do not change the format of the output. ----------------------------------------------------------------- Please note that the examples do not include video clips, as the examples are for illustration purposes only. **Example 1 - START**: <<IN CONTEXT EXAMPLE - OMITTED DUE TO ...

  25. [44]

    Do not focus on the tools used to prepare the ingredients

    You majorly focus on the ingredients used in the recipe. Do not focus on the tools used to prepare the ingredients

  26. [45]

    Squeeze some lime juice into the food processor and add some olive oil

    For each recipe step, validate if the instruction is complex or simple. If it is complex or contains multiple actions, then split it into distinct single-action steps in your mind. 2.1 Break down the instruction into distinct single-action steps in your mind. Consider the foll...

  27. [46]

    verb": "add

    Once you have split the instructions into distinct single-action steps, create semantic role labels for each step. Based on the verb of each step, decide wheather you need to fill the WHAT or WHERE_OR_WITH or both arguments. 3.1. *BOTH ARGUMENTS*: The verb such as add, mix, ch...

  28. [47]

    Make sure **SRL_ID** is unique for each action sentence

  29. [48]

    Finally, predict the semantic role labels for the given QUERY recipe steps

    Carefully observe the examples given below and format the response accordingly. Finally, predict the semantic role labels for the given QUERY recipe steps. ----------------------------------------------------------------- <<ICL_EXAMPLES>> NOW IT'S YOUR TURN! PREDICT THE SEMANT...

    2024

  30. [350]

    Springer. Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. 2021. The epic- kitchens dataset: Collection, challenges and base- lines.IEEE Transa...

    2021 arXiv

  31. [1998]

    The Berkeley FrameNet project. In36th An- nual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1, pages 86–90, Montreal, Quebec, Canada. Association for Compu- tational Linguistics. Satanjeev Baner...

    2005

  32. [2005]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu

    The proposition bank: An annotated corpus of semantic roles.Computational linguistics, 31(1):71– 106. Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. InProceedings of the 40th annual meeting ...

    2002

  33. [2013]

    Transactions of the Association for Computational Linguistics (TACL), 1:25–36

    Grounding action descriptions in videos. Transactions of the Association for Computational Linguistics (TACL), 1:25–36. David C Rubin. 1976. The effectiveness of context before, after, and around a missing word.Perception & Psychophysics, 19:214–216. Arka Sadhu, Tanmay Gupta, ...

    1976

  34. [2019]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1207–1216

    Coin: A large-scale dataset for comprehen- sive instructional video analysis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1207–1216. Andong Wang, Bo Wu, Sunli Chen, Zhenfang Chen, Haotian Guan, Wei-Ning Lee, Li Erran Li, and Chuan...

    2022 arXiv

  35. [2020]

    InProceed- ings of the Twelfth Language Resources and Evalua- tion Conference, pages 5187–5194

    English recipe flow graph corpus. InProceed- ings of the Twelfth Language Resources and Evalua- tion Conference, pages 5187–5194. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 tec...

    2024 arXiv

  36. [2022]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li

    Grounded video situation recognition.Ad- vances in Neural Information Processing Systems, 35:8199–8210. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024a. Llava- onevision: Easy visual task transfer.ar...

    arXiv

  37. [2024]

    InProceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 3: System Demonstra- tions), Bangkok, Thailand

    Llamafactory: Unified efficient fine-tuning of 100+ language models. InProceedings of the 62nd Annual Meeting of the Association for Compu- tational Linguistics (Volume 3: System Demonstra- tions), Bangkok, Thailand. Association for Computa- tional Linguistics. Luowei Zhou, Ch...

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.