Pith. sign in

REVIEW 5 major objections 5 minor 61 references

$I^2G$: Generating Instructional Illustrations via Text-Conditioned Diffusion

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pairwise diffusion design makes generated instruction images match the text better than standard baselines.

desk verdict The paper proposes a sensible approach to instructional illustration generation, but the evaluation metric is circular and the pairwise model description is internally inconsistent, so the main claim is unsupported. read the letter →

arxiv 2505.16425 v1 pith:L6RJTUAX submitted 2025-05-22 cs.CL cs.AI

Jing Bi , Pinxin Liu , Ali Vosoughi , Jiarui Wu , Jinxi He , Chenliang Xu This is my paper · ORCID
classification cs.CLcs.AI
keywords proceduraltext-to-imagegenerationinstructionalillustrationsdiffusionmodelspairwisecoherencetext-imagealignmentevaluationprotocolgoal-stepdecompositionpreferenceoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that procedural text, instructions made of a goal and ordered steps, can be translated into a matching sequence of images by treating the task as a language-conditioned diffusion problem. The proposed I2G framework splits each instruction into goal and step texts, generates images with Stable Diffusion XL, and couples neighboring steps through a masked attention layer that lets each image borrow visual context from the adjacent step. To keep long step texts intact, a constituency parser breaks each step into clauses that are encoded separately. The authors argue that this combination produces images that align with the linguistic content and the sequential order of instructions better than four strong baselines, and they support the claim with a new evaluation protocol based on captioning generated images and comparing the captions to the original instructions.

What carries the argument

The load-bearing mechanism is the pairwise factorization of the image-generation distribution, implemented as masked cross-step attention between latent pairs: two step images are encoded to latents $z_i,z_j$, stacked, and passed through a masked self-attention layer where a mask $\hat{M}$ restricts attention mostly to the same slice and the adjacent slice, so step $j$ inherits object identity and setting from step $i$. Two supporting mechanisms carry the text side: a constituency parser that splits long step descriptions into clauses before encoding, with CLIP-ViT/L for goals and OpenCLIP-ViT/G for steps, and a preference-optimization loop that treats a caption-based text-image alignment score as a reward and backpropagates it through the diffusion sampling steps.

What would settle it

A human preference study on held-out instructions from the same three datasets: show raters paired images from I2G and from each baseline for the same goal and step text, blind to method, and ask which image better matches the step. If human preference does not track the lower KL and Chi-square values, or if swapping BLIP2 for a different captioner reverses the ranking of methods, the central claim is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a pairwise factorization of the joint distribution over instruction images, $p(v_1,\dots,v_n \mid g,S) \approx \prod_{i<j} p(v_i,v_j \mid g,s_i,s_j)$, lets a diffusion model capture cross-step coherence without the cost of a fully joint model with a fixed step count. This factorization is realized by concatenating the latent representations of two images and applying a masked self-attention that restricts attention to one's own latent slice and its adjacent neighbor. On HTStep, CaptainCook4D, and WikiAll, the resulting model reports lower KL divergence and Chi-square statistics than SD1.5, SD2.1, SDXL, and StackedDiffusion, under an evaluation that first generates captions with BLIP2 or LLaVA and then measures text similarity with LLM2Vec against the original goal and step texts. The authors also show that CLIPScore and multimodal binding models track human judgments poorly on instructional content, motivating the new protocol.

Load-bearing premise

The entire comparison stands on the assumption that the automated score, caption the generated image and measure text similarity between that caption and the original instruction, faithfully matches what human raters would call text-image alignment, and that KL and Chi-square differences on that score translate into visibly better instruction illustrations.

Editorial extensions

If this is right

  • On all three tested datasets, the method's KL divergence and Chi-square values are lower than SD1.5, SD2.1, SDXL, and StackedDiffusion, meaning fewer generated images whose captions mismatch the instruction text.
  • The pairwise factorization gives multi-step coherence without a fixed number of steps, unlike the StackedDiffusion baseline, which cannot generate more than six steps.
  • The caption-then-compare protocol, BLIP2 or LLaVA captions plus LLM2Vec text similarity, can serve both as an evaluator and as a reward signal for finetuning diffusion models on procedural text.
  • Because goals are contextual rather than visual, goal faithfulness is harder to improve than step faithfulness, and the paper's human evaluations show evaluators struggle to connect goal text to images.
  • The framework is not limited to cooking: the three datasets span many how-to tasks, so the same goal-and-step decomposition applies to education and assembly instructions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same pairwise masked-attention factorization should transfer to video generation, where adjacent-frame coherence plays the role of adjacent-step coherence, making it a plausible inductive bias for text-conditioned video diffusion.
  • Editorial inference: a testable extension the paper does not run is swapping BLIP2 or LLaVA for a stronger vision-language captioner; if the reported ranking of methods reverses, part of the gain is an artifact of the specific captioner rather than of the generated images.
  • Editorial inference: because the paper observes that multimodal LLMs reward matching object names even when the described actions differ, a reward that also scores verb and action agreement would likely push generated images further toward correct procedural states.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes I2G, a text-conditioned diffusion framework for generating step-by-step instructional illustrations from procedural text. The method decomposes input into goal and step texts, encodes long steps with a constituency parser, introduces a pairwise factorization with cross-image attention for coherence, and fine-tunes an SDXL-based model using an image-text alignment reward. Experiments on HTStep, CaptainCook4D, and WikiAll report lower KL divergence and chi-square values than SD1.5, SD2.1, SDXL, and StackedDiffusion, alongside FID scores, leading to the claim that the method significantly outperforms existing baselines in procedural text-image alignment.

Significance. If the central claim were established, the paper would make a useful contribution to text-to-image generation for instructional content, a relatively underexplored task. The paper addresses a genuine problem, and the idea of using an LLM-based alignment score both as a training signal and as an evaluation tool is not unreasonable in itself. The authors also report FID as an independent image-quality check and candidly discuss failure cases in Appendix A.3. However, as written, the evidence is not convincing: the evaluation metric coincides with the training reward, the pairwise coherence model is specified inconsistently, and the quantitative results lack statistical support. The significance of the claimed improvements therefore cannot be assessed from the manuscript.

major comments (5)
  1. [§5.2 and §4.4, Table 1] The evaluation metric used in Table 1 is the same signal used as the training reward. Section 5.2 explicitly states that the BLIP2/LLaVA captioning plus LLM2Vec text-similarity framework 'serves not only as an evaluator but also as a reward model to assist in fine-tuning, discussed further in Sec 4.4,' and the objective J(θ) in Sec 4.4 uses exactly this alignment score r(·). Reporting improvements on a metric that the model was optimized against does not constitute independent evidence of better text-image alignment. This concern is amplified by the authors' own observation at the end of Sec 5.2 that the MLLM assigns high scores when object names match even when the described actions differ, which suggests the metric can be gamed by superficial object co-occurrence. The central claim of the paper therefore rests on circular evidence as it stands.
  2. [§4.1–4.2] The pairwise factorization and the adjacency mask are internally inconsistent. Section 4.1 defines p(v1,...,vn | g,S) ≈ ∏_{i<j} p(vi,vj | g,si,sj), a product over all pairs, while Section 4.2 describes an attention mask that 'restricts attention mostly to one's own latent slice and the adjacent slice.' The paper never states which formulation is actually implemented. If all pairs are used, the adjacency mask is not a faithful implementation of the model; if only adjacent pairs are used, the claimed long-range coherence across non-adjacent steps is not captured. Additionally, a product over all i<j pairs has O(n^2) terms, which is hard to reconcile with the claim that the factorization avoids a combinatorial explosion. This ambiguity bears directly on the main methodological contribution.
  3. [§5.3, Table 1] The quantitative comparison in Table 1 lacks the statistical grounding needed to support 'significantly outperforms.' No error bars, confidence intervals, significance tests, or sample sizes are reported, and the differences between some entries are small (e.g., KL 12.71 vs. 12.48 for Ours vs. SDXL on HTStep with Humanllava). It is also not defined what distributions the KL divergence and chi-square statistic are computed over, which distributions are being compared, or how the 'GT Captioner' column and the 'Humanllava'/'Humanblip2' rows interact with the protocol. As written, the improvements in Table 1 could be noise or an artifact of the metric definition.
  4. [§5.2] The paper claims that the MLLM+LLM2Vec evaluation 'aligns well with human judgment across three datasets,' but the human evaluation that this claim depends on is not reported in any detail. There is no description of the number of annotators, the number of samples judged, the scoring instructions beyond the score scale, or inter-annotator agreement. Without these details, the validity of the automated metric—and hence of the main evaluation—cannot be checked. This is load-bearing because the automated metric is the basis of Table 1.
  5. [§4.4–4.5] The training procedure of the proposed model is not reproducible from the manuscript. The paper does not specify the SDXL variant used, the fine-tuning dataset size, the number of gradient and sampling steps, the learning rate, the CFG scale, the conditioning dropout probability, or how the reward model's gradients are backpropagated through the sampling process. Given that the proposed method is a fine-tuning procedure, these details are necessary for verification and for comparison with existing reward-finetuning methods such as DDPO, DPOK, and AlignProp.
minor comments (5)
  1. [Equation (1)] There is a typo in the notation: the sentence says 'g, S, and S represent the goal text, the sequence of step instructions, and the visual information,' but the third symbol should be V, not S.
  2. [§4.4] The objective J(θ) is written with parameter θ, but the text says only the diffusion parameters ϕ are updated; this notational inconsistency should be fixed.
  3. [Table 1] The table header 'GT Captioner' is unclear, and the meaning of the 'Humanllava' and 'Humanblip2' entries is not explained in the table or its caption; the caption should state that the values are multiplied by 100 and clarify which captioner is used for which row.
  4. [§5.3] The paragraph discussing 'Goal Faithfulness' and 'Step Faithfulness' uses CLIP-based similarity, but the actual evaluation in Table 1 uses KL and chi-square statistics; the relationship between these two evaluation schemes should be made explicit.
  5. [Appendix A.3, Figure 5] The failure case in A.3, where the word 'steak' triggers an image of an angel despite a stove being in the prompt, undermines the general claim of accurate step-wise generation; this limitation is acknowledged but should be discussed in the main text as a caveat on the headline results.

Circularity Check

1 steps flagged · score 7.0 of 10

Evaluation reward and reported metric are the same BLIP2/LLaVA+LLM2Vec alignment pipeline, so Table 1 gains are not independent evidence.

  1. fitted input called prediction [Sec. 4.4 (Preference Optimization) and Sec. 5.2 (Text-Image Alignment)]
    "J(θ) = Ec,xT [ r( sample(θ, g, s, xT ), g, s ) ], where r(·) quantifies the alignment of the image-text. ... By integrating MLLMs and the text encoder, we established a robust evaluation framework that serves not only as an evaluator but also as a reward model to assist in fine-tuning, discussed further in Sec 4.4."

    The same BLIP2/LLaVA captioning plus LLM2Vec text-similarity pipeline is used (a) as the reward r in Sec. 4.4/4.5 to backpropagate through the diffusion sampling and (b) as the evaluation framework in Sec. 5.2 whose distributional KL/Chi-squared scores are reported in Table 1. The paper itself states the framework 'serves not only as an evaluator but also as a reward model.' Because the model is explicitly optimized to maximize r, the post-finetuning KL/Chi2 gains on that same r are a direct consequence of the training objective, not an independent measurement of alignment. The only independent quantity in Table 1 (FID) does not measure text-image alignment, and no human evaluation of the final models is reported.

full rationale

The paper's architectural contributions (constituency-parser encoding, pairwise coherence mask, diffusion backbone) are not definitionally circular. There is no load-bearing self-citation: the cited LLM2Vec is external work. The central circularity is that the evaluation protocol is not independent of training: the exact alignment score used as the fine-tuning reward is the same score underlying Table 1's KL/Chi2 comparisons. Since the model is optimized against that scorer, reporting lower KL/Chi2 on it is an expected optimization outcome rather than a neutral test of alignment. FID is independent but does not measure text-image alignment, and no human evaluation of final models is reported. This is a central evaluation loop rather than a fully self-referential derivation, hence a score of 7.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim depends on a pairwise independence assumption, an unvalidated automated evaluation metric, and the use of that metric as a training reward, which creates a circular loop.

free parameters (1)
  • CLIPScore scaling weight w = 2.5
    Hand-chosen in Sec 5.2 to enlarge score differences; used in CLIP-based metrics.
assumptions (4)
  • domain assumption The joint distribution over instruction images can be factorized as a product of pairwise conditionals p(vi,vj | g,si,sj).
    Introduced in Sec 4.1; this is a modeling assumption about how steps depend on each other.
  • domain assumption A captioner (BLIP2/LLaVA) plus a long-text encoder (LLM2Vec) similarity score approximates human judgments of text-image alignment for procedural images.
    Sec 5.2 asserts alignment without reporting correlation coefficients or human evaluation details.
  • domain assumption The same text-image alignment score is a valid reward signal for preference optimization of the diffusion model.
    Sec 4.4 uses r(·) as reward; this assumes the reward model is a good proxy for human preference.
  • domain assumption The pretrained LLM can sample goal/step pairs that match the training distribution.
    Sec 4.4 samples (g,s) from the LLM; no validation that the sampled text matches dataset statistics.

how reviews work

0 comments
Cite this review

Pith. "Pith review of $I^2G$: Generating Instructional Illustrations via Text-Conditioned Diffusion." pith.science (2026). https://pith.science/paper/L6RJTUAX

@misc{pith2026250516425,
  author       = {Pith},
  title        = {Pith review of: $I^2G$: Generating Instructional Illustrations via Text-Conditioned Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L6RJTUAX}},
  note         = {Machine review of arXiv:2505.16425}
}
read the original abstract

The effective communication of procedural knowledge remains a significant challenge in natural language processing (NLP), as purely textual instructions often fail to convey complex physical actions and spatial relationships. We address this limitation by proposing a language-driven framework that translates procedural text into coherent visual instructions. Our approach models the linguistic structure of instructional content by decomposing it into goal statements and sequential steps, then conditioning visual generation on these linguistic elements. We introduce three key innovations: (1) a constituency parser-based text encoding mechanism that preserves semantic completeness even with lengthy instructions, (2) a pairwise discourse coherence model that maintains consistency across instruction sequences, and (3) a novel evaluation protocol specifically designed for procedural language-to-image alignment. Our experiments across three instructional datasets (HTStep, CaptainCook4D, and WikiAll) demonstrate that our method significantly outperforms existing baselines in generating visuals that accurately reflect the linguistic content and sequential nature of instructions. This work contributes to the growing body of research on grounding procedural language in visual content, with applications spanning education, task guidance, and multimodal language understanding.

Figures

Figures reproduced from arXiv: 2505.16425 by the authors.

Figure 1
Figure 1. The qualitative results of our method, com [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. We randomly sample pairs (vi , si) and (vj , sj ), apply a custom adjacency mask to fuse latent representa￾tions, and decode them back into images. The constituency parser (Sec. 4.3) splits text to handle length constraints. 5 Experiment 5.1 Datasets CaptainCook4D: The CaptainCook4D dataset comprises 384 cooking videos, covering 24 cook￾ing tasks. According to the creator of the dataset, the primary objective of cre… view at source ↗
Figure 3
Figure 3. We demonstrate that the goal text often serves as contextual information with limited relation to the visual [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Compared to goal image alignment, MLLMs perform better in aligning step text with images, showing [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Images of successful and failed generation [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 12 canonical work pages

  1. [1]

    Triantafyllos Afouras, Effrosyni Mavroudi, Tushar Nagarajan, Huiyu Wang, and Lorenzo Torresani. 2023. https://openreview.net/forum?id=vv3cocNsEK HT -step: Aligning instructional articles with how-to videos . In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    work page 2023

  2. [2]

    Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. 2022. https://arxiv.org/abs/2104.00650 Frozen in time: A joint video and image encoder for end-to-end retrieval . Preprint, arXiv:2104.00650

    arXiv 2022

  3. [3]

    Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. https://arxiv.org/abs/2404.05961 Llm2vec: Large language models are secretly powerful text encoders . Preprint, arXiv:2404.05961

    arXiv 2024

  4. [4]

    https://api.semanticscholar.org/CorpusID:264403242 Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, † TimBrooks, Jianfeng Wang, Linjie Li, † LongOuyang, † JuntangZhuang, † JoyceLee, † YufeiGuo, † WesamManassra, † PrafullaDhariwal, † CaseyChu, † YunxinJiao, and Aditya Ramesh. https://api.semanticscholar.org/CorpusID:264403242 Improving image generation with better captions

    work page

  5. [5]

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. 2024. https://arxiv.org/abs/2305.13301 Training diffusion models with reinforcement learning . Preprint, arXiv:2305.13301

    arXiv 2024

  6. [6]

    Ziyi Chang, George Alex Koulieris, and Hubert P. H. Shum. 2023. https://arxiv.org/abs/2306.04542 On the design fundamentals of diffusion models: A survey . Preprint, arXiv:2306.04542

    arXiv 2023

  7. [7]

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Pan Zhou, Yao Wan, and Lichao Sun. 2024 a . https://arxiv.org/abs/2402.04788 Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark . Preprint, arXiv:2402.04788

    arXiv 2024

  8. [8]

    Muxi Chen, Yi Liu, Jian Yi, Changran Xu, Qiuxia Lai, Hongliang Wang, Tsung-Yi Ho, and Qiang Xu. 2024 b . Evaluating text-to-image generative models: An empirical study on human image synthesis. arXiv preprint arXiv:2403.05125

    work page Pith review arXiv 2024

Show all 61 references
  1. [9]

    Yixiong Chen, Li Liu, and Chris Ding. 2023. https://arxiv.org/abs/2305.10843 X-iqe: explainable image quality evaluation for text-to-image generation with visual large language models . Preprint, arXiv:2305.10843

    2023 arXiv

  2. [10]

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. 2023. Reproducible scaling laws for contrastive language-image learning. In Proceedings of the IEEE/CVF Conference on Computer ...

    2023

  3. [11]

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. 2023. https://arxiv.org/abs/2309.17400 Directly fine-tuning diffusion models on differentiable rewards . Preprint, arXiv:2309.17400

    2023 arXiv

  4. [12]

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xiaofang Wang, Abhimanyu Dubey, Matthew Yu, Abhishek Kadian, Filip Radenovic, Dhruv Mahajan, Kunpeng Li, Yue Zhao, Vladan Petrovic, Mitesh Kumar Singh, Simran Motwani, and ...

    2023 arXiv

  5. [13]

    Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. 2023. https://arxiv.org/abs/2304.06767 Raft: Reward ranked finetuning for generative foundation model alignment . Preprint, arXiv:2304.06767

    2023 arXiv

  6. [14]

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. 2023. https://arxiv.org/abs/2305.16381 Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models . Preprint, arXiv:...

    2023 arXiv

  7. [15]

    Daniel Fried, Jean-Baptiste Alayrac, Phil Blunsom, Chris Dyer, Stephen Clark, and Aida Nematzadeh. 2020. https://arxiv.org/abs/2005.03684 Learning to segment actions from observation and narration . Preprint, arXiv:2005.03684

    2020 arXiv

  8. [16]

    Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023. Gptscore: Evaluate as you desire. arXiv preprint arXiv:2302.04166

    2023 arXiv

  9. [17]

    Bermano, Gal Chechik, and Daniel Cohen-Or

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Or. 2022. https://arxiv.org/abs/2208.01618 An image is worth one word: Personalizing text-to-image generation using textual inversion . Preprint, arXiv:2208.01618

    2022 arXiv

  10. [18]

    Simon Ging, Mohammadreza Zolfaghari, Hamed Pirsiavash, and Thomas Brox. 2020. https://arxiv.org/abs/2011.00597 Coot: Cooperative hierarchical transformer for video-text representation learning . Preprint, arXiv:2011.00597

    2020 arXiv

  11. [19]

    Tengda Han, Weidi Xie, and Andrew Zisserman. 2022. https://arxiv.org/abs/2204.02968 Temporal alignment networks for long-term video . Preprint, arXiv:2204.02968

    2022 arXiv

  12. [20]

    Yaru Hao, Zewen Chi, Li Dong, and Furu Wei. 2023. https://arxiv.org/abs/2212.09611 Optimizing prompts for text-to-image generation . Preprint, arXiv:2212.09611

    2023 arXiv

  13. [21]

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. 2021. Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718

    2021 arXiv

  14. [22]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. https://arxiv.org/abs/2006.11239 Denoising diffusion probabilistic models . Preprint, arXiv:2006.11239

    2020 arXiv

  15. [23]

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. 2023. https://arxiv.org/abs/2305.01569 Pick-a-pic: An open dataset of user preferences for text-to-image generation . Preprint, arXiv:2305.01569

    2023 arXiv

  16. [24]

    Hilde Kuehne, Ali Arslan, and Thomas Serre. 2014. https://doi.org/10.1109/CVPR.2014.105 The language of actions: Recovering the syntax and semantics of goal-directed human activities . In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 780--787

    2014 doi

  17. [25]

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. 2023. https://arxiv.org/abs/2302.12192 Aligning text-to-image models using human feedback . Preprint, arXiv:2302.12192

    2023 arXiv

  18. [26]

    Kyungmin Lee, Hyeongjin Kang, and Sungwoong Kim. 2025. Calibrated multi-preference optimization for aligning diffusion models. arXiv preprint arXiv:2502.02588

    2025

  19. [27]

    Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, and Kazuki Kozuka. 2024. https://api.semanticscholar.org/CorpusID:269004992 Aligning diffusion models by optimizing human utility . ArXiv, abs/2404.04465

    2024 arXiv

  20. [28]

    Xudong Lin, Fabio Petroni, Gedas Bertasius, Marcus Rohrbach, Shih-Fu Chang, and Lorenzo Torresani. 2022. https://arxiv.org/abs/2201.10990 Learning to recognize procedural activities with distant supervision . Preprint, arXiv:2201.10990

    2022 arXiv

  21. [29]

    Runtao Liu, Ning Duan, Zhenyang Yang, and 1 others. 2025. Videodpo: Omni-preference alignment for video diffusion generation. arXiv preprint arXiv:2412.14167. To appear at CVPR 2025

    2025 arXiv

  22. [30]

    Ye Liu, Siyuan Li, Yang Wu, Chang Wen Chen, Ying Shan, and Xiaohu Qie. 2022. https://arxiv.org/abs/2203.12745 Umt: Unified multi-modal transformers for joint video moment retrieval and highlight detection . Preprint, arXiv:2203.12745

    2022 arXiv

  23. [31]

    Calvin Luo, Jacob Lambert, Eric Fu, and 1 others. 2025. Solving new tasks by adapting internet video knowledge. arXiv preprint arXiv:2504.15369

    2025 arXiv

  24. [32]

    Sachit Menon, Ishan Misra, and Rohit Girdhar. 2023. https://arxiv.org/abs/2312.04552 Generating illustrated instructions . Preprint, arXiv:2312.04552

    2023 arXiv

  25. [33]

    Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, and Andrew Zisserman. 2020. https://arxiv.org/abs/1912.06430 End-to-end learning of visual representations from uncurated instructional videos . Preprint, arXiv:1912.06430

    2020 arXiv

  26. [34]

    Rohith Peddi, Shivvrat Arya, Bharath Challa, Likhitha Pallapothula, Akshay Vyas, Jikai Wang, Qifan Zhang, Vasundhara Komaragiri, Eric Ragan, Nicholas Ruozzi, Yu Xiang, and Vibhav Gogate. 2023. https://arxiv.org/abs/2312.14556 Captaincook4d: A dataset for understanding errors i...

    2023 arXiv

  27. [35]

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. https://arxiv.org/abs/2307.01952 Sdxl: Improving latent diffusion models for high-resolution image synthesis . Preprint, arXiv:2307.01952

    2023 arXiv

  28. [36]

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. 2023. https://arxiv.org/abs/2310.03739 Aligning text-to-image diffusion models with reward backpropagation . Preprint, arXiv:2310.03739

    2023 arXiv

  29. [37]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, and 1 others. 2021. Learning transferable visual models from natural language supervision. In International conference on machine...

    2021

  30. [38]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684--10695

    2022

  31. [39]

    Karan Samel, Nitish Sontakke, and Irfan Essa. 2025. Leveraging procedural knowledge and task hierarchies for efficient instructional video pre-training. arXiv preprint arXiv:2502.17352

    2025 arXiv

  32. [40]

    Eyal Segalis, Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan. 2023. https://arxiv.org/abs/2310.16656 A picture is worth a thousand words: Principled recaptioning improves image generation . Preprint, arXiv:2310.16656

    2023 arXiv

  33. [41]

    Sebastian Stein and Stephen Mckenna. 2013. https://doi.org/10.1145/2506023.2506031 User-adaptive models for recognizing food preparation activities . pages 39--44

    2013

  34. [42]

    Yansong Tang, Dajun Ding, Yongming Rao, Yu Zheng, Danyang Zhang, Lili Zhao, Jiwen Lu, and Jie Zhou. 2019. https://doi.org/10.1109/CVPR.2019.00130 Coin: A large-scale dataset for comprehensive instructional video analysis . In 2019 IEEE/CVF Conference on Computer Vision and Pat...

    2019

  35. [43]

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. 2023. https://arxiv.org/abs/2311.12908 Diffusion model alignment using direct preference optimization . Preprint, arXiv:2311.12908

    2023 arXiv

  36. [44]

    Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023 a . Is chatgpt a good nlg evaluator? a preliminary study. arXiv preprint arXiv:2303.04048

    2023 arXiv

  37. [45]

    Jifang Wang, Rundi Cui, Lei Hou, and 1 others. 2025. Cigeval: A unified agentic framework for evaluating conditional image generation. arXiv preprint arXiv:2504.07046

    2025 arXiv

  38. [46]

    Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023 b . Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926

    2023 arXiv

  39. [47]

    Haiwan Wei, Chen Zhao, Abhishek Garg, and Jianbo Shi. 2025. Instructionbench: An instructional video understanding benchmark. arXiv preprint arXiv:2504.05040

    2025 arXiv

  40. [48]

    Sihao Wu, Kun Yin, Yanhong Liu, and Di He. 2025. Preference alignment on diffusion models: A comprehensive survey for image generation and editing. arXiv preprint arXiv:2502.07829

    2025 arXiv

  41. [49]

    Te-Lin Wu, Alex Spangher, Pegah Alipoormolabashi, Marjorie Freedman, Ralph Weischedel, and Nanyun Peng. 2024. https://arxiv.org/abs/2110.08486 Understanding multimodal procedural knowledge by sequencing multimodal instructional manuals . Preprint, arXiv:2110.08486

    2024 arXiv

  42. [50]

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. 2023. https://arxiv.org/abs/2306.09341 Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis . Preprint, arXiv:2306.09341

    2023 arXiv

  43. [51]

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. 2023. https://arxiv.org/abs/2304.05977 Imagereward: Learning and evaluating human preferences for text-to-image generation . Preprint, arXiv:2304.05977

    2023 arXiv

  44. [52]

    Zhiyuan Yan, Yutong Liu, Yaqi Yuan, and 1 others. 2025. Gpt-imgeval: A benchmark for diagnosing gpt-4v's image generation. arXiv preprint arXiv:2504.02782

    2025 arXiv

  45. [53]

    Dejie Yang, Zijing Zhao, and Yang Liu. 2025. Planllm: Video procedure planning with refinable large language models. In Proceedings of the 39th AAAI Conference on Artificial Intelligence (AAAI)

    2025

  46. [54]

    Jianwei Yang, Yonatan Bisk, and Jianfeng Gao. 2021 a . https://arxiv.org/abs/2108.09980 Taco: Token-aware cascade contrastive learning for video-text alignment . Preprint, arXiv:2108.09980

    2021 arXiv

  47. [55]

    Jinhui Yang, Xianyu Chen, Ming Jiang, Shi Chen, Louis Wang, and Qi Zhao. 2022. https://doi.org/10.1109/CVPR52688.2022.01518 Visualhow: Multimodal problem solving . In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15606--15616

    2022

  48. [56]

    Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Qimai Li, Weihan Shen, Xiaolong Zhu, and Xiu Li. 2023. https://api.semanticscholar.org/CorpusID:265352082 Using human feedback to fine-tune diffusion models without any reward model . ArXiv, abs/2311.13231

    2023 arXiv

  49. [57]

    Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. 2024. https://arxiv.org/abs/2209.00796 Diffusion models: A comprehensive survey of methods and applications . Preprint, arXiv:2209.00796

    2024

  50. [58]

    Yue Yang, Artemis Panagopoulou, Qing Lyu, Li Zhang, Mark Yatskar, and Chris Callison-Burch. 2021 b . Visual goal-step inference using wikihow. arXiv preprint arXiv:2104.05845

    2021 arXiv

  51. [59]

    Dimitri Zhukov, Jean-Baptiste Alayrac, Ramazan Gokberk Cinbis, David Fouhey, Ivan Laptev, and Josef Sivic. 2019. https://arxiv.org/abs/1903.08225 Cross-task weakly supervised learning from instructional videos . Preprint, arXiv:1903.08225

    2019 arXiv

  52. [60]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  53. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.