Pith. sign in

REVIEW 5 major objections 3 minor 52 references

StoryReasoning Dataset: Using Chain-of-Thought for Scene Understanding and Grounded Story Generation

T0 review · 5 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning a vision-language model on a dataset of 4,178 grounded movie-image stories reduces hallucinations per story by 12.3% and raises creativity by 31% compared with the base model.

desk verdict A genuinely useful grounded-story dataset and reproducible baseline, undermined by an evaluation whose headline hallucination and creativity numbers come from unreliable LLM judges and need proper statistics before being believed. read the letter →

arxiv 2505.10292 v2 pith:TQLSVTVO submitted 2025-05-15 cs.CV cs.CL

Daniel A. P. Oliveira , David Martins de Matos This is my paper · ORCID
classification cs.CVcs.CL
keywords visualstorytellingchain-of-thoughtreasoninggroundedstorygenerationcross-frameobjectre-identificationhallucinationreductionvision-languagemodelsdatasetsceneunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that visual storytelling can become both less hallucinated and more creative when every narrative reference is explicitly anchored to a visual entity and the model is forced to reason about the scene in structured steps first. It introduces StoryReasoning, a dataset of 4,178 stories built from 52,016 movie frames, in which characters and objects carry consistent IDs across frames and each story is paired with a structured scene analysis (character, object, setting, and narrative-phase tables). The paper then fine-tunes Qwen2.5-VL 7B on this data to produce Qwen Storyteller, a single model that does detection, re-identification, and grounded story generation end-to-end. The headline result is a reduction from 4.06 to 3.56 hallucinations per story (-12.3%) and a rise in creativity from 2.58 to 3.38 (+31.0%) relative to the non-fine-tuned model. A reader should care because referential hallucination—describing people, objects, or places that are not actually in the images—remains a known failure mode of large vision-language models, and the paper offers a concrete training recipe aimed directly at it.

What carries the argument

The load-bearing mechanism is a three-stage grounding pipeline. First, cross-frame object re-identification assigns persistent global IDs (char1, obj1, etc.) by combining masked-object visual embeddings from SigLIP with ArcFace face embeddings for people, using adaptive thresholds rather than motion prediction. Second, a chain-of-thought scene analysis renders each story as hierarchical tables—characters, objects, settings, and a five-phase narrative structure—so the model reasons about relations across frames before writing. Third, grounded story generation wraps every textual reference in XML tags (<gdo> for entities, <gda> for actions, <gdl> for locations and landmarks, <gdi> for image boundaries) that map back to those IDs, so even pronouns are traceable to a specific visual entity. The dataset supplies the ground-truth entity correspondences, and the fine-tuned model learns to reproduce the entire chain end-to-end.

What would settle it

Have human annotators count hallucinations and rate creativity on the same 626 generated stories per model; if the human-rated reduction in hallucinations or increase in creativity is absent or much smaller than 12.3% and 31%, the central claim fails. Alternatively, recompute the four LLM judges' per-story scores into a paired confidence interval: if the interval for the mean difference includes zero, the claimed improvement is not statistically supported.

Watch

Extended reading notes

Core claim

The central claim is that fine-tuning a 7-billion-parameter vision-language model on grounded, multi-frame stories yields measurably better visual storytelling: average hallucinations per story fall from 4.06 to 3.56 (-12.3%), creativity rises from 2.58 to 3.38 (+31.0%), and description accuracy edges up from 2.69 to 2.76, all relative to the same model without fine-tuning. The improvement is attributed to the training signal in StoryReasoning, where each story is decomposed into a chain-of-thought scene analysis and then generated with XML grounding tags that tie every character, object, action, and location mention—including pronouns—to entity IDs that persist across frames. The fine-tuned model performs object detection, cross-frame re-identification, and landmark recognition end-to-end, and it is evaluated by four large language model judges using a four-way hallucination taxonomy (object, attribute, relationship, environment) as well as automatic grounding metrics.

Load-bearing premise

The headline numbers come from four AI judges that often disagree with each other (their agreement score is 0.14 for description accuracy and 0.34 for hallucination rate), and no human check or statistical test is provided.

Editorial extensions

If this is right

  • A single model can detect, re-identify, and describe objects and characters across discontinuous movie frames without a separate detector or tracker, since Qwen Storyteller performs all three end-to-end.
  • The grounded output format makes stories machine-readable: every entity mention can be traced to a specific frame and bounding box, which supports automated consistency checks and interactive visualization.
  • Parameter-efficient training on this recipe outperforms full fine-tuning: the LoRA rank-2048 variant reaches 0.62 character recall and 0.27 mAP, while full fine-tuning reaches 0.37 and 0.16, so the recommended regimen is LoRA rather than full-model training.
  • The hallucination reduction is carried almost entirely by object and attribute hallucinations (1.84 to 1.54 and 1.47 to 1.28), while relationship hallucinations stay essentially flat (0.60 to 0.63), so the method fixes what is present in the scene better than how entities relate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the recipe transfers, the same entity-ID plus tag-grounding scheme could be applied to video captioning or embodied instruction tasks where referential consistency is scored; a cheap first test is to prompt a general vision-language model with the StoryReasoning format on an existing video-description benchmark.
  • Because the dataset is assembled from cinematic scenes with a fixed five-phase narrative arc, the creativity gains may partly encode movie conventions rather than general narrative skill; on candid, unstructured image sequences the gap might shrink.
  • The near-flat relationship-hallucination numbers point to the next bottleneck: relational grounding. Explicitly tagging interactions (subject, verb, object) rather than only entities could be a testable extension of the scheme.
  • Since no human evaluation is reported anywhere in the paper, the headline gains rest entirely on LLM judges; a human rating of the same 626 stories per model would settle whether the 31% creativity increase is perceptible to people.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 3 minor

Summary. The paper introduces StoryReasoning, a dataset of 4,178 grounded visual stories derived from 52,016 movie frames, with structured scene analysis (chain-of-thought) and XML-based grounded narratives. The authors also present Qwen Storyteller, a Qwen2.5-VL 7B model fine-tuned on this dataset. The central empirical claim is that fine-tuning reduces average hallucinations per story from 4.06 to 3.56 (-12.3%) and improves creativity from 2.58 to 3.38 (+31.0%) relative to the non-fine-tuned Qwen2.5-VL 7B model, based on ratings from four LLM judges.

Significance. If the empirical claims are established, the dataset and model release would be a useful contribution: the dataset is publicly available, contains multi-frame entity tracking with grounding tags, and the fine-tuned model is also released. The structured tabular CoT representation and the explicit grounding scheme are sensible design choices, and the paper's automatic-metric results across several training configurations provide useful baseline evidence. The main quantitative evidence for the headline improvement, however, rests entirely on LLM-judge scores with reported inter-rater reliability as low as alpha = 0.14, no human validation, and no significance testing. The central claim is therefore plausible but not yet established to the standard required for publication.

major comments (5)
  1. [§5.2, Tables 2 and 3] The headline deltas (-12.3% hallucinations, +31.0% creativity) are averages over four LLM judges whose inter-rater reliability is very low: Krippendorff's alpha is 0.14 for description accuracy, 0.20 for creativity, and 0.34 for hallucination rate. An alpha of 0.14 indicates agreement barely above chance, and the reported model differences are within the range one would expect from judge noise. The paper gives no confidence intervals, paired significance tests, effect-size distributions, or human validation. Without these, the claim that fine-tuning reduces hallucinations and improves creativity is not established.
  2. [Table 2, hallucination categories] The reduction in total hallucinations from 4.06 to 3.56 is not uniform across categories: relationship hallucinations increase from 0.60 to 0.63 after fine-tuning, while object and attribute hallucinations decrease. Since the paper motivates the work by referential hallucinations in Section 1, the increase in the relationship category, which includes incorrect actions or interactions between entities, undercuts that motivation. Moreover, no inter-rater reliability is reported for the object and attribute categories that actually drive the decrease, so the reliability of the category-level decomposition is unknown.
  3. [Appendix B, evaluation subset] The hallucination histograms in Appendix B are based on N=626 stories per model, but the main text never states that the LLM evaluation was performed on a subset of the 4,178 stories, nor how that subset was selected. If the subset is not a random sample, the average counts in Table 2 may not generalize to the full dataset. The subset size, sampling procedure, and per-judge score distributions should be reported in the main text or in the appendix.
  4. [§5.1, Table 1] Table 1 reports automatic grounding metrics (Precision, Recall, mAP, METEOR, ROUGE-L, BLEU-4) only for the four fine-tuned configurations; there is no row for the non-fine-tuned Qwen2.5-VL 7B baseline. Consequently, the claim that the fine-tuned model improves grounding over the base model is not supported by the automatic metrics. The only comparison against the base model is the low-reliability LLM evaluation in Table 2. Adding the base model to Table 1, or explaining why it cannot be evaluated with the same automatic pipeline, is necessary.
  5. [§5.2, evaluation protocol] The evaluation protocol for the LLM judges is not fully specified. The paper does not provide the exact prompts or rubrics used for description accuracy, creativity, and the four hallucination categories, nor does it give per-judge score distributions. Without these, the reader cannot reproduce the numbers or assess whether the judges were instructed to count hallucination types consistently. This is a reproducibility gap for the paper's only quantitative comparison against the base model.
minor comments (3)
  1. [§5.3, Table 3] The text says 'agreement between the three LLM evaluators,' but Section 5.2 lists four evaluators. Please clarify how many judges were used and over which judges the alpha coefficients were computed.
  2. [§5.2, decoding settings] The comparison in Table 2 does not report decoding hyperparameters (temperature, top-p, max tokens) for either model. Sampling temperature affects creativity and hallucination counts, so these settings should be specified for reproducibility.
  3. [Appendix A.4] The figure showing 100% of stories containing all five narrative phases reflects the dataset construction rule rather than an emergent property; the text should state explicitly that this is a design constraint, not an empirical finding.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central fine-tuning claim is tested against a base model via external LLM judges and standard metrics, not derived from the paper's inputs by construction.

full rationale

The central empirical claim (Sections 5.2-5.3, Tables 2-3) compares Qwen Storyteller against the non-fine-tuned Qwen2.5-VL 7B on the same image sequences. The reported reduction from 4.06 to 3.56 hallucinations per story and the creativity gain from 2.58 to 3.38 are averages over four external LLM judges, not quantities computed from the training labels or from a fitted equation, so the result is not forced by construction. The authors' citation of their own GroundCap dataset [9] explains the provenance of the image sequences and grounding scheme, but StoryReasoning is a new dataset assembled from those images with additional cross-frame re-identification, chain-of-thought analysis, and grounded-story generation; the evaluation outcome is not entailed by that provenance. No parameter is fitted to the hallucination or creativity numbers, no uniqueness theorem is imported from prior work, and no ansatz is smuggled in via citation: the grounding tags are an explicit method adapted from GroundCap, not a hidden assumption that determines the measured improvements. The low Krippendorff alpha values (0.14-0.35, Table 3) and the absence of confidence intervals or significance tests are genuine reliability threats to the empirical claim, but they are not circularity: unreliable measurement does not make the prediction equivalent to its input. Under the stated review rules, reliability concerns belong to correctness risk, not to the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities and has no fitted equations. The main numerical choices are heuristic thresholds in the re-identification and landmark pipelines, and the dataset design assumes Freytag's narrative structure and LLM-based evaluation are valid.

free parameters (5)
  • minimum frames per story = 5
    Chosen as 'empirically determined to be sufficient' in Section 3.1; affects dataset composition and the context available to the CoT process.
  • landmark confidence threshold = 0.5
    Section 3.3: below this threshold the transformer-based landmark detector fails and an LLM-based detector is used.
  • face confidence threshold = not specified
    Section 3.4: faces below a confidence threshold are excluded from ArcFace embedding extraction.
  • minimum face resolution = 128 pixels
    Section 3.4: faces below this resolution are skipped to avoid identity confusion from small-face embeddings.
  • adaptive re-identification similarity threshold = not specified
    Section 3.4: matching uses adaptive thresholding based on absolute similarity and statistical distribution; exact values are not reported, making the re-identification pipeline hard to replicate exactly.
assumptions (4)
  • domain assumption Frames selected from the same movie and sequential scenes are temporally connected enough to form a coherent narrative.
    Section 3.1: dataset construction assumes movie shot order implies narrative coherence.
  • domain assumption Automatic cross-frame re-identification via SigLIP embeddings and ArcFace produces correct ground-truth entity identities.
    Section 3.4: entity IDs from the matching algorithm are used as ground truth for training and evaluation without human validation.
  • domain assumption LLM-based 5-point Likert ratings and hallucination counts are valid measurements of story quality.
    Sections 5.2 and 5.3: headline results depend on LLM judges; the paper reports low inter-evaluator agreement (alpha 0.14 to 0.35).
  • ad hoc to paper Freytag's five-phase narrative structure applies to all stories in the dataset.
    Sections 3.5 and 3.8: all 4,178 stories are forced to contain Introduction, Development, Conflict, Turning Point, and Conclusion phases.

how reviews work

0 comments
Cite this review

Pith. "Pith review of StoryReasoning Dataset: Using Chain-of-Thought for Scene Understanding and Grounded Story Generation." pith.science (2026). https://pith.science/paper/TQLSVTVO

@misc{pith2026250510292,
  author       = {Pith},
  title        = {Pith review of: StoryReasoning Dataset: Using Chain-of-Thought for Scene Understanding and Grounded Story Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TQLSVTVO}},
  note         = {Machine review of arXiv:2505.10292}
}
read the original abstract

Visual storytelling systems struggle to maintain character identity across frames and link actions to appropriate subjects, frequently leading to referential hallucinations. These issues can be addressed through grounding of characters, objects, and other entities on the visual elements. We propose StoryReasoning, a dataset containing 4,178 stories derived from 52,016 movie images, with both structured scene analyses and grounded stories. Each story maintains character and object consistency across frames while explicitly modeling multi-frame relationships through structured tabular representations. Our approach features cross-frame object re-identification using visual similarity and face recognition, chain-of-thought reasoning for explicit narrative modeling, and a grounding scheme that links textual elements to visual entities across multiple frames. We establish baseline performance by fine-tuning Qwen2.5-VL 7B, creating Qwen Storyteller, which performs end-to-end object detection, re-identification, and landmark detection while maintaining consistent object references throughout the story. Evaluation demonstrates a reduction from 4.06 to 3.56 (-12.3%) hallucinations on average per story and an improvement in creativity from 2.58 to 3.38 (+31.0%) when compared to a non-fine-tuned model.

Figures

Figures reproduced from arXiv: 2505.10292 by the authors.

Figure 1
Figure 1. Visualization of cross-frame object re-identification. Objects with the same colored [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Distribution of frame counts across stories in the Story Reasoning dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Entity persistence across multiple frames. The graph shows the percentage of characters [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Average number of grounding references by type of entity per story. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Distribution of narrative phases across stories, showing the percentage of stories containing [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Relationship between story length (in words) and the number of characters and objects, [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Distribution of grounding references across narrative phases, showing the density of [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Comparison of pronoun usage between first, second, and third person across stories. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Grounding rates between subject pronouns and possessive pronouns. [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Character sentiment evolution across narrative phases in the StoryReasoning dataset. The [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Distribution of hallucinations in stories generated by Qwen2.5-VL 7B and fine-tuned [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: Interface for visualizing the CoT analysis tables in real-time. The tabbed interface [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Interface for visualizing the grounded story in real-time. The left panel displays the [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: The five-image sequence from sample 974 depicting a narrative arc with multiple characters [PITH_FULL_IMAGE:figures/full_fig_p026_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 43 canonical work pages

  1. [1]

    D. A. P. Oliveira, E. Ribeiro, D. M. de Matos, Story generation from visual inputs: Techniques, related tasks, and challenges, ArXiv abs/2406.02748 (2024)

    arXiv 2024

  2. [2]

    J. Li, D. Li, S. Savarese, S. Hoi, BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models, in: Proceedings of the 40th International Conference on Machine Learning, ICML’23, Journal of Machine Learning Research, 2023

    work page 2023

  3. [3]

    Zhang, Z

    Y . Zhang, Z. Ma, X. Gao, S. Shakiah, Q. Gao, J. Chai, GROUNDHOG: Grounding large language models to holistic segmentation, in: IEEE Conf. on Computer Vision and Pattern Recognition, 2024, pp. 14227–14238

    work page 2024

  4. [4]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y . Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y . Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, J. Lin, Qwen2.5-VL technical report, ArXiv abs/2502.13923 (2025)

    arXiv 2025

  5. [5]

    Cheng, I

    B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, R. Girdhar, Masked-attention mask transformer for universal image segmentation, in: IEEE Conf. on Computer Vision and Pattern Recognition, 2022, pp. 1280–1289

    work page 2022

  6. [6]

    X. Hong, A. Sayeed, K. Mehra, V . Demberg, B. Schiele, Visual writing prompts: Character- grounded story generation with curated image sequences, Transactions of the Association for Computational Linguistics 11 (2023) 565–581

    work page 2023

  7. [7]

    Farquhar, J

    S. Farquhar, J. Kossen, L. Kuhn, Y . Gal, Detecting hallucinations in large language models using semantic entropy, Nature 630 (8017) (2024) 625–630

    work page 2024

  8. [8]

    Huang, W

    L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, T. Liu, A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, ACM Trans. Inf. Syst. 43 (2) (Jan. 2025)

    work page 2025

Show all 52 references
  1. [9]

    D. A. P. Oliveira, L. Teodoro, D. M. de Matos, GroundCap: A visually grounded image captioning dataset, ArXiv abs/2502.13898 (2025)

    2025 arXiv

  2. [10]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V . Le, D. Zhou, Chain-of-thought prompting elicits reasoning in large language models, in: Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Curran As...

    2022

  3. [11]

    Zhang, A

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, A. J. Smola, Multimodal chain-of-thought reasoning in language models, Trans. Mach. Learn. Res. 2024 (2023)

    2023

  4. [12]

    Zheng, L

    W. Zheng, L. Yan, C. Gou, F.-Y . Wang, Two heads are better than one: Hypergraph-enhanced graph reasoning for visual event ratiocination, in: M. Meila, T. Zhang (Eds.), Proceedings of the 38th International Conference on Machine Learning, V ol. 139 of Proceedings of Machine Le...

    2021

  5. [13]

    X. E. Wang, J. Wu, J. Chen, L. Li, Y . fang Wang, W. Y . Wang, VaTeX: A large-scale, high- quality multilingual dataset for video-and-language research, 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019) 4580–4590. 10

    2019

  6. [14]

    OpenAI, Hello GPT-4o, https://openai.com/index/hello-gpt-4o , accessed: 2025-03- 18 (2024)

    2024

  7. [15]

    Anthropic, Claude 3.7 sonnet and claude code, https://www.anthropic.com/news/ claude-3-7-sonnet , accessed: 2025-03-18 (2025)

    2025

  8. [16]

    Google, Gemini 2.5 flash is now in preview, https://blog.google/products/gemini/ gemini-2-5-flash-preview/ , accessed: 2025-04-30 (2025)

    2025

  9. [17]

    T.-H. K. Huang, F. Ferraro, N. Mostafazadeh, I. Misra, A. Agrawal, J. Devlin, R. Girshick, X. He, P. Kohli, D. Batra, C. L. Zitnick, D. Parikh, L. Vanderwende, M. Galley, M. Mitchell, Visual storytelling, in: Proceedings of the 2016 Conference of the North American Chapter of ...

    2016

  10. [18]

    Y . Yu, J. Chung, H. Yun, J. Kim, G. Kim, Transitional adaptation of pretrained models for visual storytelling, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 12658–12668

    2021

  11. [19]

    Y . Zhou, D. Zhou, M.-M. Cheng, J. Feng, Q. Hou, Storydiffusion: Consistent self-attention for long-range image and video generation, in: The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  12. [20]

    H. Chen, R. Han, T.-L. Wu, H. Nakayama, N. Peng, Character-centric story visualization via visual planning and token alignment, in: Y . Goldberg, Z. Kozareva, Y . Zhang (Eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Association ...

    2022

  13. [21]

    K. Park, M. Kim, K. Jung, A character-centric creative story generation via imagination, ArXiv abs/2409.16667 (2024)

    2024 arXiv

  14. [22]

    Z. Liu, A. Bahety, S. Song, REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction, in: J. Tan, M. Toussaint, K. Darvish (Eds.), Proceedings of The 7th Conference on Robot Learning, V ol. 229 of Proceedings of Machine Learning Research, PMLR, 2023, pp. 3468–3484

    2023

  15. [23]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, IEEE Intl. Conf. on Computer Vision (2021)

    2021

  16. [24]

    Weyand, A

    T. Weyand, A. F. de Araújo, B. Cao, J. Sim, Google landmarks dataset v2 – a large-scale bench- mark for instance-level recognition and retrieval, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020) 2572–2581

    2020

  17. [25]

    Welch, G

    G. Welch, G. Bishop, An introduction to the kalman filter, Tech. Rep. TR 95-041, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA (1995)

    1995

  18. [26]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: International Conference on Machine Learning, 2021

    2021

  19. [27]

    Schuhmann, R

    C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, A. Komatsuzaki, LAION-400M: Open dataset of CLIP-filtered 400 million image-text pairs, ArXiv abs/2111.02114 (2021)

    2021 arXiv

  20. [28]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jégou, J. Mairal, P. Labatut,...

    2024

  21. [29]

    X. Zhai, B. Mustafa, A. Kolesnikov, L. Beyer, Sigmoid loss for language image pre-training, 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (2023) 11941–11952. 11

    2023

  22. [30]

    Alabdulmohsin, X

    I. Alabdulmohsin, X. Zhai, A. Kolesnikov, L. Beyer, Getting vit in shape: scaling laws for compute-optimal model design, in: Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Curran Associates Inc., Red Hook, NY , USA, 2023

    2023

  23. [31]

    J. Deng, J. Guo, N. Xue, S. Zafeiriou, Arcface: Additive angular margin loss for deep face recognition, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019

    2019

  24. [32]

    Die Technik des Dramas

    G. Freytag, Freytag’s Technique of the Drama: An Exposition of Dramatic Composition and Art, Scott, Foresman and Company, Chicago, 1894, an authorized translation from the 6th German edition of “Die Technik des Dramas”

  25. [33]

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chen, LoRA: Low-Rank Adaptation of Large Language Models, in: Intl. Conf. on Learning Representations, 2022

    2022

  26. [34]

    Loshchilov, F

    I. Loshchilov, F. Hutter, SGDR: stochastic gradient descent with warm restarts, in: 5th Interna- tional Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017

    2017

  27. [35]

    Loshchilov, F

    I. Loshchilov, F. Hutter, Decoupled weight decay regularization, in: International Conference on Learning Representations, New Orleans, LA, USA, 2019, paper originally submitted in November 2017 on arXiv (arXiv:1711.05101)

    2019 arXiv

  28. [36]

    B. Hsu, Y . Dai, V . Kothapalli, Q. Song, S. Tang, S. Zhu, S. Shimizu, S. Sahni, H. Ning, Y . Chen, Liger kernel: Efficient triton kernels for llm training, ArXiv abs/2410.10989 (2024)

    2024 arXiv

  29. [37]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, A. Zisserman, The pascal visual object classes (voc) challenge, International Journal of Computer Vision 88 (2) (2010) 303–338

    2010

  30. [38]

    Banerjee, A

    S. Banerjee, A. Lavie, METEOR: An automatic metric for MT evaluation with improved correlation with human judgments, in: J. Goldstein, A. Lavie, C.-Y . Lin, C. V oss (Eds.), ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarizatio...

    2005

  31. [39]

    Krippendorff, Estimating the reliability, systematic error and random error of interval data, Educational and Psychological Measurement 30 (1970) 61 – 70

    K. Krippendorff, Estimating the reliability, systematic error and random error of interval data, Educational and Psychological Measurement 30 (1970) 61 – 70

    1970

  32. [40]

    C. J. Hutto, E. Gilbert, V ADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text, in: Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media, V ol. 8, AAAI Press, Ann Arbor, Michigan, USA, 2014, pp. 216–225

    2014

  33. [41]

    we´´ at 99.5%, “you

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, I. Stoica, Efficient memory management for large language model serving with pagedattention, in: Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. A Dataset An...

    2023

  34. [42]

    0-person-0

    Each detected object has a unique object ID (like "0-person-0", "1-car-5")

  35. [43]

    (for people or characters) - Objects: obj1, obj2, etc

    I will provide a mapping from these object IDs to consistent entity IDs, which come in four types: - Characters: char1, char2, etc. (for people or characters) - Objects: obj1, obj2, etc. (for things) - Landmarks: lm1, lm2, etc. (for recognizable places/landmarks) - Background:...

  36. [44]

    Objects with the same entity ID represent the same element appearing across multiple images

  37. [45]

    Bounding box coordinates are provided in pixel values (actual image dimensions) as x1,y1,x2,y2

  38. [46]

    landmark-ID: landmark-name: x1,y1,x2,y2

    Landmarks provide additional context about the setting and location - Landmarks are formatted as "landmark-ID: landmark-name: x1,y1,x2,y2" - If landmarks appear in multiple images, they will have entity IDs in the mapping section

  39. [47]

    dog" that is actually a

    Some objects may be incorrectly classified or detected: - You can override object clas- sifications if they’re clearly wrong (e.g., a "dog" that is actually a "cat") - You can ignore detections if no actual object exists at that location - Use your visual understanding to corr...

  40. [48]

    Image grounding: <gdi image1>Text describing events in image 1</gdi> Each part of the story must be inside a image tag indicating which image it describes

  41. [49]

    Character and action tags: For character references: <gdo char1>Character name, pronoun or description</gdo> or <gdo char1 char2>They</gdo> for multiple characters For character actions: <gda char1>action description</gda> or <gda char1 char2>action description</gda> for multi...

  42. [50]

    Object grounding: <gdo obj1>Object reference</gdo> or <gdo obj1 obj2>Objects reference</gdo> Use this for specific objects in the scene

  43. [51]

    Landmark grounding: <gdl lm1>Landmark description</gdl> or <gdl lm1 lm2>Landmarks description</gdl> Use this for landmarks or recognizable locations

  44. [52]

    Is this really necessary, Tom?

    Background grounding: <gdl bg1>Background element description</gdl> or <gdl bg1 bg2>Background elements description</gdl> Use this for background elements or general settings Example of properly formatted text: <gdi image1> <gdo char1>Sarah</gdo> <gda char1>held</gda> <gdo obj...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.