Pith. sign in

REVIEW 4 major objections 5 minor 26 references

Entity Re-identification in Visual Storytelling via Contrastive Reinforcement Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training a visual storyteller on deliberately incoherent image sequences improves cross-frame entity re-identification and grounding.

desk verdict Plausible contrastive-DPO idea for visual storytelling, but the reward and evaluation both measure label persistence rather than true re-identification, so the headline gains aren't established. read the letter →

arxiv 2507.07340 v2 pith:IDV7O2P3 submitted 2025-07-09 cs.CV

Daniel A. P. Oliveira , David Martins de Matos This is my paper · ORCID
classification cs.CV
keywords visualstorytellingentityre-identificationcross-frameconsistencycontrastivereinforcementlearningdirectpreferenceoptimizationsyntheticnegativestoriesgroundingpronoun
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a visual storytelling model can learn when to connect characters and objects across frames if it is trained not only on coherent stories but also on deliberately incoherent ones. It constructs synthetic negative stories by sampling frames from unrelated movies, then fine-tunes a Qwen2.5-VL 7B-based storyteller with Direct Preference Optimization—a training method that optimizes a policy directly from preference pairs—under a dual reward that rewards grounding and cross-frame re-identification on real stories and penalizes both on synthetic ones. Against a supervised fine-tuned baseline, grounding mAP rises from 0.27 to 0.31, grounding F1 from 0.35 to 0.41, pronoun grounding improves for nearly every pronoun type except "its", and the share of entities appearing in five or more frames rises from 29.3% to 33.3%. The authors take this as evidence that explicit positive/negative training on entity connectivity reduces referential inconsistency and hallucination in visual storytelling.

What carries the argument

The load-bearing mechanism is contrastive Direct Preference Optimization over synthetic negative stories. The paper extends the Story Reasoning dataset with 4,178 synthetic stories built by deterministically sampling 5 to 15 frames from far-apart real stories, giving a 2:1 ratio of real to synthetic stories. The reward function combines an entity re-identification component, which measures how many frames each detected character and object persists across (characters weighted 0.6, objects 0.4), with a grounding component, which measures the fraction of pronouns and proper nouns placed inside entity tags; the re-identification term is inverted for synthetic stories so that connecting entities is penalized, and structurally invalid outputs receive $-1.0$. Direct Preference Optimization converts this reward into offline preference pairs, requiring the chosen response to beat the rejected one by at least 0.05, and optimizes the policy directly.

What would settle it

Evaluate the same contrastive RL model on a subset of Story Reasoning where entity IDs and bounding boxes have been manually corrected or where re-identification uses whole-image context: if the reported mAP and F1 gains shrink or disappear, the improvement is an artifact of noisy automatic annotations rather than genuine entity re-identification. A second decisive test is to replace the cross-movie synthetic negatives with shuffled-frame sequences from the same movie; if the gains vanish, the model learned a movie-level incoherence cue rather than entity-level discrimination.

Watch

Extended reading notes

Core claim

The central claim is that cross-frame entity re-identification in visual storytelling is a trainable behavior, not an emergent byproduct of scale or language modeling. A model that is explicitly rewarded for linking entities in coherent sequences and penalized for linking them in incoherent sequences learns to persist characters and objects across frames: the authors report grounding mAP rising from 0.27 to 0.31, grounding F1 from 0.35 to 0.41, pronoun grounding gains for every pronoun type except "its", and cross-frame entity persistence increasing across all frame counts, with entities appearing in five or more frames rising from 29.3% to 33.3%. The authors also report that well-structured stories containing both chain-of-thought and grounded story increased from 79.1% to 97.5%, and that standard language metrics did not degrade.

Load-bearing premise

The load-bearing premise is that the Story Reasoning annotations—entity identifiers, bounding boxes, and the adapted mAP computation—are accurate enough to serve both as the reward that trains the model and as the ground truth that measures improvement; the paper itself notes that re-identification was based on visual similarity within cropped boxes and that bounding boxes were not validated.

Editorial extensions

If this is right

  • Adding synthetic negative stories and contrastive DPO to a 7B visual storyteller improves grounding mAP by 14.8% and grounding F1 by 17.1% over the supervised fine-tuned baseline.
  • Cross-frame entity persistence improves at every frame count, with the share of entities appearing in five or more frames rising from 29.3% to 33.3%, indicating more reliable re-identification.
  • Pronoun grounding improves across nearly all pronoun types, most sharply for gender-specific "he" (to 99.1%) and "she" (to 98.6%), reducing ambiguous references.
  • Well-structured outputs containing both chain-of-thought and grounded story rise from 79.1% to 97.5%, showing the reward's structural validation also shapes output compliance.
  • The contrastive framework teaches the model when not to establish cross-frame connections, targeting the false connections that arise when visually similar entities appear in unrelated images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not test same-movie shuffled frames as negatives; because their synthetic stories come from different movies, the model may be learning a coarse "different movie means no connection" heuristic rather than fine-grained entity discrimination. Shuffled same-movie frames would separate those two explanations.
  • The reward and evaluation both rely on Story Reasoning annotations whose re-identification was itself based on visual similarity within cropped bounding boxes; training on that signal may entrench the context-blindness the authors identify, such as two same-colored cars from different scenes. A human-verified subset of boxes and IDs would reveal whether the gains survive cleaner supervision.
  • Table 2 shows the best model still scores 0.67 on synthetic-story re-identification versus 0.72 for the baseline, so it continues to connect entities in incoherent sequences; the reward analysis suggests additional training on negatives or harder negatives could push this discrimination further.
  • The framework of synthetic negatives plus a dual rule-based reward is directly portable to video captioning and visual question answering, which the authors list as future work, though the format tags and story-specific structure validation would need to be redefined for those tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a contrastive reinforcement learning framework to improve entity re-identification and grounding in visual storytelling. It extends the Story Reasoning dataset with synthetic negative stories, designs a dual-component rule-based reward function (re-identification and grounding), and fine-tunes the Qwen Storyteller model (Qwen2.5-VL 7B) using Direct Preference Optimization. The authors report improvements over a supervised fine-tuned baseline, including grounding mAP from 0.27 to 0.31 (+14.8%), F1 from 0.35 to 0.41 (+17.1%), and increased cross-frame entity persistence. The central claim is that contrastive RL with synthetic negatives teaches the model when to establish entity connections across frames, improving genuine re-identification and grounding.

Significance. The problem of cross-frame entity consistency in visual storytelling is timely and important, and the idea of using synthetic negative examples with DPO is a plausible and interesting direction. The paper is clearly written and the authors release the trained models and synthetic dataset, which supports reproducibility. However, the evaluation is largely circular with the reward objective, and the reward itself measures label persistence rather than verified visual identity. As a result, the reported improvements do not convincingly establish the central claim. The lack of statistical validation and the omission of the promised full fine-tuning experiment further weaken the evidence. If the core issues were addressed, the contrastive framework could be a useful contribution, but in its current form the results are not reliable.

major comments (4)
  1. [Section 3.2.2, Eqs. (3)-(4)] The re-identification reward R_reid is computed as the average number of frames per entity label in the model's generated chain-of-thought, normalized by the total frame count. It never verifies that two occurrences of the same label correspond to the same physical entity, nor that different labels refer to distinct identities. A degenerate policy that reuses a single 'char1' label in every frame scores R_reid = 1 on real stories with no visual evidence. Consequently, the persistence gains reported in Fig. 1 and Table 2 can be manufactured by label reuse rather than by genuine cross-frame re-identification, which is the central claim of the paper.
  2. [Section 4.1 and Section 3.2] The evaluation metrics (mAP, precision, recall, F1, persistence) are computed against the Story Reasoning annotations, which are the same annotations used to define the reward function in Eqs. (1)-(5). The reward is computed from the model's own generated CoT and story using these annotations, and the evaluation measures the same persistence and grounding quantities against the same ground-truth source. This circularity means the reported improvements may simply reflect the model learning to reproduce the annotation conventions rather than learning to track visual entities. The paper itself acknowledges in Section 5 that the underlying bounding boxes are not validated, further compromising both the reward signal and the evaluation.
  3. [Section 3.3 and Table 1] The text states that a second experiment uses full fine-tuning to assess the impact of training all model parameters, but Table 1 reports only the baseline and LoRA variants (R=512, 1024, 2048). The full fine-tuning results are absent from the table and from the rest of the paper. This omission makes it impossible to verify the authors' claim that the approach is effective beyond parameter-efficient fine-tuning, and it leaves a described experiment unreported.
  4. [Section 4] No error bars, confidence intervals, or statistical significance tests are reported anywhere in the evaluation. All metrics are single-run point estimates, and several differences are very small (e.g., BLEU-4 0.054 vs. 0.057, ROUGE-L 0.16 vs. 0.18). Without any measure of variance, the robustness of the claimed improvements cannot be assessed, especially given the multiple free parameters (alpha, beta, gamma, delta, DPO temperature, real-to-synthetic ratio, preference threshold, etc.) that are not varied in a sensitivity analysis.
minor comments (5)
  1. [Section 3.1] The paper states a 2:1 ratio of real to synthetic stories, but later says the synthetic construction creates 4,178 synthetic stories alongside 4,178 real stories, which is a 1:1 ratio. Please clarify the intended ratio.
  2. [Section 4.1] The text refers to 'LoRA rank 2028' when comparing with the baseline, but Table 1 lists the LoRA rank as 2048. This appears to be a typo.
  3. [Section 3.3] The sentence 'Both experiments employee the temperature parameter' should read 'employ' instead of 'employee'.
  4. [Table 1 caption] The caption states that best and worst values are highlighted, but the table as rendered in the manuscript shows no highlighting. Please add highlighting or remove the statement.
  5. [Section 4.4] The discussion of Table 2 is ambiguous: for synthetic stories, a lower R_reid is the desired outcome (since the reward is 1.0 - (alpha*R_char + beta*R_obj)), yet the authors say the Rank 2048 model's decrease from 0.72 to 0.67 indicates the model 'would benefit from more training on negative stories.' Clarify the desired direction and interpretation.

Circularity Check

3 steps flagged · score 6.0 of 10

Rreid (Eq. 3) and the Fig. 1 'persistence' headline are the same label-frame count over the model's own CoT; Rgrounding (Eq. 5) directly controls the reported pronoun grounding percentage. With no identity check and a self-cited, unvalidated ground truth, the reported gains largely reflect reward optimization, not re-identification.

  1. fitted input called prediction [Section 3.2.2 (Eqs. 2-4), Section 4.1, Fig. 1; Abstract]
    "Rchar = min(1.0, Σ_{ci∈C}|F_{ci}| / (|C|×|I|)) ... This formulation rewards models for re-identifying entities across frames in authentic stories while penalizing re-identification of entities in synthetic stories. ... We measure entity persistence by tracking characters and objects that appear across multiple frames ... showing what percentage of all entities from all stories appear in N or more frames, demonstrating improved entity re-identification."

    The persistence metric in Fig. 1 is the same per-label frame count that Eq. 3 maximizes: both count, for each detected CoT label, how many frames contain that label. Since Rreid is computed from the model's own generated CoT, a model can raise both the reward and the reported 'persistence' by emitting one recurring label in every frame without checking whether frames show the same physical entity. The abstract's headline gain (entities appearing in 5+ frames, 29.3% to 33.3%) is therefore the training objective itself measured again, not an independent measurement of re-identification.

  2. fitted input called prediction [Section 3.2.3 (Eq. 5), Section 4.3, Fig. 2]
    "Rgrounding(s) = γ×(Gchar+Pchar)/Tchar + δ×(Gobj+Pobj)/Tobj ... We extract grounded references using regular expressions to identify entity tags, then employ spaCy for part-of-speech analysis to classify the content within those tags as pronouns or proper nouns."

    The reward directly increases the fraction of pronouns/proper nouns that appear inside entity tags, and Fig. 2's reported 'grounding accuracy' is the same tagged-versus-untagged percentage. Optimizing Eq. 5 therefore reduces the very quantity reported as improved grounding. The observed precision drop (0.57 to 0.45) with recall rise (0.40 to 0.48) is exactly the signature of tagging more referential expressions regardless of whether the tag points to the correct visual entity, and Eq. 5 never checks correspondence to a bounding box.

1 more flagged steps
  1. self citation load bearing [Section 1 and Section 5; Section 4.1 (mAP adaptation)]
    "The entity re-identification approach used to generate the Story Reasoning dataset relies primarily on visual similarity within cropped bounding boxes, without considering the whole image context. ... Despite improved entity re-identification performance, we do not validate whether the underlying bounding boxes accurately correspond to the referenced objects."

    The evaluation uses the authors' own Story Reasoning dataset as ground truth for mAP and persistence. The paper itself states that this dataset's entity re-identification can create incorrect connections between visually similar but contextually distinct entities, and it explicitly does not validate the bounding boxes. The central claim of improved re-identification thus rests on a self-cited, admittedly unvalidated labeler; without independent identity annotations, the reported numbers cannot establish genuine cross-frame identity tracking.

full rationale

The DPO loss itself (Eq. 6) is standard and not circular. The circularity is in the evidence chain. Eq. 3 computes Rreid from label-frame counts in the generated CoT, and Section 4.1/Fig. 1 report essentially the same label-frame counts as 'entity persistence' and headline them as improved re-identification. Because the reward never compares a repeated label to ground-truth identity, both training and evaluation can be satisfied by label reuse alone. Similarly, Eq. 5's grounding reward counts pronouns/proper nouns inside entity tags, and Fig. 2's 'grounding accuracy' is the same ungrounded-percentage; the observed precision drop and recall rise are the expected result of optimizing tag coverage without correctness. The benchmark ground truth is the authors' Story Reasoning dataset, whose re-identification the paper itself says relies on cropped-region visual similarity and whose boxes are not validated. Thus the reported gains largely reduce to matching the training objective on a self-cited, unvalidated benchmark. If the reward and evaluation were checked against independent, validated identity annotations, the result could be informative; as reported, the central claim is partially circular, so the score is 6 rather than 0.

Assumptions & free parameters 8 free parameters · 4 assumptions · 0 invented entities

The method relies on no new physical or conceptual entities, but the evaluation depends heavily on the authors' own dataset, metrics, and baseline, which are loaded into the ledger here as domain assumptions rather than external ground truth.

free parameters (8)
  • Reward weight alpha (character re-ID) = 0.6
    Hand-chosen to prioritize characters in Eq. 2.
  • Reward weight beta (object re-ID) = 0.4
    Hand-chosen complement of alpha in Eq. 2.
  • Reward weight gamma (character grounding) = 0.5
    Hand-chosen in Eq. 5 to weight character grounding equally with object grounding.
  • Reward weight delta (object grounding) = 0.5
    Hand-chosen in Eq. 5.
  • DPO temperature beta = 0.1
    KL constraint strength in Eq. 6; chosen without sensitivity analysis.
  • Real-to-synthetic story ratio = 2:1
    Training mix; the paper notes it may not be optimal (Section 5).
  • Preference reward gap threshold = 0.05
    Minimum reward difference for choosing a preferred response in DPO pair generation (Section 3.3).
  • Synthetic story length range = 5 to 15 frames
    Deterministic sampling range for synthetic negatives (Section 3.1).
assumptions (4)
  • domain assumption Story Reasoning dataset entity IDs and bounding boxes are accurate enough to serve as training signal and evaluation ground truth.
    Used throughout for reward computation and for the mAP/F1 evaluation; the paper admits in Section 5 that bounding boxes are not validated and may not contain the referenced objects.
  • domain assumption Synthetic stories sampled from different movies via the deterministic formula are visually incoherent.
    Section 3.1 assumes story indices far apart in dataset ordering imply different movies; no verification of actual movie or source identity is reported.
  • domain assumption The mAP adaptation from Oliveira and de Matos (2025) is a valid measure of grounding quality.
    Section 4.1 uses this metric without independent validation or comparison to standard referring-expression benchmarks.
  • domain assumption Rule-based rewards avoid reward hacking (following DeepSeek-R1).
    Section 3.2 adopts this assumption; the paper's own reward is computed on the generated structure, which the model can learn to game, as evidenced by precision dropping from 0.57 to 0.45.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Entity Re-identification in Visual Storytelling via Contrastive Reinforcement Learning." pith.science (2026). https://pith.science/paper/IDV7O2P3

@misc{pith2026250707340,
  author       = {Pith},
  title        = {Pith review of: Entity Re-identification in Visual Storytelling via Contrastive Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IDV7O2P3}},
  note         = {Machine review of arXiv:2507.07340}
}
read the original abstract

Visual storytelling systems, particularly large vision-language models, struggle to maintain character and object identity across frames, often failing to recognize when entities in different images represent the same individuals or objects, leading to inconsistent references and referential hallucinations. This occurs because models lack explicit training on when to establish entity connections across frames. We propose a contrastive reinforcement learning approach that trains models to discriminate between coherent image sequences and stories from unrelated images. We extend the Story Reasoning dataset with synthetic negative examples to teach appropriate entity connection behavior. We employ Direct Preference Optimization with a dual-component reward function that promotes grounding and re-identification of entities in real stories while penalizing incorrect entity connections in synthetic contexts. Using this contrastive framework, we fine-tune Qwen Storyteller (based on Qwen2.5-VL 7B). Evaluation shows improvements in grounding mAP from 0.27 to 0.31 (+14.8%), F1 from 0.35 to 0.41 (+17.1%). Pronoun grounding accuracy improved across all pronoun types except "its", and cross-frame character and object persistence increased across all frame counts, with entities appearing in 5 or more frames advancing from 29.3% to 33.3% (+13.7%). Well-structured stories, containing the chain-of-thought and grounded story, increased from 79.1% to 97.5% (+23.3%).

Figures

Figures reproduced from arXiv: 2507.07340 by the authors.

Figure 1
Figure 1. Cross-frame entity persistence comparison between baseline Qwen Storyteller model (left) and contrastive RL model with LoRA fine-tuning R [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Pronoun grounding performance comparison between baseline Qwen Storyteller model (left) and contrastive RL model with LoRA fine-tuning R [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 19 canonical work pages

  1. [1]

    and Lavie, A

    Banerjee, S. and Lavie, A. (2005). METEOR : An automatic metric for MT evaluation with improved correlation with human judgments. In Goldstein, J., Lavie, A., Lin, C.-Y., and Voss, C., editors, ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization , pages 65--72, Ann Arbor, Michigan. ACL

    work page 2005

  2. [2]

    Chen, W., Li, X., Su, J., Zhu, G., Li, Y., Ji, Y., and Liu, C. (2024). TARN - VIST : Topic aware reinforcement network for visual storytelling. In Calzolari, N., Kan, M.-Y., Hoste, V., Lenci, A., Sakti, S., and Xue, N., editors, Proceedings of the 2024 Joint International Conf. on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2...

    work page 2024

  3. [3]

    F., Leike, J., Brown, T

    Christiano, P. F., Leike, J., Brown, T. B., Martic, M., Legg, S., and Amodei, D. (2017). Deep reinforcement learning from human preferences. ArXiv , abs/1706.03741

    arXiv 2017

  4. [4]

    DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J.-M., and et al. (2025). DeepSeek-R1 : Incentivizing reasoning capability in LLMs via reinforcement learning. ArXiv , abs/2501.12948

    arXiv 2025

  5. [5]

    Farquhar, S., Kossen, J., Kuhn, L., and Gal, Y. (2024). Detecting hallucinations in large language models using semantic entropy. Nature , 630(8017):625--630

    work page 2024

  6. [6]

    F\" u rst, A., Rumetshofer, E., Lehner, J., and et al. (2022). CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP . In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A., editors, Advances in Neural Information Processing Systems , volume 35, pages 20450--20468. Curran Associates, Inc

    work page 2022

  7. [7]

    Hong, X., Sayeed, A., Mehra, K., Demberg, V., and Schiele, B. (2023). Visual writing prompts: Character-grounded story generation with curated image sequences. Transactions of the Association for Computational Linguistics , 11:565--581

    work page 2023

  8. [8]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022). LoRA: Low-Rank Adaptation of Large Language Models . In Intl. Conf. on Learning Representations

    work page 2022

Show all 26 references
  1. [9]

    Huang, L., Yu, W., Ma, W., and Zhong, e. a. (2025). A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. , 43(2)

    2025

  2. [10]

    K., Ferraro, F., Mostafazadeh, N., Misra, I., and et al

    Huang, T.-H. K., Ferraro, F., Mostafazadeh, N., Misra, I., and et al. (2016). Visual storytelling. In Proceedings of the 2016 Conf. of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 1233--1239, San Diego, Calif...

    2016

  3. [11]

    Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., and et al. (2021). Scaling up visual and vision-language representation learning with noisy text supervision. In International Conf. on Machine Learning

    2021

  4. [12]

    Li, Y., Liang, F., Zhao, L., Cui, Y., Ouyang, W., Shao, J., Yu, F., and Yan, J. (2022). Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm. In International Conf. on Learning Representations

    2022

  5. [13]

    Lin, C.-Y. (2004). ROUGE : A package for automatic evaluation of summaries. In Text Summarization Branches Out , pages 74--81, Barcelona, Spain. Association for Computational Linguistics

    2004

  6. [14]

    and Hutter, F

    Loshchilov, I. and Hutter, F. (2019). Decoupled weight decay regularization. In International Conf. on Learning Representations , New Orleans, LA, USA. Paper originally submitted in November 2017 on arXiv (arXiv:1711.05101)

    2019 arXiv

  7. [15]

    Oliveira, D. A. P. and de Matos, D. M. (2025). StoryReasoning dataset: Using chain-of-thought for scene understanding and grounded story generation

    2025

  8. [16]

    Oliveira, D. A. P., Ribeiro, E., and de Matos, D. M. (2024). Story generation from visual inputs: Techniques, related tasks, and challenges. ArXiv , abs/2406.02748

    2024 arXiv

  9. [17]

    Oliveira, D. A. P., Teodoro, L., and de Matos, D. M. (2025). GroundCap : A visually grounded image captioning dataset. ArXiv , abs/2502.13898

    2025 arXiv

  10. [18]

    L., and et al

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., and et al. (2022). Training language models to follow instructions with human feedback. ArXiv , abs/2203.02155

    2022 arXiv

  11. [19]

    Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics , ACL '02, page 311–318, USA. Association for Computational Linguistics

    2002

  12. [20]

    W., Hallacy, C., Ramesh, A., and et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., and et al. (2021). Learning transferable visual models from natural language supervision. In International Conf. on Machine Learning

    2021

  13. [21]

    D., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. (2023). Direct preference optimization: your language model is secretly a reward model. In Proceedings of the 37th International Conference on Neural Information Processing Systems , NIPS '23, Red ...

    2023

  14. [22]

    J., Marcheret, E., Mroueh, Y., Ross, J., and Goel, V

    Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., and Goel, V. (2016). Self-critical sequence training for image captioning. 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , pages 1179--1195

    2016

  15. [23]

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. ArXiv , abs/1707.06347

    2017 arXiv

  16. [24]

    Song, Y., Paperno, D., and Gatt, A. (2024). Context-aware visual storytelling with visual prefix tuning and contrastive learning. In Mahamood, S., Minh, N. L., and Ippolito, D., editors, Proceedings of the 17th International Natural Language Generation Conf. , pages 384--401, ...

    2024

  17. [25]

    Wang, X., Chen, W., Wang, Y.-F., and Wang, W. Y. (2018). No metrics are perfect: Adversarial reward learning for visual storytelling. In Gurevych, I. and Miyao, Y., editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long...

    2018

  18. [26]

    Yu, Y., Chung, J., Yun, H., Kim, J., and Kim, G. (2021). Transitional adaptation of pretrained models for visual storytelling. In Proceedings of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) , pages 12658--12668

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.