Pith. sign in

REVIEW 3 major objections 6 minor 26 references

Vision-Language Model Dialog Games for Self-Improvement

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning a vision-language model on successful self-play dialog games, filtered automatically by the Guesser's correct pick, improves performance on downstream VQA benchmarks and on robotics success detection, with gains that grow over…

desk verdict A plausible and well-executed self-improvement loop for VLMs, but the general VQA experiments lack a non-dialog fine-tuning control, so the headline claim is conditional until that baseline is added. read the letter →

arxiv 2502.02740 v1 pith:W5VGGY53 submitted 2025-02-04 cs.LG cs.AI

Ksenia Konyushkova , Christos Kaplanis , Serkan Cabi , Misha Denil This is my paper · ORCID
classification cs.LGcs.AI
keywords vision-languagemodelsself-improvementdialoggamesself-playvisualquestionansweringroboticssuccessdetectionsyntheticdatareference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a vision-language model (VLM) can improve itself without new human-annotated multimodal data by playing a two-agent dialog game over unlabeled images: a Describer answers questions about a hidden target image while a Guesser asks questions and finally picks the target from a set of distractors. Dialogs that end in a correct pick are kept; the rest are discarded, and the model is fine-tuned on the surviving question-answer turns and guessing moves. The claim is that this self-generated, automatically filtered data transfers: fine-tuning on DOCCI or OpenImages games raised VQAv2 yes/no accuracy from 73.0% to 79.8% or 83.4% respectively, and two rounds of games built from ALOHA robot videos raised success detection from 56.5% to 73.0%, beating baselines that fine-tune on image descriptions or self-generated QA pairs. If right, this is a scalable path for data-scarce domains, because the game needs only unlabeled images from the target domain and no extra supervision.

What carries the argument

The central mechanism is the VLM Dialog Game, a reference-game variant played entirely by two prompted copies of the same VLM: the Describer answers questions about one hidden target image, and the Guesser, seeing the target among $N$ images, asks information-seeking questions, maintains a running summary, and eventually guesses an index. The game's load-bearing feature is automatic, objective success detection: only dialogs in which the Guesser's final pick is the real target, and which survive re-runs with the target in every position, enter the training set. Each successful game yields training examples for both roles, so the same data supervises question-asking, answering, and summarising; game difficulty is tuned through the number of distractors and by grouping visually similar images, and the whole process can be re-run with the improved model as a new player. In the paper's framing the game completes a cycle — image to dialog and back to image via contrastive selection — that avoids needing a text-to-image generator.

What would settle it

Fine-tune the same base model on non-dialog image-description or question-answer pairs from the same OpenImages or DOCCI images, matched for data volume and tokens; if VQAv2 yes/no accuracy reaches the same ~83% level, the claim that the game structure drives the improvement is falsified. For the iterative claim, a third round of robotics games in which game success and success detection stop rising would also undercut it.

Watch

Extended reading notes

Core claim

The paper's core discovery is that goal-oriented self-play, filtered by success, is a viable source of training signal for VLMs. In the VLM Dialog Game, the Guesser's final selection determines success, and the authors add a permutation check (the dialog is re-run with the target placed at each position) to avoid rewarding guesses that happen to be right by chance. Fine-tuning on successful dialogs improves both game play and downstream understanding: the OpenImages-game model reaches 83.4% on VQAv2 yes/no (from 73.0%), and the DOCCI-game model reaches 79.8% yes/no and lifts counting accuracy from 56% to 58.3%, gains that transfer from the game's source images to the unseen VQAv2 test set. In the robotics domain the improvement compounds across rounds: success detection rises from 56.5% (base) to 69.5% after round 1 and to 73.0% after round 2, with game success rising from 14.4% to 53.7%, and the full dialog data beats both description-SFT (65.0%) and Self-QA (67.0%) baselines. The authors also show that the Guesser's questions, not just the Describer's answers, are what makes iterative improvement possible, since an answers-only model reaches comparable success detection but cannot improve its own game play.

Load-bearing premise

That the VQAv2 gains in the general experiments are caused by the goal-oriented dialog-game structure itself, rather than by simply fine-tuning on a comparable volume of image-text pairs drawn from the same source images.

Editorial extensions

If this is right

  • Fine-tuning on dialog-game data transfers to a benchmark whose images never appear in the games: DOCCI/OpenImages games improve VQAv2 yes/no accuracy by 6.8 and 10.4 percentage points respectively.
  • The loop is self-reinforcing at least across two rounds: higher game success after round 1 yields larger, higher-quality filtered datasets, lifting robotics success detection from 56.5% through 69.5% to 73.0%.
  • The Guesser's question-generation role is the part that unlocks iteration; dropping it (answers-only fine-tuning) keeps success detection comparable but leaves game success near base level (17.92% vs 40.15% after round 1), stopping the cycle.
  • Game difficulty controls the quality-quantity trade-off in generated data: $N=4$ with similar-image distractors gave the best VQAv2 result in the authors' sweep, while $N=2$ produced more data and $N=8$ almost none.
  • The framework applies to scarce specialized domains with minimal task-specific supervision: only unlabelled video episodes from the target domain are needed to build the games.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My reading: in the general-VQA setting, the authors do not include a non-dialog control from the same source images, so the cleanest way to isolate the game structure's contribution is to run the robotics-style SFT-Description and Self-QA baselines on OpenImages and DOCCI.
  • My reading: the permutation-verification filter is a denoising procedure that could be useful beyond games; applying it to filter self-generated QA or caption data by consistency across image orderings is a natural extension.
  • My reading: a stronger reward signal than binary success — e.g., weighting dialogs by the minimal number of questions needed for disambiguation, or by whether the Describer's answers were decisive — might amplify the downstream gains, since the current filter treats all successful dialogs as equally good.
  • My reading: the method's gains scale with the base model's instruction-following and the diversity of the source images; the failure mode the authors name (agents finding trivial pixel-color or private-vocabulary strategies) would be detectable and presumably fixable by measuring how much downstream transfer drops when those dialogs dominate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces VLM Dialog Games, a self-improvement framework in which two prompted instances of a vision-language model play a reference game: a Describer answers questions about a target image while a Guesser asks questions to identify that image among distractors. Dialogs that end with the Guesser correctly selecting the target, verified across target-position permutations, are filtered into a synthetic interleaved image-text dataset used for supervised fine-tuning. The authors report that fine-tuning on games built from DOCCI or OpenImages improves accuracy on subsets of VQAv2 (yes/no and counting questions), and that games built from ALOHA robot frames improve success detection, with a second round of iterative fine-tuning yielding further gains in the robotics setting. The paper also presents ablations on the number of images per game and on distractor grouping, plus robotics baselines including plain description SFT, a simpler self-question-answering loop, and an answers-only variant of the dialog data.

Significance. If the reported results hold, the framework is a potentially useful recipe for generating domain-specific interleaved vision-language training data with minimal annotation, which is valuable in data-scarce settings such as robotics. The paper has clear strengths: the robotics section includes meaningful baselines (SFT-Description, Self-QA, answers-only), demonstrates iterative gains over two rounds, and evaluates on an external downstream task (ALOHA success detection); the permutation-consistency filter is a sensible and concrete guard against chance-level game successes; and the cross-dataset evaluation from DOCCI/OpenImages to VQAv2 provides some evidence of generalization. The main weakness is that the general VQA experiments lack a non-dialog fine-tuning control, so the paper does not yet establish that the dialog-game structure, rather than mere exposure to additional image-text pairs, drives the reported VQAv2 improvements. The absence of error bars and the use of only two VQAv2 question-type subsets further temper the strength of the claims.

major comments (3)
  1. [Section 4.2.3, Table 1] The general VQA experiments compare only against the base Gemini 1.5 Flash checkpoint. To support the claim that the dialog-game structure, rather than exposure to additional image-text data, drives the improvements, the authors should include a control fine-tuned on non-dialog image-description or image-label pairs from DOCCI and OpenImages, matched in data size and training settings. The robotics experiments already include such a control: SFT-Description in Table 4 improves success detection by 8.5 points, showing that plain exposure accounts for a large portion of the 16.5-point dialog-game gain. Without the analogous control for VQAv2, the 6.8-10.4 point gains in Table 1 cannot be attributed to the game mechanism.
  2. [Tables 1-4] All tables report only single accuracy values with no error bars, confidence intervals, or number of independent runs. Several comparisons that support specific conclusions are small: Self-QA vs VLM Dialog Games (round 1) differs by 2.5 points (67.0 vs 69.5) in Table 4, and similar vs random image grouping differs by 0.8 points (83.4 vs 82.6) in Table 3. The authors should report variance estimates or run multiple seeds before drawing conclusions from these differences.
  3. [Section 4.2.3 and Abstract] The VQAv2 evaluation uses only the yes/no and counting subsets, not the full VQAv2 benchmark, so the abstract's claim that the method 'generalises across datasets' should be qualified accordingly. The counting result for OpenImages shows zero improvement (56% vs 56%), while the DOCCI condition shows +2.3%; this dataset-dependent pattern should be discussed explicitly, especially since the paper uses the DOCCI counting gain to support a hypothesis about dataset suitability.
minor comments (6)
  1. [Section 4.2.3] Typo: 'Fist' should be 'First', and 'We hypothesis' should be 'We hypothesize'.
  2. [Section 5] The heading text 'introduced VML Dialog Games' should read 'VLM Dialog Games'.
  3. [Appendix 9] Typos: 'dailog' should be 'dialog', and 'evalaution' should be 'evaluation'.
  4. [Table 4] The entry '68% (+12.5)%' has a misplaced percent sign; it should read '68.0% (+12.5%)'.
  5. [Section 4.2.1 and 4.2.3] The paper does not report the number of training examples used for fine-tuning in the general VQA experiments; please state the dataset sizes explicitly, not just the number of games and game success rates.
  6. [Appendix 6.1] The prompt transcriptions contain inconsistent line breaks and formatting artifacts that make them harder to read than necessary; consider cleaning up the presentation while preserving the exact prompt text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: success filtering uses external ground truth and downstream evaluation uses external benchmarks.

full rationale

The paper's derivation chain is self-contained and not circular. Game success is defined by the Guesser selecting the known target image among distractors (Section 3.1), and this selection is verified against the external ground truth identity of the target; the additional permutation check in Section 3.2.3 also uses the known target as ground truth. The training data are filtered by this external criterion, not by downstream task performance. Fine-tuning is then evaluated on external benchmarks (VQAv2, and ALOHA success detection treated as zero-shot VQA), which are not used to select or filter training dialogs or to tune the method. The robotics section includes matched non-dialog baselines (SFT-Description, Self-QA, answers-only), so the claim that game structure contributes beyond simple data exposure is supported there. The general VQA section (Section 4.2.3) lacks such a control, but that is an experimental-design limitation, not a circularity: no reported quantity reduces by construction to an input fit. The self-referential aspect of self-improvement, where the same model generates its own future training data, is an intended property of the loop, and the filtering ground truth remains external to the model. The only self-citations (Du et al. 2023 for the success-detection evaluation protocol; Sharifzadeh et al. 2024 cited in related work) are contextual and not load-bearing for the central derivation.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method has no fitted parameters in the derivation sense, but several hand-chosen hyperparameters (N, dialog length, filtering rule) shape the generated dataset. The main unproved domain assumptions are that the base model can play the game and that success-filtered dialogs transfer to downstream tasks; the latter is only partially controlled.

free parameters (4)
  • Number of images per game N = 4 in main experiments; 2, 4, 8 tested in ablation
    Controls game difficulty and data quantity; Table 2 shows N=4 is best, but the central results use this hand-chosen value.
  • Maximum dialog length = 3 question-answer turns
    Limits game length; not ablated, and it constrains the depth of generated dialogs.
  • Permutation consistency filter = Correct guess across all N target positions
    Ad hoc data inclusion rule used in Section 3.2.3 to reduce chance successes; stricter than single-success filtering and affects dataset composition.
  • Nucleus sampling top-p = 0.8
    Sampling parameter for dialog generation (Appendix 9); standard but unablated.
assumptions (3)
  • domain assumption Gemini 1.5 Flash is instruction-following enough to play the dialog game at a non-trivial success rate.
    Required for dialog generation; empirically supported by 14 to 25 percent success rates in Tables 1 and 4, but the method would not work with a weaker base model.
  • domain assumption The experimenter knows which image is the target, enabling objective success determination.
    Used in Section 3.1 'Success Determination'; this is true by construction in the game setup, not an unverified assumption.
  • domain assumption Filtering successful dialogs yields training data that transfers to downstream tasks beyond mere exposure to domain images.
    This is the central premise of the method; it is tested via baselines in robotics (Section 4.4.3) but not in the general VQA experiments (Section 4.2.3), where a simple SFT baseline is missing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vision-Language Model Dialog Games for Self-Improvement." pith.science (2026). https://pith.science/paper/W5VGGY53

@misc{pith2026250202740,
  author       = {Pith},
  title        = {Pith review of: Vision-Language Model Dialog Games for Self-Improvement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W5VGGY53}},
  note         = {Machine review of arXiv:2502.02740}
}
read the original abstract

The increasing demand for high-quality, diverse training data poses a significant bottleneck in advancing vision-language models (VLMs). This paper presents VLM Dialog Games, a novel and scalable self-improvement framework for VLMs. Our approach leverages self-play between two agents engaged in a goal-oriented play centered around image identification. By filtering for successful game interactions, we automatically curate a high-quality dataset of interleaved images and text. We demonstrate that fine-tuning on this synthetic data leads to performance gains on downstream tasks and generalises across datasets. Moreover, as the improvements in the model lead to better game play, this procedure can be applied iteratively. This work paves the way for self-improving VLMs, with potential applications in various real-world scenarios especially when the high-quality multimodal data is scarce.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 19 canonical work pages

  1. [1]

    Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al. Constitutional AI : Harmlessness from AI feedback. arXiv:2212.08073, 2022

    arXiv 2022

  2. [2]

    Chalamalasetti, J

    K. Chalamalasetti, J. G \"o tze, S. Hakimov, B. Madureira, P. Sadler, and D. Schlangen. clembench: Using game play to evaluate chat-optimized language models as conversational agents. In EMNLP, 2023

    work page 2023

  3. [3]

    Z. Chen, Y. Deng, H. Yuan, K. Ji, and Q. Gu. Self-play fine-tuning converts weak language models to strong language models. In ICML, 2024

    work page 2024

  4. [4]

    A. Das, S. Kottur, K. Gupta, A. Singh, D. Yadav, J. M. Moura, D. Parikh, and D. Batra. Visual dialog. In CVPR, 2017

    work page 2017

  5. [5]

    De Vries, F

    H. De Vries, F. Strub, S. Chandar, O. Pietquin, H. Larochelle, and A. Courville. Guesswhat?! visual object discovery through multi-modal dialogue. In CVPR, 2017

    work page 2017

  6. [6]

    Y. Du, K. Konyushkova, M. Denil, A. Raju, J. Landon, F. Hill, N. de Freitas, and S. Cabi. Vision-language models as success detectors. In Proceedings of The 2nd Conference on Lifelong Learning Agents, 2023

    work page 2023

  7. [7]

    T. Gemini. Gemini 1.5: U nlocking multimodal understanding across millions of tokens of context. arXiv:2403.05530, 2024

    arXiv 2024

  8. [8]

    Goyal, T

    Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In CVPR, 2017

    work page 2017

Show all 26 references
  1. [9]

    Haber, T

    J. Haber, T. Baumg \"a rtner, E. Takmaz, L. Gelderloos, E. Bruni, and R. Fern \'a ndez. The P hoto B ook dataset: Building common ground through visually-grounded dialogue. In ACL, 2019

    2019

  2. [10]

    Hakimov, Y

    S. Hakimov, Y. Abdullayeva, K. Koshti, A. Schmidt, Y. Weiser, A. Beyer, and D. Schlangen. Using game play to investigate multimodal and conversational grounding in large multimodal models. arXiv:2406.14035, 2024

    2024 arXiv

  3. [11]

    Huang, S

    J. Huang, S. S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han. Large language models can self-improve. In EMNLP, 2023

    2023

  4. [12]

    C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 2021

    2021

  5. [13]

    Kaplan, S

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv:2001.08361, 2020

    2001 arXiv

  6. [14]

    Krauss and S

    R. Krauss and S. Weinheimer. Changes in reference phrases as a function of frequency of usage in social interactions. Psychonomic Science, 1964

    1964

  7. [15]

    Kuznetsova, H

    A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, A. Kolesnikov, T. Duerig, and V. Ferrari. The open images dataset v4: U nified image classification, object detection, and visual relationship detection at scale. Interna...

    2020

  8. [16]

    H. Li, J. Gu, R. Koner, S. Sharifzadeh, and V. Tresp. Do DALL-E and F lamingo understand each other? In ICCV, 2023 a

    2023

  9. [17]

    T. Li, S. Bhardwaj, Y. Tian, H. Zhang, J. Barber, D. Katabi, G. Lajoie, H. Chang, and D. Krishnan. Leveraging unpaired data for vision-language generative models via cycle consistency. arXiv 2310.03734, 2023 b

    2023 arXiv

  10. [18]

    Luu, V.-T

    D.-T. Luu, V.-T. Le, and D. M. Vo. Questioning, answering, and captioning for zero-shot detailed image caption. In Proceedings of the Asian Conference on Computer Vision, 2024

    2024

  11. [19]

    Y. Onoe, S. Rane, Z. Berger, Y. Bitton, J. Cho, R. Garg, A. Ku, Z. Parekh, J. Pont-Tuset, G. Tanzer, S. Wang, and J. Baldridge. DOCCI: Descriptions of Connected and Contrasting Images . In ECCV, 2024

    2024

  12. [20]

    Achiam, S

    OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv:2303.08774, 2023

    2023 arXiv

  13. [21]

    Sharifzadeh, C

    S. Sharifzadeh, C. Kaplanis, S. Pathak, D. Kumaran, A. Ilic, J. Mitrovic, C. Blundell, and A. Banino. Synth ^2 : Boosting visual-language models with synthetic captions and image embeddings. arXiv:2403.07750, 2024

    2024 arXiv

  14. [22]

    Subramaniam, Y

    V. Subramaniam, Y. Du, J. B. Tenenbaum, A. Torralba, S. Li, and I. Mordatch. Multiagent finetuning: Self improvement with diverse reasoning chains. arXiv:2501.05707, 2025

    2025 arXiv

  15. [23]

    J. Wang, K. Wang, Y. Yu, Y. Lu, W. Xiao, Z. Sun, F. Liu, Z. Zou, Y. Gao, L. Yang, et al. Self-improving generative foundation model for synthetic medical image generation and clinical applications. Nature Medicine, 2024

    2024

  16. [24]

    W. Yuan, R. Y. Pang, K. Cho, X. Li, S. Sukhbaatar, J. Xu, and J. E. Weston. Self-rewarding language models. In ICML, 2024

    2024

  17. [25]

    T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In RSS, 2023

    2023

  18. [26]

    J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In ICCV, 2017

    2017

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.