Pith. sign in

REVIEW 3 major objections 6 minor 58 references

VLMs often answer UML relation questions from class-name priors, not from the arrow they are shown.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 06:35 UTC pith:44LV4BQX

load-bearing objection Clean matched reverse-arrow UML benchmark shows open-source VLMs follow class-name priors over notation; design is solid and the residual risk is mainly prior-construction, not a broken claim. the 3 major comments →

arxiv 2607.02853 v1 pith:44LV4BQX submitted 2026-07-03 cs.CV cs.SE

Prior Bias in Vision Language Models on UML Diagram Interpretation

Zaiyu Cheng , Khai-Nguyen Nguyen , Antonio Mastropaolo This is my paper
classification cs.CV cs.SE
keywords vision-language modelsUML class diagramsprior biasknowledge conflictrelation directionsoftware engineering diagramsmodel scaling
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language models are being asked to read software diagrams whose meaning is fixed by visual notation alone. This paper asks whether those models actually use the arrow on a UML class diagram or instead fall back on what they already know about how the named classes usually relate. The authors build matched diagram pairs that keep the same class names and layout and reverse only the relation arrow, then measure how much relation-direction accuracy collapses under that single change. On eight open-source models the average drop is about one-third; three-class diagrams make the failure worse, and even frontier models that nearly close the simple two-class gap still lose accuracy once structure is added. Scaling helps only in a family-dependent way and does not remove the bias. The result matters because software-engineering pipelines that treat diagrams as ground truth can silently inherit pretrained design stereotypes instead of the diagram that is actually drawn.

Core claim

When a UML class diagram is rewritten so that only the relation arrow is reversed while class names and layout stay fixed, open-source vision-language models lose roughly 33 percent relation-direction accuracy on average; the gap is larger still on three-class variants and is only partially closed by frontier models. The models therefore often answer from the knowledge prior implied by the class names rather than from the visual notation.

What carries the argument

A matched prior-conforming / prior-conflicting UML benchmark: each diagram pair shares class names, layout, and query, differs only by arrow direction, and is supplemented by a prior-free control that replaces names with opaque labels so perception failures can be separated from prior-driven inference.

Load-bearing premise

The controlled class-name vocabulary and generation checks really produce unambiguous canonical priors for every ordered pair, so that the accuracy drop can be attributed to knowledge conflict rather than residual name ambiguity or diagram artifacts.

What would settle it

Find a model and evaluation setting in which relation-direction accuracy on the prior-conflicting diagrams stays within a few points of the prior-conforming diagrams across inheritance and aggregation, including the three-class conditions, while class-name recovery and prior-free arrow reading remain high.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper argues that current VLMs often answer UML class-diagram relation questions from pretrained class-name priors rather than from the depicted arrow direction. It introduces a controlled benchmark in which each prior-conforming diagram is paired with a prior-conflicting counterpart that keeps class names and layout fixed while reversing only the relation arrow, plus a prior-free control that replaces names with opaque labels. Using separate class-name exact-match (EM) and relation-direction accuracy (RelAcc) metrics, the authors evaluate InternVL3.5 and Qwen3 size variants plus GPT-5.4 / GPT-5.4 Mini. They report a large open-source conflict gap (mean Δ ≈ 33.48% on two-class reverse; larger on three-class variants), scale-invariance of the gap, stronger effects on Aggregation/Inheritance, family-dependent scaling, and a reduced but non-zero gap for frontier models on harder structural conditions.

Significance. If the diagnostic holds, the work is a clear and useful contribution to multimodal evaluation for software-engineering artifacts: it shows that aggregate UML VQA accuracy can mask a post-reading prior override, and it supplies a falsifiable matched-pair test rather than only naturalistic counterfactuals. Strengths include the paired prior-conform/prior-conflict design, the prior-free disentangling control, separate perception vs. inference metrics, multi-relation and multi-scale slices, the prerequisite-skill analysis that correctly qualifies InternVL scaling claims, the full per-cell appendix table, and a public artifact. These make the central claim actionable for both VLM robustness research and SE tooling that relies on diagram-grounded reasoning.

major comments (3)
  1. Section 3.1 (Generation/Validation) and the definition of Δ: the causal claim that Δ isolates knowledge-prior override assumes that each ordered class pair has an unambiguous, model-shared 'canonical' direction. The four structural validation constraints are necessary but not sufficient; the manuscript does not report an independent prior-strength check (e.g., text-only or image-free probing of the same pairs, or human prior ratings). Without that measurement, residual name ambiguity or generation artifacts remain a load-bearing alternative explanation for part of the gap, even though prior-free and scale controls already rule out pure unreadability and pure resolution bottlenecks.
  2. Section 4.1 / Fig. 5: the further RelAcc drop from 2-reverse to 3-reverse/3-mixed is interpreted as 'adding more visual context exacerbates prior-over-vision.' That interpretation is plausible but not fully isolated. The paper needs a clearer three-class prior-free (or prior-conform three-class) control reported on the same footing, or an analysis that holds visual complexity fixed while varying only prior alignment of the auxiliary edge. Otherwise complexity, attention dilution, or multi-edge parsing difficulty can co-explain the ~12-point additional drop and the open-source 45.28% three-class figure.
  3. Section 3.2 scoring protocol: counting Unknown as incorrect is a defensible conservative choice, but it interacts with the main claim. If models systematically abstain more under conflict than under conform (or under three-class than two-class), part of Δ could be refusal/uncertainty rather than confident prior substitution. Please report Unknown rates by condition (at least for the family means) so readers can separate forced prior answers from abstention-driven accuracy loss.
minor comments (6)
  1. Abstract and §1: spacing/punctuation glitches around percentages (e.g., '33.48%on', '10%gap', '45.28%for') should be cleaned for camera-ready readability.
  2. Fig. 1 caption and body: the Mammal/Elephant example is clear, but the figure panel labels (a/b/c) and the 'Yes! ❌' annotation are a bit dense; a single sentence stating the gold label under each panel would help skimmers.
  3. Table 1 / Appendix D: prior-free is much smaller after abbreviation deduplication; state explicitly in the main text that prior-free is a diagnostic control, not size-matched to the semantic conditions, so readers do not over-interpret absolute prior-free RelAcc as comparable sample power.
  4. §4.3 / Fig. 9: the EM≥95% and RelAcc_prior-free≥50% thresholds are reasonable analysis filters; briefly justify the 50% baseline (always-True / chance) in the main text rather than only in the figure discussion.
  5. Related Work §2.1: the UML extraction literature is well covered; a short note on how the present conflict design differs from ordinary UML VQA datasets (Shehzadi et al., Naboichenko & Peinl) would sharpen novelty for SE readers.
  6. Prompts in Appendix F are a strength; consider releasing the exact PlantUML templates and the controlled vocabulary list in the artifact README so the 'unambiguous prior' assumption can be audited externally.

Circularity Check

0 steps flagged

No significant circularity: empirical matched-pair evaluation; Δ is a plain accuracy difference, not a fitted or self-defined prediction.

full rationale

This is an experimental diagnostic paper, not a first-principles derivation. The central quantity is defined as Δ = RelAcc_prior-conform − RelAcc_prior-conflict on matched diagrams that differ only by arrow reversal (Section 3.2); the reported drops (open-source mean 33.48 % two-class, larger on three-class) are direct measurements on held-out model outputs, not quantities forced by construction from fitted constants. Prior-free, image-scale, and relation-type slices serve as controls, not circular redefinitions. Prerequisite filters (EM ≥ 95 %, RelAcc_prior-free ≥ 50 %) are post-hoc analysis subsets that do not redefine the headline claim. The sole minor self-citation (Vo et al. 2025, overlapping author) appears only in related-work framing of natural-image prior bias and is not load-bearing for the UML numbers or uniqueness claims. No self-definitional loop, fitted-input-as-prediction, uniqueness theorem imported from the authors, ansatz smuggled via citation, or renaming of a known result is present. Residual risk about vocabulary unambiguity is a validity concern, not circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 2 invented entities

The central claim rests on empirical measurement under standard UML notation and an experimental construction that treats class-name semantics as inducing priors while arrow direction is ground truth. No physical constants or fitted scientific laws are involved; the main modeling choices are benchmark-construction and scoring conventions.

free parameters (3)
  • prerequisite EM threshold = 95%
    EM ≥ 95% is chosen by hand to define the prerequisite-passing subset used to interpret scaling (Section 4.3).
  • prerequisite RelAcc_prior-free threshold = 50%
    RelAcc ≥ 50% is chosen as above-always-True baseline for arrow reading without priors (Section 4.3).
  • image scale factors = 1x/1.5x/2x
    Evaluations use 1×, 1.5×, 2× resizes of the same PlantUML renders; scales are experimental design choices, not fits, but they define the robustness slice.
axioms (5)
  • domain assumption Standard UML arrow/diamond notation uniquely determines relation direction for inheritance, aggregation, composition, and dependency.
    Used throughout Section 3 as the visual ground truth for RelAcc labels.
  • domain assumption Selected class-name pairs induce a clear canonical prior direction that can be intentionally violated by arrow reversal.
    Core of prior-conform vs prior-conflict construction in Section 3.1; validated via controlled vocabulary and LLM checks.
  • ad hoc to paper Exact-match class-name recovery after normalization is a valid proxy for whether the model can read diagram labels.
    Defines EM metric in Section 3.2; reasonable but not the only possible perception measure.
  • ad hoc to paper Abstentions (Unknown) count as incorrect for scoring.
    Stated in Section 3.1/3.2; conservative scoring choice that can lower absolute RelAcc.
  • ad hoc to paper Averaging across relations, scales, and family members yields a meaningful conflict-gap summary.
    Used for headline 33.48% and related means; paper also reports disaggregations.
invented entities (2)
  • UMLKnowledgeConflict benchmark (prior-conform / 2-reverse / 3-reverse / 3-mixed / prior-free) independent evidence
    purpose: Isolate prior-over-vision failure on formal UML class diagrams via matched reverse-arrow pairs and controls.
    New evaluation construct introduced by the paper; independent evidence is the released artifact and reported measurements, not an external physical entity.
  • conflict gap Δ = RelAcc_prior-conform − RelAcc_prior-conflict independent evidence
    purpose: Quantify how much accuracy falls when visual evidence contradicts class-name priors.
    Defined metric, not a postulated physical object; useful summary statistic for the experiments.

pith-pipeline@v1.1.0-grok45 · 25402 in / 3184 out tokens · 33115 ms · 2026-07-12T06:35:16.670794+00:00 · methodology

0 comments
read the original abstract

Vision Language Models (VLMs) are increasingly applied to software engineering artifacts, especially UML class diagrams whose meaning depends on visual notation. Yet, it is unclear whether VLMs actually read such diagrams or instead answer from pretrained priors about how classes typically relate. We introduce a controlled UML benchmark in which each prior-conforming diagram is paired with its prior-conflicting counterpart that (1) preserves the same class names and layout while (2) reverses only the relation arrow. We evaluate eight open-source VLMs from two model families, InternVL3.5 and Qwen3, alongside two closed-source frontier models GPT-5.4 and GPT-5.4 Mini. Across the eight open-source models, reversing the arrow reduces relation-direction accuracy by 33.48% on average, while GPT-5.4 Mini retains a 10% gap. In the harder three-class condition, accuracy drops sharply by 45.28% for open-source models, and even 18.62% for the GPT-5.4 family on average. Scaling provides only limited improvements and is family-dependent. Our benchmark presents a diagnostic prior-driven failure in diagram-grounded software understanding. Our artifact is available at https://anonymous.4open.science/r/UMLKnowledgeConflict-8461.

Figures

Figures reproduced from arXiv: 2607.02853 by Antonio Mastropaolo, Khai-Nguyen Nguyen, Zaiyu Cheng.

Figure 1
Figure 1. Figure 1: VLMs follow the class names, not the arrow. (a) On a prior-conforming diagram, VLMs correctly answer “Is Mammal the parent class of Elephant?” with “Yes.” (b) Flipping the arrow makes “No” the correct answer. (c) VLMs still answer “Yes”, following the prior knowledge rather than the depicted arrow. To study this question, we narrow our focus specifically to UML diagrams, which expose this fragility cleanly… view at source ↗
Figure 2
Figure 2. Figure 2: Data generation pipeline. (1) We prompt SOTA LLMs to generate candidate UML diagrams across the three conditions; (2) We then validate the diagrams against four constraints, ensuring consistency and correctness of these images; (3) If there is any error ($), we generate a structured error description and redo step (1), else (") we accept the candidate. image evidence (Goyal et al., 2017; Ramakrishnan et al… view at source ↗
Figure 3
Figure 3. Figure 3: Examples for each condition type. We report only the 2-reverse subset for the prior-conflict condition. Additional illustrations, including other prior-conflict subsets, are shown in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Reversing the direction of a prior-conform diagram lowers accuracy by 27.75% on the all-model mean, but the drop is concentrated in the open￾source families (Qwen3 ∆ = 26.44%, InternVL3.5 ∆ = 40.52%) while the GPT-5.4 family makes the two-class gap smaller (∆ = 4.82%). Per-family RelAcc (%) on prior-conform (blue) and prior-conflict (orange), with the conflict gap ∆ (red); each family bar is the simple mea… view at source ↗
Figure 5
Figure 5. Figure 5: Adding a prior-conflicting (3-reverse) or prior-conforming (3-mixed) auxiliary edge to 2- reverse reduces relation-direction accuracy in every family; the GPT-5.4 family stays much higher overall but still loses ∼14 points from 2-reverse to 3-reverse. Per-family and overall mean RelAcc (%) on 2-reverse (yellow), 3-reverse (orange), and 3-mixed (red), aver￾aged over 4 relations and 3 image scales; each fami… view at source ↗
Figure 6
Figure 6. Figure 6: The conflict gap is large and positive for all four UML relations in the open-source 8-model mean (top), with Aggregation and Inheritance largest; the GPT-5.4 family (bottom) keeps the gap below 7% on every relation. Per-relation and overall mean RelAcc (%) on prior-conform (blue) and prior-conflict (orange), with the conflict gap ∆ (red); “Mean” on each panel is the simple mean over the four relations. ti… view at source ↗
Figure 8
Figure 8. Figure 8: Within-family scaling looks positive for Qwen3 on 2-reverse but inverse for InternVL3.5 (left); on 3-reverse most larger open-source variants collapse below the 2B. The GPT-5.4 family (right) sits well above the open-source variants on both con￾ditions, and scales positively from GPT-5.4 Mini to GPT-5.4 on 3-reverse (65.41 → 86.05). Per-variant RelAcc (%) on 2-reverse (solid) and 3-reverse (dashed) vs. wit… view at source ↗
Figure 10
Figure 10. Figure 10: expands the condition illustration in [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: System prompt. General instruction shared by all evaluation prompts, emphasizing image-grounded reasoning and strict output formatting. Stage-1 class-name recovery prompt The image provided is a UML diagram showing <Relation> relationships. The UML diagram contains <Number> classes. Each class is represented as a box, with the class name at the top. The green circled letter ’C’ inside each box is only a c… view at source ↗
Figure 12
Figure 12. Figure 12: Class-name recovery prompt. Prompt used to recover the class names explicitly visible in the UML diagram before relation-direction evaluation. Stage-2 relation-direction QA prompt The image provided is a UML diagram showing <Relation> relationships. The diagram contains <Two/Three> classes: <Class names>. Each class is represented as a box, with the class name at the top. The green circled letter ’C’ insi… view at source ↗
Figure 13
Figure 13. Figure 13: Relation-direction QA prompt. Prompt used to answer the relation-direction question after the class names have been specified. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Relation-specific UML rules and query templates. UML notation rules and question templates used to instantiate the relation-direction QA prompt [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

58 extracted references · 1 canonical work pages

  1. [1]

    Communications of the ACM , volume=

    How UML is used , author=. Communications of the ACM , volume=. 2006 , publisher=

    2006

  2. [2]

    Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement , pages=

    A survey into the rigor of UML use and its perceived impact on quality and productivity , author=. Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement , pages=

  3. [3]

    , author=

    What are the used UML diagrams? A Preliminary Survey. , author=. EESSMod@ MoDELS , pages=

  4. [4]

    Journal of Systems Architecture , volume=

    A survey on modeling and model-driven engineering practices in the embedded software industry , author=. Journal of Systems Architecture , volume=. 2018 , publisher=

    2018

  5. [5]

    Information and Software Technology , volume=

    A survey on the practical use of UML for different software architecture viewpoints , author=. Information and Software Technology , volume=. 2020 , publisher=

    2020

  6. [6]

    Proceedings , volume=

    UML diagrams in software engineering research: a systematic literature review , author=. Proceedings , volume=. 2021 , organization=

    2021

  7. [7]

    Journal of Object Technology , volume=

    On the use of GPT-4 in the reverse engineering of class diagrams , author=. Journal of Object Technology , volume=

  8. [8]

    Software , volume=

    Automatic Complexity Analysis of UML Class Diagrams Using Visual Question Answering (VQA) Techniques , author=. Software , volume=. 2025 , publisher=

    2025

  9. [9]

    Assessing UML diagrams by GPT: Implications for education , journal =

    Chong Wang and Beian Wang and Peng Liang and Jie Liang , keywords =. Assessing UML diagrams by GPT: Implications for education , journal =. 2026 , issn =. doi:https://doi.org/10.1016/j.jss.2025.112709 , url =

    work page doi:10.1016/j.jss.2025.112709 2026

  10. [10]

    , author=

    An Extensive Analysis of Data Clumps in UML Class Diagrams. , author=. ENASE , pages=

  11. [11]

    ACM SIGGRAPH 2006 Courses , pages=

    Tahuti: A geometrical sketch recognition system for uml class diagrams , author=. ACM SIGGRAPH 2006 Courses , pages=

    2006

  12. [12]

    Proceedings of the 2000 conference of the Centre for Advanced Studies on Collaborative research , pages=

    An interactive system for recognizing hand drawn UML diagrams , author=. Proceedings of the 2000 conference of the Centre for Advanced Studies on Collaborative research , pages=

    2000

  13. [13]

    Journal of Systems and Software , volume=

    Automatically recognizing the semantic elements from UML class diagram images , author=. Journal of Systems and Software , volume=. 2022 , publisher=

    2022

  14. [14]

    Classification of role stereotypes for classes in UML class diagrams using machine learning , author=

  15. [15]

    Journal of Systems and Software , volume=

    Evaluating the layout quality of UML class diagrams using machine learning , author=. Journal of Systems and Software , volume=. 2022 , publisher=

    2022

  16. [16]

    Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

    Making the v in vqa matter: Elevating the role of image understanding in visual question answering , author=. Proceedings of the IEEE conference on computer vision and pattern recognition , pages=

  17. [17]

    , author=

    Online Img2UML Repository: An Online Repository for UML Models. , author=. EESSMod@ MoDELS , pages=

  18. [18]

    2013 39th Euromicro Conference on Software Engineering and Advanced Applications , pages=

    Img2uml: A system for extracting uml models from images , author=. 2013 39th Euromicro Conference on Software Engineering and Advanced Applications , pages=. 2013 , organization=

    2013

  19. [19]

    arXiv preprint arXiv:2404.11376 , year=

    From image to UML: First results of image based UML diagram generation using LLMs , author=. arXiv preprint arXiv:2404.11376 , year=

    Pith/arXiv arXiv

  20. [20]

    Advances in neural information processing systems , volume=

    Overcoming language priors in visual question answering with adversarial regularization , author=. Advances in neural information processing systems , volume=

  21. [21]

    Proceedings of the 29th International Conference on Computational Linguistics , pages=

    Overcoming language priors in visual question answering via distinguishing superficially similar instances , author=. Proceedings of the 29th International Conference on Computational Linguistics , pages=

  22. [22]

    European conference on computer vision , pages=

    A diagram is worth a dozen images , author=. European conference on computer vision , pages=. 2016 , organization=

    2016

  23. [23]

    Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

    Docvqa: A dataset for vqa on document images , author=. Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages=

  24. [24]

    arXiv preprint arXiv:1710.07300 , year=

    Figureqa: An annotated figure dataset for visual reasoning , author=. arXiv preprint arXiv:1710.07300 , year=

    Pith/arXiv arXiv

  25. [25]

    Findings of the association for computational linguistics: ACL 2022 , pages=

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning , author=. Findings of the association for computational linguistics: ACL 2022 , pages=

    2022

  26. [26]

    Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    Evaluating object hallucination in large vision-language models , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    2023

  27. [27]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  28. [28]

    arXiv preprint arXiv:2412.20622 , year=

    Towards a systematic evaluation of hallucinations in large-vision language models , author=. arXiv preprint arXiv:2412.20622 , year=

    Pith/arXiv arXiv

  29. [29]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Knowledge conflicts for llms: A survey , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  30. [30]

    arXiv preprint arXiv:2310.00935 , year=

    Resolving knowledge conflicts in large language models , author=. arXiv preprint arXiv:2310.00935 , year=

    Pith/arXiv arXiv

  31. [31]

    Advances in Neural Information Processing Systems , volume=

    Wikicontradict: A benchmark for evaluating llms on real-world knowledge conflicts from wikipedia , author=. Advances in Neural Information Processing Systems , volume=

  32. [32]

    ACM Transactions on Software Engineering and Methodology , volume=

    Large language models for software engineering: A systematic literature review , author=. ACM Transactions on Software Engineering and Methodology , volume=. 2024 , publisher=

    2024

  33. [33]

    Proceedings of the 1st International Workshop on Large Language Models for Code , pages=

    Toward a new era of rapid development: Assessing GPT-4-Vision's capabilities in UML-based code generation , author=. Proceedings of the 1st International Workshop on Large Language Models for Code , pages=

  34. [34]

    arXiv preprint arXiv:2410.00193 , year=

    Do vision-language models really understand visual language? , author=. arXiv preprint arXiv:2410.00193 , year=

    Pith/arXiv arXiv

  35. [35]

    arXiv preprint arXiv:2501.00569 , year=

    Probing visual language priors in vlms , author=. arXiv preprint arXiv:2501.00569 , year=

    Pith/arXiv arXiv

  36. [36]

    Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement , pages=

    Evaluating large language models in exercises of uml class diagram modeling , author=. Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement , pages=

  37. [37]

    Intelligent Systems Conference , pages=

    Multi-modal summarization in model-based engineering: Automotive software development case study , author=. Intelligent Systems Conference , pages=. 2025 , organization=

    2025

  38. [38]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Benchmarking and improving large vision-language models for fundamental visual graph understanding and reasoning , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  39. [39]

    Advances in Neural Information Processing Systems , volume=

    Image2struct: Benchmarking structure extraction for vision-language models , author=. Advances in Neural Information Processing Systems , volume=

  40. [40]

    2016 , publisher=

    Modeling with UML , author=. 2016 , publisher=

    2016

  41. [41]

    arXiv preprint arXiv:2505.23941 , year=

    Vision language models are biased , author=. arXiv preprint arXiv:2505.23941 , year=

    Pith/arXiv arXiv

  42. [42]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Phd: A chatgpt-prompted visual hallucination evaluation dataset , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  43. [43]

    arXiv preprint arXiv:2410.08145 , year=

    Insight over sight: Exploring the vision-knowledge conflicts in multimodal llms , author=. arXiv preprint arXiv:2410.08145 , year=

    Pith/arXiv arXiv

  44. [44]

    arXiv preprint arXiv:2402.00253 , year=

    A survey on hallucination in large vision-language models , author=. arXiv preprint arXiv:2402.00253 , year=

    Pith/arXiv arXiv

  45. [45]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Vlind-bench: Measuring language priors in large vision-language models , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  46. [46]

    Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

    Breaking common sense: Whoops! a vision-and-language benchmark of synthetic and compositional images , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages=

  47. [47]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Visual hallucinations of multi-modal large language models , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    2024

  48. [48]

    5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=

    Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency , author=. arXiv preprint arXiv:2508.18265 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2511.21631 , year=

    Qwen3-vl technical report , author=. arXiv preprint arXiv:2511.21631 , year=

    Pith/arXiv arXiv

  50. [50]

    arXiv preprint arXiv:2306.01879 , year=

    Revisiting the role of language priors in vision-language models , author=. arXiv preprint arXiv:2306.01879 , year=

    Pith/arXiv arXiv

  51. [51]

    Advances in Neural Information Processing Systems , volume=

    Naturalbench: Evaluating vision-language models on natural adversarial samples , author=. Advances in Neural Information Processing Systems , volume=

  52. [52]

    European Conference on Computer Vision , pages=

    Blink: Multimodal large language models can see but not perceive , author=. European Conference on Computer Vision , pages=. 2024 , organization=

    2024

  53. [53]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Vgbench: Evaluating large language models on vector graphics understanding and generation , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  54. [54]

    Proceedings of the 41st International Conference on Machine Learning , articleno =

    Li, Yunxin and Hu, Baotian and Shi, Haoyuan and Wang, Wei and Wang, Longyue and Zhang, Min , title =. Proceedings of the 41st International Conference on Machine Learning , articleno =. 2024 , publisher =

    2024

  55. [55]

    and Bar, Amir and Singh, Ritambhara and Eickhoff, Carsten

    Golovanevsky, Michal and Rudman, William and Lepori, Michael A. and Bar, Amir and Singh, Ritambhara and Eickhoff, Carsten. Pixels Versus Priors: Controlling Knowledge Priors in Vision-Language Models through Visual Counterfacts. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.1262

    doi:10.18653/v1/2025.emnlp-main.1262 2025

  56. [56]

    arXiv preprint arXiv:2502.12359 , year=

    Lanp: Rethinking the impact of language priors in large vision-language models , author=. arXiv preprint arXiv:2502.12359 , year=

    Pith/arXiv arXiv

  57. [57]

    arXiv preprint arXiv:2605.11634 , year=

    Unlocking UML Class Diagram Understanding in Vision Language Models , author=. arXiv preprint arXiv:2605.11634 , year=

    Pith/arXiv arXiv

  58. [58]

    Roques, Arnaud and PlantUML Contributors , title =