Overcoming Vision Language Model Challenges in Diagram Understanding: A Proof-of-Concept with XML-Driven Large Language Models Solutions

Hayato Motegi; Masumi Morishige; Ryo Koshihara; Shue Shiinoki

REVIEW 3 major objections 6 minor 32 references

Overcoming Vision Language Model Challenges in Diagram Understanding: A Proof-of-Concept with XML-Driven Large Language Models Solutions

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that diagram understanding can bypass visual recognition entirely by parsing the diagram's source XML into structured text for an LLM.

desk verdict A modest, honest proof-of-concept: XML extraction from xlsx beats GPT-4o on one diagram, but the abstract overclaims and the connector-inference heuristic is the fragile part. read the letter →

arxiv 2502.04389 v1 pith:7ETOV2TX submitted 2025-02-05 cs.SE cs.AI

Shue Shiinoki , Ryo Koshihara , Hayato Motegi , Masumi Morishige This is my paper · ORCID

classification cs.SEcs.AI

keywords diagramunderstandingvision-languagemodelslargelanguageXMLparsingxlsxsourcefilesconnectorrelationextractionOfficeOpenproofofconcept

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that diagrams stored in editable Office source files can be understood by large language models without any image processing. The authors parse the XML inside an .xlsx file, convert shapes and connectors into structured JSON, and feed that text to an LLM. In a proof-of-concept on an Excel-drawn system architecture diagram, the text-driven method recovered every connector correctly, while an image-input vision-language model misidentified a curved connector, invented two nonexistent connections, and missed a real one. The claim is that reliable diagram question answering does not require visual recognition when the diagram's source file is available, and that the same principle should carry over to .pptx and .docx.

What carries the argument

The load-bearing mechanism is a parser for Office Open XML that turns a diagram's drawing elements into a compact JSON schema of shapes and connectors. For each connector, the parser records start and end coordinates plus a derived cardinal direction for each endpoint, deliberately ignoring the file's explicit connector-to-shape attachment records because diagram creators may leave connectors and shapes logically disconnected. For each shape, it records type, fill and border color, edge coordinates, and contained text, after converting rotation, flip, scale, and theme colors so the numbers match the rendered appearance. This JSON, accompanied by attribute descriptions in the prompt, is what lets an LLM reconstruct which shapes a connector links without ever seeing pixels.

What would settle it

Run the same text-driven prompt on xlsx diagrams that use grouped shapes or SmartArt and compare connector accuracy with the image-input baseline; if accuracy drops to or below the VLM's, the coordinate-based direction decoding is the reason.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the bottleneck in diagram understanding is visual recognition, and that bottleneck can be bypassed by replacing the rendered image with the diagram's source-level representation. Using a system design diagram drawn in Excel, the authors show that a vision-language model given the rendered image lists the components correctly but hallucinates when asked which components are connected: it misreads a curved connector as pointing to a nearby but unrelated component, omits the connector to the URL text box, and invents connections to Key Vault and Microsoft Entra that do not exist. The same model, given the JSON parsed from the XML, identifies all seven connectors with their correct endpoints and annotations. The paper therefore claims that XML-driven text input yields more accurate answers for questions requiring detailed structural comprehension of diagrams.

Load-bearing premise

The method assumes it can guess which shapes a connector touches from coordinates and four direction labels, so diagrams with grouped shapes, SmartArt, or connectors floating near edges may lose the relationships it is supposed to recover.

Editorial extensions

If this is right

An LLM equipped with parsed source files can answer relationship questions about a diagram without a vision-language model, so diagram QA no longer has to pass through image recognition.
The same XML-parsing strategy transfers to other Office formats whose source is XML, such as .pptx and .docx, so the approach is not specific to Excel.
The known failure modes of vision-language models on curved connectors, overlapping elements, and fine-grained line relations can be sidestepped by supplying coordinates and endpoint directions as text.
Because the method needs only shape and connector attributes, it can work with commodity text-only LLMs rather than requiring multimodal models.
Document loaders that extract only tables and paragraphs miss diagram information; parsing drawing elements as first-class text unlocks that information.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the approach generalizes, diagram understanding could be treated as a data-extraction problem rather than a perception problem, which would make accuracy easier to audit because each claimed connection can be traced to a JSON record.
A testable extension would be to add explicit connector-to-shape attachment metadata when the source file stores it, and compare against the coordinate-decoding heuristic on grouped shapes and SmartArt.
The same XML-to-JSON pipeline could feed retrieval-augmented generation over a corpus of design documents, letting a search index store diagram elements alongside tables and paragraphs.
Hybrid designs could delegate icons and free-form artwork to a vision-language model while trusting XML for connectors and shapes, limiting visual hallucination to elements that genuinely need image understanding.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a modest, honest proof-of-concept that extracting xlsx DrawingML shape and connector data into JSON lets an LLM answer relation questions about one system-design diagram where GPT-4o's image input hallucinated. That is a real result, but the comparative claim in the abstract is broader than the evidence, and the pipeline's topology-reconstruction choice may not survive contact with messier real diagrams.

What's new: the specific pipeline — parse the xlsx zip/XML, normalize coordinates and colors, emit JSON with shapes and connectors, feed to GPT-4o text — is, as far as I can tell, not in the prior work they cite. The closest analogues are Mermaid/HTML/docx representations, but xlsx connector topology is a different beast, and they show a concrete failure mode of GPT-4o (curved connector near a corner) where the text version succeeds. The code is open-sourced, which makes the feasibility claim checkable. The related-work citations are appropriate and not padded.

Soft spots, in order. First, the evidence is one diagram, one VLM, one prompt set. The abstract says 'experimental comparisons demonstrated more accurate answers', but there is no accuracy metric, no repeated trials, no second diagram. The limitations section concedes this, so the paper is honest, but the abstract oversells. Second, Section 3.2 explicitly declines to use the saved stCxn/endCxn connection info and reconstructs topology from connector endpoint directions and coordinates. The stress-test is right that this is the fragile part: for floating endpoints, grouped shapes, SmartArt, or overlapping shapes, the nearest-shape inference can be wrong, and Appendix A.1 admits those object types aren't fully supported. It's a real limitation, but it is disclosed, and for the one diagram it works. I wouldn't call it load-bearing to the point of invalidating the paper; it does limit generality. Third, minor: the placeholder 'Fig. XX' in Section 3.2 suggests sloppy finalization.

Who is this for? Someone building business-document QA pipelines over Office files, especially Excel-centric workflows. It won't redirect research, but it is a useful data point that source-file metadata can beat VLM pixels on connector-relation questions. It deserves peer review as a proof-of-concept; a serious referee should push for a small benchmark (a dozen diagrams, a few formats, some with grouped shapes) and a quantitative comparison before publication.

Recommendation: send it out, but expect major revision. The authors should either narrow the abstract or add that benchmark.

Referee Report

3 major / 6 minor

Summary. The paper proposes an XML-driven diagram understanding approach that bypasses vision-language models (VLMs) by extracting shape, connector, and text information from editable Office source files (in this proof-of-concept, .xlsx) and feeding a structured JSON representation to an LLM. The approach is evaluated on a single system-design diagram created in Microsoft Excel, with qualitative comparisons against GPT-4o image input. The authors report that the XML-driven method correctly enumerated components and connectors without hallucination, whereas the VLM misidentified a bent connector's target, missed a connector to a URL text box, and invented connectors to unrelated components. The paper concludes that the text-driven framework yields more accurate answers for questions requiring detailed comprehension of diagram structures, and it open-sources the parsing library and experimental code.

Significance. If the central claim is correct, the approach offers a practical alternative for diagram understanding in business documents, where source-file metadata is available and where VLM visual recognition errors are a known bottleneck. The paper's strengths include a reproducible open-source pipeline, a clearly described extraction and transformation process, and a direct comparison against a strong VLM baseline on the same diagram. The idea of using native XML topology instead of image rendering is well motivated by prior work on flowchart-to-Mermaid bottlenecks. However, the evidence is limited to one self-selected diagram, no quantitative metrics are reported, and the parser deliberately discards the exact connector-to-shape connection metadata that the OOXML format provides. The stated advantage over VLMs is therefore demonstrated only for a convenient layout, and the generality of the claim is not established.

major comments (3)

[§4.3 and Abstract] The core comparative claim that the proposed method 'yielded more accurate answers' is supported by a single hand-picked diagram and qualitative output in Figs. 4 and 5. There are no accuracy or F1 metrics, no repeated trials, no error bars, and no diversity of diagrams or question types. Appendix A.1 explicitly concedes 'insufficient quantitative evaluation' and that the study is 'case-based examples.' As written, the abstract and conclusion overstate the evidence: a proof-of-concept can report an existence proof, but it cannot claim general superiority over VLM-based methods without a systematic evaluation.
[§3.2] The parser chooses not to use the explicit connector-to-shape connection information that OOXML/DrawingML can store (e.g., stCxn and endCxn elements referencing shape IDs), and instead reconstructs connections by decoding each connector endpoint into one of four cardinal directions from coordinate data. This reconstruction is exact only when connectors terminate near shape boundaries and shapes are ungrouped rectangles or text boxes. Grouped shapes, SmartArt, WordArt, floating endpoints, and overlapping bounding boxes make nearest-shape inference ambiguous. Appendix A.1 itself admits that 'Grouped or composite shapes, SmartArt, and specialized WordArt objects may not be fully supported.' Since the paper's stated advantage is specifically about 'detailed comprehension of diagram structures,' this design choice is load-bearing: on less convenient layouts, the connection-inference heuristic can fail, and the claimed advantage over VLMs may collapse. The authors should either use the available connection metadata, or evaluate the heuristic on a set of diagrams that exercises these failure modes.
[§5] The conclusion states that the approach 'proved successful in enabling LLMs to understand the diagram without being affected by the detection errors inherent in visual recognition.' This is too strong given that only one diagram was tested and the comparison is qualitative. The manuscript's own limitation statement in A.1 contradicts this degree of certainty. I recommend softening the claim to a proof-of-concept result and explicitly stating the scope of the evidence.

minor comments (6)

[§3.2] The text contains a placeholder reference 'Fig. XX' where the JSON format is described; this should be replaced with the actual figure number.
[§2.1] There are typos in the text: 'halucinate' and 'halcination' should be 'hallucinate' and 'hallucination'.
[§4.3] The section heading 'Comparision with VLM' should be 'Comparison with VLM'.
[Figures 4 and 5] The figure text is very small and the shape/connector IDs are overlaid on a dense diagram, making the outputs difficult to read. Consider enlarging the annotated diagram or presenting the outputs in a tabular form.
[References] Several references contain formatting errors, such as 'Information F orensics' with a stray space and 'Lingustics' for 'Linguistics'; these should be corrected.
[Appendix A.3] The prompts for the entity-understanding and relation-understanding stages are shown in Figure 6, but it is unclear whether the system message is the same for both stages and whether the human messages in the figure include both task instructions. Clarify the exact prompt composition used in each of the two stages.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the XML-to-text pipeline is mechanical, the baseline is external, and no fitted parameter or self-cited theorem carries the argument.

full rationale

The paper's central derivation is the XML extraction and JSON transformation pipeline (Sections 3.1-3.2). This is a deterministic parsing and formatting step, not a fitted model, and no equation or parameter is tuned to the evaluation data. The comparative claim in Section 4.3 is tested against GPT-4o image input, an external baseline, with prompts kept as similar as possible; the observed differences are therefore not forced by construction. The method's input (JSON coordinates, directions, colors, and text) does not contain the final structured answer as a simple lookup table that makes the evaluation definitionally circular: the LLM must still map endpoint coordinates and cardinal directions to shape IDs and distinguish real connectors from near-miss alternatives, and Section 4.2 reports a case where the text-driven model did so correctly while the image baseline hallucinated. The explicit limitations in Appendix A.1 (single diagram, case-based evaluation, unsupported SmartArt/grouped shapes) concern external validity and generalizability, not circularity. The GitHub links are code releases, not load-bearing self-citations. Overall, the proof-of-concept is self-contained, and its reported success is not equivalent to its inputs by definition.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The method introduces no free numerical parameters and no new physical or conceptual entities. It relies on domain assumptions about XML fidelity, coordinate transformation, and LLM reasoning from coordinates, and on the representativeness of a single comparison diagram.

assumptions (4)

domain assumption The xlsx source file's XML preserves all diagram-relevant shapes, connectors, colors, and text in parseable form.
Section 3.1 states that Office source files are zip-compressed XML and that user-placed objects with editable attributes can be retrieved; the entire method depends on this.
domain assumption After applying transformations for rotation, flip, and scaling, the JSON coordinates accurately represent the rendered diagram.
Section 3.2 says raw XML coordinates do not reflect transforms and a conversion process was applied, but no verification against rendered output is reported.
domain assumption LLMs can infer shape-to-shape connectivity from connector endpoint coordinates and four cardinal directions, without explicit connection metadata or grouping information.
Section 3.2 explicitly chooses to decode endpoint directions instead of relying on connector-to-shape connection info, and ignores groups. Section 4.2 shows this worked for one diagram, but complex or overlapping diagrams could break the inference.
domain assumption One Excel system design diagram is sufficient to compare the proposed method against VLM performance.
Section 4.3 performs a single qualitative comparison, and Appendix A.1 concedes the evaluation is not statistically rigorous and lacks generalizability.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Overcoming Vision Language Model Challenges in Diagram Understanding: A Proof-of-Concept with XML-Driven Large Language Models Solutions." pith.science (2026). https://pith.science/paper/7ETOV2TX

@misc{pith2026250204389,
  author       = {Pith},
  title        = {Pith review of: Overcoming Vision Language Model Challenges in Diagram Understanding: A Proof-of-Concept with XML-Driven Large Language Models Solutions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7ETOV2TX}},
  note         = {Machine review of arXiv:2502.04389}
}

read the original abstract

Diagrams play a crucial role in visually conveying complex relationships and processes within business documentation. Despite recent advances in Vision-Language Models (VLMs) for various image understanding tasks, accurately identifying and extracting the structures and relationships depicted in diagrams continues to pose significant challenges. This study addresses these challenges by proposing a text-driven approach that bypasses reliance on VLMs' visual recognition capabilities. Instead, it utilizes the editable source files--such as xlsx, pptx or docx--where diagram elements (e.g., shapes, lines, annotations) are preserved as textual metadata. In our proof-of-concept, we extracted diagram information from xlsx-based system design documents and transformed the extracted shape data into textual input for Large Language Models (LLMs). This approach allowed the LLM to analyze relationships and generate responses to business-oriented questions without the bottleneck of image-based processing. Experimental comparisons with a VLM-based method demonstrated that the proposed text-driven framework yielded more accurate answers for questions requiring detailed comprehension of diagram structures.The results obtained in this study are not limited to the tested .xlsx files but can also be extended to diagrams in other documents with source files, such as Office pptx and docx formats. These findings highlight the feasibility of circumventing VLM constraints through direct textual extraction from original source files. By enabling robust diagram understanding through LLMs, our method offers a promising path toward enhanced workflow efficiency and information analysis in real-world business scenarios.

Figures

Figures reproduced from arXiv: 2502.04389 by the authors.

**Figure 1.** Our approach does not rely on the visual recognition of the VLM, but rather inputs and analyzes the LLM as text, with the underlying graphic information stored in the source files referenced to render the Diagram image. efficient document creation and information sharing through automated summarization and report generation (Thoppilan et al., 2022; Liu et al., 2025), and automation of compliance verification tasks (… view at source ↗

**Figure 2.** The diagram used in this study, a system blueprint drawn in Excel using rectangles, text [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** On the left is an XML excerpt of shape data from an xlsx file, difficult to interpret due to raw numerical values and aliased strings. It also contains excessive, scattered information across multiple files. On the right is a JSON format that parses, transforms, and summarizes meaningful information on connectors and shapes, optimized for diagram understanding and LLM input. An example of a system design diagram cre… view at source ↗

Figures from the paper (5 more)

**Figure 2.** Figure 2: This includes components composed of rectangles and text boxes, annotation information [PITH_FULL_IMAGE:figures/full_fig_p006_2.png]

**Figure 4.** Figure 4: Comparison of the output of the proposed method (Text-driven) and VLM (Image Input) for understanding entities in a diagram. The figure above shows the input system design diagram. The ID assigned to each shape indicates the ID assigned by the proposed method when pas…

**Figure 5.** Figure 5: Output comparison of the proposed method (XML-driven) and VLM (Image Input) for understanding relationships in a diagram. The figure above shows the ID assigned to each connector. The connector understanding is correct without omission in the XML-driven approach, whil…

**Figure 6.** Figure 6: Prompts used in text-driven diagram comprehension. The prompts were used where the blue text was replaced with the corresponding information. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: Prompts used in diagram comprehension with image input 14 [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 10 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

GPT - 4V (ision) System Card . 2023. URL https://www.semanticscholar.org/paper/GPT-4V(ision)-System-Card/7a29f47f6509011fe5b19462abf6607867b68373

work page 2023
[3]

PaLM 2 Technical Report , September 2023

Rohan Anil et al. PaLM 2 Technical Report , September 2023. URL http://arxiv.org/abs/2305.10403. arXiv:2305.10403 [cs]

arXiv 2023
[4]

Safeguarding Decentralized Social Media: LLM Agents for Automating Community Rule Compliance

Lucio La Cava and Andrea Tagarelli. Safeguarding Decentralized Social Media : LLM Agents for Automating Community Rule Compliance , September 2024. URL http://arxiv.org/abs/2409.08963. arXiv:2409.08963 [cs]

work page Pith review arXiv 2024
[5]

How far are we to GPT - 4V ? Closing the gap to commercial multimodal models with open-source suites

Zhe Chen et al. How far are we to GPT - 4V ? Closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67 0 (12): 0 220101, December 2024. ISSN 1869-1919. doi:10.1007/s11432-024-4231-5. URL https://doi.org/10.1007/s11432-024-4231-5

doi:10.1007/s11432-024-4231-5 2024
[6]

RAG Foundry : A Framework for Enhancing LLMs for Retrieval Augmented Generation , August 2024

Daniel Fleischer, Moshe Berchansky, Moshe Wasserblat and Peter Izsak. RAG Foundry : A Framework for Enhancing LLMs for Retrieval Augmented Generation , August 2024. URL http://arxiv.org/abs/2408.02545. arXiv:2408.02545 [cs]

arXiv 2024
[7]

VITA -1.5: Towards GPT -4o Level Real - Time Vision and Speech Interaction , January 2025

Chaoyou Fu et al. VITA -1.5: Towards GPT -4o Level Real - Time Vision and Speech Interaction , January 2025. URL http://arxiv.org/abs/2501.01957. arXiv:2501.01957 [cs]

arXiv 2025
[8]

Do Vision - Language Models Really Understand Visual Language ?, September 2024

Buse Giledereli, Yifan Hou, Yilei Tu and Mrinmaya Sachan. Do Vision - Language Models Really Understand Visual Language ?, September 2024. URL http://arxiv.org/abs/2410.00193. arXiv:2410.00193 [cs]

arXiv 2024

Show all 32 references

[9]

Introducing Gemini 2.0: our new AI model for the agentic era, December 2024

Google. Introducing Gemini 2.0: our new AI model for the agentic era, December 2024. URL https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/

2024
[10]

Layoutlmv3: Pre-training for document ai with unified text and image masking

Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu and Furu Wei. Layoutlmv3: Pre-training for document ai with unified text and image masking. In Proceedings of the 30th ACM International Conference on Multimedia, pp.\ 4083--4091, 2022

2022
[11]

ISO / IEC 29500-1:2016, 2016

ISO. ISO / IEC 29500-1:2016, 2016. URL https://www.iso.org/standard/71691.html

2016
[12]

VisOnlyQA : Large Vision Language Models Still Struggle with Visual Perception of Geometric Information , December 2024

Ryo Kamoi, Yusen Zhang, Sarkar Snigdha Sarathi Das, Ranran Haoran Zhang and Rui Zhang. VisOnlyQA : Large Vision Language Models Still Struggle with Visual Perception of Geometric Information , December 2024. URL http://arxiv.org/abs/2412.00947. arXiv:2412.00947 [cs]

2024 arXiv
[13]

( Security ) Assertions by Large Language Models

Rahul Kande, Hammond Pearce, Benjamin Tan, Brendan Dolan-Gavitt, Shailja Thakur, Ramesh Karri and Jeyavijayan Rajendran. ( Security ) Assertions by Large Language Models . IEEE Transactions on Information Forensics and Security, 19: 0 4374--4389, 2024. ISSN 1556-6013, 1556-602...

2024
[14]

Fact, Fetch , and Reason : A Unified Evaluation of Retrieval - Augmented Generation , October 2024

Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay and Manaal Faruqui. Fact, Fetch , and Reason : A Unified Evaluation of Retrieval - Augmented Generation , October 2024. URL http://arxiv.org/abs/2409.12941. arXiv:2409.12941 [cs]

2024 arXiv
[15]

Ferret-ui 2: Mastering universal user interface understanding across platforms, 2024

Zhangheng Li, Keen You, Haotian Zhang, Di Feng, Harsh Agrawal, Xiujun Li, Mohana Prasad Sathya Moorthy, Jeff Nichols, Yinfei Yang and Zhe Gan. Ferret-ui 2: Mastering universal user interface understanding across platforms, 2024

2024
[16]

Yuan Liu et al. MMBench : Is Your Multi -modal Model an All - Around Player ? In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler and Gül Varol (eds.), Computer Vision – ECCV 2024 , pp.\ 216--233, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-0...

2024 doi
[17]

MathVista : Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts , January 2024

Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley and Jianfeng Gao. MathVista : Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts , January 2024. URL http://arxiv.org/abs/2310.02255....

2024 arXiv
[18]

Unraveling the Truth : Do VLMs really Understand Charts ? A Deep Dive into Consistency and Robustness

Srija Mukhopadhyay, Adnan Qidwai, Aparna Garimella, Pritika Ramu, Vivek Gupta and Dan Roth. Unraveling the Truth : Do VLMs really Understand Charts ? A Deep Dive into Consistency and Robustness . In Yaser Al-Onaizan, Mohit Bansal and Yun-Nung Chen (eds.), Findings of the Assoc...

2024 doi
[19]

GPT -4 Technical Report , March 2024 a

OpenAI et al. GPT -4 Technical Report , March 2024 a . URL http://arxiv.org/abs/2303.08774. arXiv:2303.08774 [cs]

2024 arXiv
[20]

GPT -4o System Card , October 2024 b

OpenAI et al. GPT -4o System Card , October 2024 b . URL http://arxiv.org/abs/2410.21276. arXiv:2410.21276 [cs]

2024 arXiv
[21]

FlowLearn : Evaluating Large Vision - Language Models on Flowchart Understanding , July 2024

Huitong Pan, Qi Zhang, Cornelia Caragea, Eduard Dragut and Longin Jan Latecki. FlowLearn : Evaluating Large Vision - Language Models on Flowchart Understanding , July 2024. URL http://arxiv.org/abs/2407.05183. arXiv:2407.05183 [cs]

2024 arXiv
[22]

Vision language models are blind, July 2024

Pooyan Rahmanzadehgervi, Logan Bolton, Mohammad Reza Taesiri and Anh Totti Nguyen. Vision language models are blind, July 2024. URL http://arxiv.org/abs/2407.06581. arXiv:2407.06581 [cs]

2024 arXiv
[23]

FlowVQA : Mapping Multimodal Logic in Visual Question Answering with Flowcharts

Shubhankar Singh, Purvi Chaurasia, Yerram Varun, Pranshu Pandya, Vatsal Gupta, Vivek Gupta and Dan Roth. FlowVQA : Mapping Multimodal Logic in Visual Question Answering with Flowcharts . In Lun-Wei Ku, Andre Martins and Vivek Srikumar (eds.), Findings of the Association for Co...

2024 doi
[24]

LLM4VV : Exploring LLM -as-a- Judge for Validation and Verification Testsuites , August 2024

Zachariah Sollenberger, Jay Patel, Christian Munley, Aaron Jarmusch and Sunita Chandrasekaran. LLM4VV : Exploring LLM -as-a- Judge for Validation and Verification Testsuites , August 2024. URL http://arxiv.org/abs/2408.11729. arXiv:2408.11729 [cs] version: 1

2024 arXiv
[25]

Htmlrag: Html is better than plain text for modeling retrieved knowledge in rag systems

Jiejun Tan, Zhicheng Dou, Wen Wang, Mang Wang, Weipeng Chen and Ji-Rong Wen. Htmlrag: Html is better than plain text for modeling retrieved knowledge in rag systems. arXiv preprint arXiv:2411.02959, 2024

2024 arXiv
[26]

Feighelstein, Jasmina Bogojeska, Joseph Shtok, Assaf Arbelle, Peter W

Simon Tannert, Marcelo G. Feighelstein, Jasmina Bogojeska, Joseph Shtok, Assaf Arbelle, Peter W. J. Staar, Anika Schumann, Jonas Kuhn and Leonid Karlinsky. FlowchartQA : The First Large - Scale Benchmark for Reasoning over Flowcharts . In Piush Aggarwal, \ textbackslash”O\ zge...

2023
[27]

LaMDA : Language Models for Dialog Applications , February 2022

Romal Thoppilan et al. LaMDA : Language Models for Dialog Applications , February 2022. URL http://arxiv.org/abs/2201.08239. arXiv:2201.08239 [cs]

2022 arXiv
[28]

Layoutlmv2: Multi-modal pre-training for visually-rich document understanding

Yang Xu et al. Layoutlmv2: Multi-modal pre-training for visually-rich document understanding. arXiv preprint arXiv:2012.14740, 2020

2012 arXiv
[29]

Beyond End -to- End VLMs : Leveraging Intermediate Text Representations for Superior Flowchart Understanding , December 2024

Junyi Ye, Ankan Dash, Wenpeng Yin and Guiling Wang. Beyond End -to- End VLMs : Leveraging Intermediate Text Representations for Superior Flowchart Understanding , December 2024. URL http://arxiv.org/abs/2412.16420. arXiv:2412.16420 [cs]

2024 arXiv
[30]

Ferret- UI : Grounded Mobile UI Understanding with Multimodal LLMs , April 2024

Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang and Zhe Gan. Ferret- UI : Grounded Mobile UI Understanding with Multimodal LLMs , April 2024. URL http://arxiv.org/abs/2404.05719. arXiv:2404.05719 [cs]

2024 arXiv
[31]

MMMU : A Massive Multi -discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Xiang Yue et al. MMMU : A Massive Multi -discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI . pp.\ 9556--9567, 2024. URL https://openaccess.thecvf.com/content/CVPR2024/html/Yue_MMMU_A_Massive_Multi-discipline_Multimodal_Understanding_and_Reasoning_Bench...

2024
[32]

MathVerse : Does Your Multi -modal LLM Truly See the Diagrams in Visual Math Problems ?, August 2024

Renrui Zhang et al. MathVerse : Does Your Multi -modal LLM Truly See the Diagrams in Visual Math Problems ?, August 2024. URL http://arxiv.org/abs/2403.14624. arXiv:2403.14624 [cs]

2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

GPT - 4V (ision) System Card . 2023. URL https://www.semanticscholar.org/paper/GPT-4V(ision)-System-Card/7a29f47f6509011fe5b19462abf6607867b68373

work page 2023

[3] [3]

PaLM 2 Technical Report , September 2023

Rohan Anil et al. PaLM 2 Technical Report , September 2023. URL http://arxiv.org/abs/2305.10403. arXiv:2305.10403 [cs]

arXiv 2023

[4] [4]

Safeguarding Decentralized Social Media: LLM Agents for Automating Community Rule Compliance

Lucio La Cava and Andrea Tagarelli. Safeguarding Decentralized Social Media : LLM Agents for Automating Community Rule Compliance , September 2024. URL http://arxiv.org/abs/2409.08963. arXiv:2409.08963 [cs]

work page Pith review arXiv 2024

[5] [5]

How far are we to GPT - 4V ? Closing the gap to commercial multimodal models with open-source suites

Zhe Chen et al. How far are we to GPT - 4V ? Closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67 0 (12): 0 220101, December 2024. ISSN 1869-1919. doi:10.1007/s11432-024-4231-5. URL https://doi.org/10.1007/s11432-024-4231-5

doi:10.1007/s11432-024-4231-5 2024

[6] [6]

RAG Foundry : A Framework for Enhancing LLMs for Retrieval Augmented Generation , August 2024

Daniel Fleischer, Moshe Berchansky, Moshe Wasserblat and Peter Izsak. RAG Foundry : A Framework for Enhancing LLMs for Retrieval Augmented Generation , August 2024. URL http://arxiv.org/abs/2408.02545. arXiv:2408.02545 [cs]

arXiv 2024

[7] [7]

VITA -1.5: Towards GPT -4o Level Real - Time Vision and Speech Interaction , January 2025

Chaoyou Fu et al. VITA -1.5: Towards GPT -4o Level Real - Time Vision and Speech Interaction , January 2025. URL http://arxiv.org/abs/2501.01957. arXiv:2501.01957 [cs]

arXiv 2025

[8] [8]

Do Vision - Language Models Really Understand Visual Language ?, September 2024

Buse Giledereli, Yifan Hou, Yilei Tu and Mrinmaya Sachan. Do Vision - Language Models Really Understand Visual Language ?, September 2024. URL http://arxiv.org/abs/2410.00193. arXiv:2410.00193 [cs]

arXiv 2024