Pith. sign in

REVIEW 4 major objections 6 minor 47 references

ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ChartMind and ChartLLM claim that feeding only a chart's title, legend, and axes to vision-language models outperforms OCR, chain-of-thought, and plain instruction following.

desk verdict ChartMind is a useful bilingual open-ended chart QA benchmark, but its headline comparison on that benchmark rests on a judge that never sees the chart. read the letter →

arxiv 2505.23242 v1 pith:SMXYYZ5S submitted 2025-05-29 cs.CL

Jingxuan Wei , Nan Xu , Junnan Zhu , Yanni Hao , Gaowei Wu , Bihui Yu , Lei Wang This is my paper · ORCID
classification cs.CL
keywords chartquestionansweringmultimodallargelanguagemodelsbenchmarkconstructionopen-endedevaluationmultilingualchartscontextextractionchain-of-thoughtcomparisonOCRenhancement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the standard ways of prompting vision-language models for chart questions—plain instructions, OCR text dumps, and chain-of-thought decomposition—are not the most effective route, and that extracting a small structured context first is better. To make the case it builds ChartMind, a 757-question benchmark with seven task types, seven chart formats, and English and Chinese charts, with open-ended answers instead of single labels. It then proposes ChartLLM, a model-agnostic wrapper that pulls out a chart's title, legend, and axis labels and gives them to the model before answering. Across 14 models and four benchmarks, the wrapper consistently outscores the three competing paradigms, especially on Chinese charts and on open-ended tasks. If the claim holds, simple input restructuring is a cheap and portable way to improve real-world chart reasoning.

What carries the argument

The mechanism is ChartLLM's context extraction: given a chart $C$, it defines $C_{\text{context}} = \{T, L, X, Y\}$, the chart's title, legend, X-axis label, and Y-axis label, and asks the model to answer using these cues rather than the raw image alone. The paper contrasts this with chain-of-thought reasoning, which decomposes the answer into steps $r_1,\dots,r_k$; ChartLLM instead reduces the perceptual burden before reasoning begins. The objective is to choose the answer $a$ that maximizes the expected log-likelihood $\sum_i E_{C_{\text{context}},Q}[\log P(a_i | C_{\text{context}}, Q; \Theta)]$. The benchmark itself is built through chart collection, GPT-4o-based QA generation, and human validation with answer rewriting, which supplies the evaluation setting where the method's advantage is measured.

What would settle it

Score the same 757 ChartMind answers with human raters on a disclosed sample (for example, 200 answers per model) and compare the resulting model ranking to the GPT-4o ranking; if the ChartLLM advantage reverses or the GPT-4o-human correlation falls well below 93, the central claim fails. A second check is to swap the context extractor (the paper uses Qwen2-VL) for another model and see whether ChartLLM's edge survives.

Watch

Extended reading notes

Core claim

The central discovery is that a context-aware input representation, not a more elaborate reasoning procedure, is what most improves current multimodal chart question answering. ChartMind provides a real-world evaluation setting where ChartLLM-based methods achieve the best average GPT-4o scores across seven task categories in both English and Chinese, and the same pattern holds on ChartQA, Chart-to-Text, and OpenCQA. The paper interprets this as evidence that pre-structuring the semantically relevant parts of a chart reduces perceptual noise and lets the underlying model reason from the components that matter, a strategy that generalizes across model families and chart formats. The paper also reports that GPT-4o scoring correlates with human scoring at 93.09, which it uses to justify measuring open-ended answers automatically.

Load-bearing premise

The load-bearing premise is that the GPT-4o-based automatic scorer measures open-ended answer quality the same way a human would; the paper supports this with a single Pearson correlation of 93.09 and does not report the human-evaluation sample size.

Editorial extensions

If this is right

  • ChartLLM-based methods outperform instruction-following, OCR-enhanced, and chain-of-thought paradigms on ChartMind and on ChartQA, Chart-to-Text, and OpenCQA, across most of the 14 models tested.
  • Adding raw OCR text is not always helpful: it improves structured-output tasks but lowers accuracy on open-ended reasoning, so input noise matters as much as reasoning style.
  • Chinese chart questions are consistently harder than English ones, establishing a measurable cross-lingual gap that monolingual benchmarks cannot expose.
  • GPT-4o's automatic open-ended scoring tracks human scores closely enough (PCC = 93.09) that future CQA work can use it as a cheaper stand-in for human evaluation.
  • Because ChartLLM is model-agnostic, the same extracted context can be plugged into any multimodal model, making the gain a property of the input format rather than of one architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that ChartLLM's context extraction depends on a strong reader (it uses Qwen2-VL); a natural extension is to test whether weaker extractors erode the advantage, which would make the method's value contingent on extraction quality.
  • A cross-lingual implication: since Chinese tasks are harder for every paradigm, bilingual benchmarks could use the English-Chinese gap as a diagnostic for how much of a model's chart reasoning is language-grounded versus vision-grounded.
  • A testable extension is to feed ChartLLM's extracted context into the OCR and CoT paradigms as well; if those also improve, the three 'competing' paradigms may converge on the same best practice rather than being fundamentally different.
  • If the advantage replicates on unseen chart types and languages, chart QA practice could shift from prompt engineering toward explicit semantic parsing of chart structure as a standard preprocessing step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ChartMind, a 757-item bilingual (English/Chinese) chart question-answering benchmark spanning seven task categories and seven chart types, with questions and reference answers generated by GPT-4o and then human-validated. It also proposes ChartLLM, a model-agnostic prompting framework that extracts structured chart context (title, legend, axis labels) and feeds it to a multimodal LLM before answering. The authors compare four paradigms—instruction-following, OCR-enhanced, chain-of-thought, and ChartLLM—across 14 MLLMs on ChartMind, ChartQA, Chart-to-Text, and OpenCQA, and claim that ChartLLM-based methods consistently outperform the other three paradigms, particularly on ChartMind's open-ended tasks.

Significance. If the empirical claims held, ChartMind could be a valuable resource for evaluating open-ended, multilingual chart reasoning, and ChartLLM is a simple, model-agnostic context-extraction recipe that could transfer to other MLLMs. The benchmark construction pipeline is described in sufficient detail to be reproduced, and the inclusion of both English and Chinese material across multiple chart and task types fills a gap in existing chart QA benchmarks. The paper also explicitly acknowledges limitations around automated evaluation and data bias. However, the validity of the headline results currently hinges on an automated judge that never receives the chart and on correlation evidence that is too weak to rule out judge bias, so the central quantitative conclusions are not yet established.

major comments (4)
  1. [Appendix C, Figure 8] The GPT-4o scoring prompt takes as input only a JSON object containing "question", "human_reference", and "model_answer"; no chart image or chart-derived data is provided to the judge. The rubric nevertheless asks the judge to assess consistency with "the reference answer or the chart content", and the scoring criteria state that data should be "consistent with the reference answer or the chart content". Since the chart content is absent from the judge's inputs, the correctness score can only measure textual similarity to the reference answer. A fluent answer that contradicts the chart but paraphrases the reference would score high, while a correct answer with different wording could score low. Because the ChartMind open-ended results in Table 3 and the cross-lingual claims in Section 5.4 rest on this score, the central comparison is confounded.
  2. [Table 4, Section 5.3] The reported validation of the automated score is insufficient. The Pearson correlation coefficient of 93.09 is computed on average GPT-4o and human scores across 14 models, not on item-level judgments; the number of human-annotated items is not disclosed, and no inter-annotator agreement is reported. An aggregate correlation across model means can be high even if the judge systematically prefers reference-like phrasing, because that bias would affect all models to different degrees. To validate the metric, the authors need item-level human scores with agreement statistics, a description of how human annotators accessed the chart, and either per-item correlation or a bias analysis that includes adversarial answers that are fluent but chart-contradicting or correct but differently worded.
  3. [Table 3, Section 5.2] The abstract and Section 5.2 claim that ChartLLM-based methods "significantly outperform" the other paradigms, but no error bars, repeated runs, significance tests, or effect sizes are reported. Several deltas in Table 3 are small—for example, TinyChart gains of +1.35 on ChartMind ACC and +1.62 on GPT-4o score, and ChartLlama loses 0.27 on Chart-to-Text Avg. BLEU under OCR enhancement—while some OCR-enhanced results degrade substantially. The claim of consistent superiority therefore requires at least bootstrap or permutation tests over chart/task samples, or confidence intervals for the reported differences.
  4. [Sections 3.2, 3.3, and Appendix C] There is a residual circularity risk: the gold answers are generated by GPT-4o, and GPT-4o is also used as the automated judge. The high GPT-4o scores on ChartMind (e.g., 73.89 under ChartLLM in Table 3) may partly reflect stylistic similarity to GPT-4o-generated references rather than reasoning quality. The human validation and answer rewriting in Stage III reduce but do not eliminate this risk, especially because the judge does not see the chart. A concrete test would be to score the same model answers with a judge that receives only the chart and question (not the human reference), or to include reference-paraphrase and chart-contradicting answers in the human evaluation sample to measure whether the judge is biased toward reference-like wording.
minor comments (6)
  1. [Section 3.3] The text contains a LaTeX artifact "textcolorblue" that should be removed or rendered properly.
  2. [Appendix C, Figure 8] The scoring prompt says scores are 0 or 1, while Table 3 reports GPT-4o scores on a 0-100 scale; the scaling should be stated explicitly.
  3. [Appendix B / Data Availability] The paper does not provide a download link, repository URL, or dataset license for ChartMind; a data availability statement should be added.
  4. [Table 3] There are formatting errors in Table 3, such as "23.04 (+0.72 19.73 (-8.75)", where a closing parenthesis is missing; the table should be proofread.
  5. [Section 1] The claim that ChartMind is the "first dual-language evaluation setting" for chart QA is stronger than necessary and should be qualified with a comparison to existing multilingual chart benchmarks.
  6. [Appendix C, Figure 8] The prompt text says the judge should evaluate based on "charts", but the input JSON has no chart field; this internal inconsistency should be fixed or explained.

Circularity Check

1 steps flagged · score 4.0 of 10

GPT-4o both generates the ChartMind reference answers and serves as the automated judge for open-ended answers, making the primary open-ended metric partially self-referential.

  1. other [Appendix C, Figure 8; Section 3.2; Appendix B.3]
    "Output Correctness (0-1 points): Assess whether the reasoning is correct overall, whether most of the data is accurate, and whether the model's answer aligns with the logic of the human reference answer. ... The input is a JSON object with the following fields: "question": "string, the question description", "human_reference": "string, the human reference answer", "model_answer": "string, the model's generated answer" ... Given a chart, we generate diverse QA pairs for seven tasks (e.g., summarization, classification, suggestion) using GPT-4o."

    The scoring prompt's input contains only the question, the human reference answer, and the model answer; no chart image or chart-derived data is included. The rubric tells the judge to check consistency with 'the reference answer or the chart content,' but since the chart content is absent from the input, the only checkable object is the human reference. Those references were originally drafted by GPT-4o in Stage II and then human-rewritten, while the judge is also GPT-4o. Thus the automated 'Output Correctness' score reduces, by construction, to GPT-4o's assessment of textual alignment between a model answer and a GPT-4o-influenced reference, rather than to chart-grounded factuality.

full rationale

The paper's central framework claim is not wholly circular: ChartLLM also shows improvements on three public benchmarks scored with objective metrics (ChartQA ACC, Chart-to-Text CIDEr, OpenCQA BLEU), and the ChartMind gold answers were human-validated and rewritten, so the benchmark is not purely GPT-4o output. No load-bearing self-citation chain or uniqueness-theorem import was found. However, the open-ended portion of ChartMind is scored by GPT-4o using a prompt that excludes the chart and includes a human reference originally generated by GPT-4o; the judge cannot verify chart content and therefore scores a form of textual consistency with a GPT-4o-derived reference. This is a partial evaluator circularity that specifically affects the headline cross-lingual and paradigm comparisons on ChartMind's open-ended tasks. The human-correlation check in Table 4 mitigates but does not eliminate the issue, since it is computed on 14 model-level averages and the sample size is undisclosed. Overall score 4 reflects a real, bounded circularity in the primary open-ended metric rather than a collapse of the entire derivation chain.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rest on the quality of the GPT-4o-generated and human-validated data, the validity of the GPT-4o scoring metric, and the assumption that the chosen chart/task taxonomy is representative. No free parameters are fit from the data; the method's prompts and thresholds are hand-designed.

assumptions (5)
  • domain assumption GPT-4o-generated QA pairs after human review are valid ground truth for ChartMind.
    Section 3.2 and 3.3 describe using GPT-4o for initial generation and human annotators for validation, but inter-annotator agreement is not reported.
  • domain assumption GPT-4o score is a valid proxy for human judgment in open-ended CQA.
    Appendix C and Section 5.3 compare GPT-4o score to human scores via PCC, but the human evaluation sample size is undisclosed.
  • ad hoc to paper The seven task categories and seven chart types cover representative real-world chart analysis.
    The taxonomy is defined by the authors and not derived from a systematic analysis of real-world chart use cases.
  • domain assumption Context extraction of title, legend, and axis labels using Qwen2-VL is accurate enough to improve reasoning.
    The ChartLLM framework in Section 4.3 assumes this extraction is reliable, but no error analysis of the extraction step is provided.
  • domain assumption All collected charts are appropriately licensed and representative.
    Section 3.1 states charts come from open-source platforms with permissive licenses, but the representativeness bias is acknowledged as a limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering." pith.science (2026). https://pith.science/paper/SMXYYZ5S

@misc{pith2026250523242,
  author       = {Pith},
  title        = {Pith review of: ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SMXYYZ5S}},
  note         = {Machine review of arXiv:2505.23242}
}
read the original abstract

Chart question answering (CQA) has become a critical multimodal task for evaluating the reasoning capabilities of vision-language models. While early approaches have shown promising performance by focusing on visual features or leveraging large-scale pre-training, most existing evaluations rely on rigid output formats and objective metrics, thus ignoring the complex, real-world demands of practical chart analysis. In this paper, we introduce ChartMind, a new benchmark designed for complex CQA tasks in real-world settings. ChartMind covers seven task categories, incorporates multilingual contexts, supports open-domain textual outputs, and accommodates diverse chart formats, bridging the gap between real-world applications and traditional academic benchmarks. Furthermore, we propose a context-aware yet model-agnostic framework, ChartLLM, that focuses on extracting key contextual elements, reducing noise, and enhancing the reasoning accuracy of multimodal large language models. Extensive evaluations on ChartMind and three representative public benchmarks with 14 mainstream multimodal models show our framework significantly outperforms the previous three common CQA paradigms: instruction-following, OCR-enhanced, and chain-of-thought, highlighting the importance of flexible chart understanding for real-world CQA. These findings suggest new directions for developing more robust chart reasoning in future research.

Figures

Figures reproduced from arXiv: 2505.23242 by the authors.

Figure 1
Figure 1. Key Challenges in CQA Benchmarks: (A) Predominantly monolingual, limiting multilingual applicability [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Data Construction Pipeline for the ChartMind. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Topic distribution in ChartMind. Task Samples Query Length Answer Length (Min / Max) (Min / Max) Chart Conversion 140 11 / 477 5 / 55 Chart OCR Recognition 139 13 / 351 8 / 59 Suggestions 88 17 / 492 13 / 53 Chart Classification Analysis 37 360 / 503 72 / 79 Chart Summarization 34 76 / 335 12 / 113 Chart Assistance 76 9 / 276 12 / 41 Information Positioning 140 11 / 208 11 / 35 Total 757 9 / 503 5 / 113 [PITH_FULL_… view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Performance of multimodal models across Chinese and English datasets in ChartMind. hen et al., 2009) is 93.09, indicating a strong linear relationship. High-performing models like GPT￾4o (Achiam et al., 2023) and Qwen2-VL (Bai et al., 2023) show strong alignment betwee…
Figure 6
Figure 6. Figure 6: Performance of multimodal models on seven tasks in ChartMind. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Overview of the seven chart types and seven reasoning tasks included in ChartMind. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Prompt design for GPT-4o score. accurate arithmetic operations (calculation), and misassociation of chart elements with their respec￾tive colors in legends or overlapping areas (color recognition). The figure provides detailed scenar￾ios, such as errors in identifying …
Figure 9
Figure 9. Figure 9: Performance of multimodal models across chart types, categorized by four paradigms. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: The four major error types in ChartMind. [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 27 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, et al. 2023. Gpt-4 technical report. In arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. In arXiv preprint arXiv:2308.12966

    arXiv 2023

  5. [5]

    Filip Baji \'c and Josip Job. 2023. Review of chart image detection and classification. IJDAR, 26(4):453--474

    work page 2023

  6. [6]

    Mu Cai, Haotian Liu, and others. 2024. Vip-llava: Making large multimodal models understand arbitrary visual prompts. In CVPR, pages 12914--12923

    work page 2024

  7. [7]

    Shuaichen Chang, David Palzer, Jialin Li, Eric Fosler-Lussier, and Ningchuan Xiao. 2022. Mapqa: A dataset for question answering on choropleth maps. In arXiv preprint arXiv:2211.08545

    arXiv 2022

  8. [8]

    Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, et al. 2009. Pearson correlation coefficient. Noise reduction in speech processing, 16(4):1--4

    work page 2009

Show all 47 references
  1. [9]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. 2024. A survey on in-context learning. In EMNLP, pages 1107--1128

    2024

  2. [10]

    Team GLM, Aohan Zeng, Bin Xu, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. In arXiv preprint arXiv:2406.12793

    2024 arXiv

  3. [11]

    Yucheng Han, Chi Zhang, Xin Chen, Xu Yang, Zhibin Wang, Gang Yu, Bin Fu, and Hanwang Zhang. 2023. Chartllama: A multimodal llm for chart understanding and generation. In arXiv preprint arXiv:2311.16483

    2023 arXiv

  4. [12]

    Shengding Hu, Yuge Tu, et al. 2024. Minicpm: Unveiling the potential of small language models with scalable training strategies. In COLM, pages 1--33

    2024

  5. [13]

    Kung-Hsiang Huang, Hou Pong Chan, Yi R Fung, Haoyi Qiu, Mingyang Zhou, Shafiq Joty, Shih-Fu Chang, and Heng Ji. 2024. From pixels to insights: A survey on automatic chart understanding in the era of large foundation models. In arXiv preprint arXiv:2403.12027

    2024 arXiv

  6. [14]

    Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. 2018. Dvqa: Understanding data visualizations via question answering. In CVPR, pages 5648--5656

    2018

  7. [15]

    Shankar Kantharaj, Xuan Long Do, Rixie Tiffany Leong, Jia Qing Tan, Enamul Hoque, and Shafiq Joty. 2022 a . Opencqa: Open-ended question answering with charts. In EMNLP, pages 11817--11837

    2022

  8. [16]

    Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. 2022 b . Chart-to-text: A large-scale benchmark for chart summarization. In ACL, pages 4005--4023

    2022

  9. [17]

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022. Ocr-free document understanding transformer. In ECCV, volume 13688, pages 498--517

    2022

  10. [18]

    Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2struct: Screenshot parsing as pretraining for visual language understanding. In ICML, pages 18893--18912

    2023

  11. [19]

    Lei Li, Yuqi Wang, Runxin Xu, Peiyi Wang, Xiachong Feng, Lingpeng Kong, and Qi Liu. 2024 a . Multimodal arxiv: A dataset for improving scientific comprehension of large vision-language models. In arXiv preprint arXiv:2403.00231

    2024 arXiv

  12. [20]

    Zhe Li, Xinyu Wang, Yuliang Liu, Lianwen Jin, et al. 2023. Improving handwritten mathematical expression recognition via similar symbol distinguishing. TMM, 26:90--102

    2023

  13. [21]

    Zhuowan Li, Bhavan Jasani, et al. 2024 b . Synthesize step-by-step: Tools templates and llms as data generators for reasoning-based chart vqa. In CVPR, pages 13613--13623

    2024

  14. [22]

    Ziyi Lin, Chris Liu, Renrui Zhang, et al. 2023. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. In arXiv preprint arXiv:2311.07575

    2023 arXiv

  15. [23]

    Fangyu Liu, Julian Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, and Yasemin Altun. 2023. Deplot: One-shot visual language reasoning by plot-to-table translation. In ACL, pages 10381--10399

    2023

  16. [24]

    Fuxiao Liu, Xiaoyang Wang, et al. 2024 a . Mmc: Advancing multimodal chart understanding with large-scale instruction tuning. In NAACL, pages 1287--1310

    2024

  17. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024 b . Llava-next: Improved reasoning, ocr, and world knowledge

    2024

  18. [26]

    Haotian Liu, Chunyuan Li, et al. 2024 c . Improved baselines with visual instruction tuning. In CVPR, pages 26296--26306

    2024

  19. [27]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In arXiv preprint arXiv:2310.02255

    2023 arXiv

  20. [28]

    Linfeng Ma, Han Fang, Zehua Ma, Zhaoyang Jia, Weiming Zhang, and Nenghai Yu. 2024. C 3 hartmark: A chart watermarking scheme with consecutive-encoding and concurrent-decoding. TCSVT, 34(10):4005--4018

    2024

  21. [29]

    Anita Mahinpei, Zona Kostic, and Chris Tanner. 2022. Linecap: Line charts for data visualization captioning models. In 2022 IEEE VIS, pages 35--39

    2022

  22. [30]

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In ACL, pages 2263--2279

    2022

  23. [31]

    Ahmed Masry, Mehrad Shahmohammadi, Md Rizwan Parvez, Enamul Hoque, and Shafiq Joty. 2024. Chartinstruct: Instruction tuning for chart comprehension and reasoning. In arXiv preprint arXiv:2403.09028

    2024 arXiv

  24. [32]

    Bosheng Qin, Haoji Hu, and Yueting Zhuang. 2022. Deep residual weight-sharing attention network with low-rank attention for visual question answering. TMM, 25:4282--4295

    2022

  25. [33]

    Cheng Tan, Jingxuan Wei, Zhangyang Gao, Linzhuang Sun, Siyuan Li, Ruifeng Guo, Bihui Yu, and Stan Z Li. 2024. Boosting the power of small multimodal reasoning models to match larger models with self-consistency training. In ECCV, pages 305--322. Springer

    2024

  26. [34]

    Peifang Wang, Olga Golovneva, Armen Aghajanyan, Xiang Ren, Muhao Chen, Asli Celikyilmaz, and Maryam Fazel-Zarandi. 2023 a . Domino: A dual-system for multi-step visual language reasoning. In arXiv preprint arXiv:2310.02804

    2023 arXiv

  27. [35]

    Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. 2023 b . Cogvlm: Visual expert for pretrained language models. In arXiv preprint arXiv:2311.03079

    2023 arXiv

  28. [36]

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. In arXiv preprint arXiv:2109.01652

    2021 arXiv

  29. [37]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 35:24824--24837

    2022

  30. [38]

    Jingxuan Wei, Nan Xu, Guiyong Chang, Yin Luo, BiHui Yu, and Ruifeng Guo. 2024. mchartqa: A universal benchmark for multimodal chart question answer based on vision-language alignment and reasoning. In arXiv preprint arXiv:2404.01548

    2024 arXiv

  31. [39]

    Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xinrun Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, et al. 2024 a . Tablebench: A comprehensive and complex benchmark for table question answering. In arXiv preprint arXiv:2408.09174

    2024 arXiv

  32. [40]

    Yifan Wu, Lutao Yan, Leixian Shen, Yunhai Wang, Nan Tang, and Yuyu Luo. 2024 b . Chartinsights: Evaluating multimodal large language models for low-level chart question answering. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 12174--12200

    2024

  33. [41]

    Jie Xu, Xiaoqian Zhang, Changming Zhao, et al. 2023. Improving fine-grained image classification with multimodal information. TMM, 25(8):2082 -- 2095

    2023

  34. [42]

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. 2024. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In CVPR, pages 13040--13051

    2024

  35. [43]

    Munazza Zaib, Wei Emma Zhang, Quan Z Sheng, Adnan Mahmood, and Yang Zhang. 2022. Conversational question answering: A survey. KIS, 64(12):3151--3195

    2022

  36. [44]

    Xingchen Zeng, Haichuan Lin, Yilin Ye, and Wei Zeng. 2024. Advancing multimodal large language models in chart question answering with visualization-referenced instruction tuning. TVCG, 30(11):1--11

    2024

  37. [45]

    Liang Zhang, Anwen Hu, Haiyang Xu, Ming Yan, Yichen Xu, Qin Jin, Ji Zhang, and Fei Huang. 2024 a . Tinychart: Efficient chart understanding with visual token merging and program-of-thoughts learning. In arXiv preprint arXiv:2404.16635

    2024 arXiv

  38. [46]

    Pan Zhang, Xiaoyi Dong, et al. 2024 b . Internlm-xcomposer-2.5: A versatile large vision language model supporting long-contextual input and output. In arXiv preprint arXiv:2407.03320

    2024 arXiv

  39. [47]

    Jeffrey Zhou, Tianjian Lu, et al. 2023. Instruction-following evaluation for large language models. In arXiv preprint arXiv:2311.07911

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.