REVIEW 7 cited by
ChartGemma: Visual Instruction-tuning for Chart Reasoning in the Wild
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Given the ubiquity of charts as a data analysis, visualization, and decision-making tool across industries and sciences, there has been a growing interest in developing pre-trained foundation models as well as general purpose instruction-tuned models for chart understanding and reasoning. However, existing methods suffer crucial drawbacks across two critical axes affecting the performance of chart representation models: they are trained on data generated from underlying data tables of the charts, ignoring the visual trends and patterns in chart images, and use weakly aligned vision-language backbone models for domain-specific training, limiting their generalizability when encountering charts in the wild. We address these important drawbacks and introduce ChartGemma, a novel chart understanding and reasoning model developed over PaliGemma. Rather than relying on underlying data tables, ChartGemma is trained on instruction-tuning data generated directly from chart images, thus capturing both high-level trends and low-level visual information from a diverse set of charts. Our simple approach achieves state-of-the-art results across $5$ benchmarks spanning chart summarization, question answering, and fact-checking, and our elaborate qualitative studies on real-world charts show that ChartGemma generates more realistic and factually correct summaries compared to its contemporaries. We release the code, model checkpoints, dataset, and demos at https://github.com/vis-nlp/ChartGemma.
Forward citations
Cited by 7 Pith papers
-
ChartCap: Mitigating Hallucination of Dense Chart Captioning
A new 565K-pair chart-caption dataset with schema-based dense captions and a reference-free visual consistency metric improves VLM captioning and reduces hallucination.
-
FinReportBench: Measuring and Improving Institution-Grade Financial Report Generation
FinReportBench is a fine-grained, expert-grounded benchmark for institution-grade LLM financial report generation, and its skill-evolution method improves G1 and G2 scores across model families.
-
Visual Programmability: A Guide for Code-as-Thought in Chart Understanding
A vision-language model learns to dynamically switch between code-based and visual reasoning for chart questions, improving average accuracy by about one point over fixed strategies.
-
Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approach
A draft-and-repair agentic loop using GPT-4o-mini reduces text-to-chart execution errors to 4.5-4.6% on two benchmarks, suggesting execution is nearly solved and future work should focus on quality and accessibility.
-
AutoTool: Dynamic Tool Selection and Integration for Agentic Reasoning
AutoTool's two-phase SFT/RL plus ranking training lets 8B LLM agents beat larger fixed-tool agents across math, search, code, and vision benchmarks, though unseen-tool gains are asserted, not isolated.
-
Bootstrapping Grounded Chain-of-Thought in Multimodal LLMs for Data-Efficient Model Adaptation
Injecting self-verified bounding boxes into Chain-of-Thought data improves few-shot adaptation of multimodal LLMs on charts, tables, receipts, and reports.
-
Adaptive Sparse Softmax: An Effective and Efficient Softmax Variant
The preprint's abstract claims a sparse softmax variant that masks non-competitive classes and accelerates training, but the provided body contains an unrelated chart-captioning paper and none of the claimed method.
Discussion (0). Continue with ORCID to comment.