Pith. sign in

REVIEW 10 cited by

ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.02384 v3 pith:46H2ZY75 submitted 2024-01-04 cs.CV

Fanqing Meng , Wenqi Shao , Quanfeng Lu , Peng Gao , Kaipeng Zhang , Yu Qiao , Ping Luo This is my paper · ORCID
classification cs.CV
keywords chartdatachartassistantbarschallengeschart-to-tablecomprehensionmodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Charts play a vital role in data visualization, understanding data patterns, and informed decision-making. However, their unique combination of graphical elements (e.g., bars, lines) and textual components (e.g., labels, legends) poses challenges for general-purpose multimodal models. While vision-language models trained on chart data excel in comprehension, they struggle with generalization. To address these challenges, we propose ChartAssistant, a chart-based vision-language model for universal chart comprehension and reasoning. ChartAssistant leverages ChartSFT, a comprehensive dataset covering diverse chart-related tasks with basic (e.g. bars and pies) and specialized (e.g. radars, and bubbles) chart types. It undergoes a two-stage training process, starting with pre-training on chart-to-table parsing to align chart and text, followed by multitask instruction-following fine-tuning. This approach enables ChartAssistant to achieve competitive performance across various chart tasks. Experimental results demonstrate significant performance gains over the state-of-the-art UniChart and Chartllama method, especially outperforming them on real-world chart data with zero-shot setting. The code and data are available at https://github.com/OpenGVLab/ChartAst.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ChartCap: Mitigating Hallucination of Dense Chart Captioning

    cs.CV 2025-08 conditional novelty 7.0 of 10

    A new 565K-pair chart-caption dataset with schema-based dense captions and a reference-free visual consistency metric improves VLM captioning and reduces hallucination.

  2. Chart Specification: Structural Representations for Incentivizing VLM Reasoning in Chart-to-Code Generation

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A 7B VLM trained with a structured chart-specification reward beats larger and commercial models on chart-to-code benchmarks using only 3K-4K training samples.

  3. Visual Programmability: A Guide for Code-as-Thought in Chart Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A vision-language model learns to dynamically switch between code-based and visual reasoning for chart questions, improving average accuracy by about one point over fixed strategies.

  4. FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A new benchmark of real-world financial charts shows current vision-language models lag badly on questions that require reading values from chart axes.

  5. VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual Tool Selection

    cs.CV 2025-05 conditional novelty 6.0 of 10

    VisTA uses GRPO reinforcement learning to train a vision-language agent to select external visual tools for a frozen reasoning model, improving accuracy on ChartQA, Geometry3K, BlindTest, and MathVerse.

  6. ChartLens: Fine-grained Visual Attribution in Charts

    cs.CL 2025-05 conditional novelty 6.0 of 10

    ChartLens uses segmentation and set-of-marks prompting to attribute chart-based answers to specific visual elements, and the authors release a new benchmark for evaluating such attribution.

  7. Bootstrapping Grounded Chain-of-Thought in Multimodal LLMs for Data-Efficient Model Adaptation

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Injecting self-verified bounding boxes into Chain-of-Thought data improves few-shot adaptation of multimodal LLMs on charts, tables, receipts, and reports.

  8. CHAOS: Chart Analysis with Outlier Samples

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A chart perturbation robustness benchmark with five textual and ten visual distortion types, three human-calibrated severity levels, and evaluations of 13 MLLMs on ChartQA and chart summarization.

  9. Adaptive Sparse Softmax: An Effective and Efficient Softmax Variant

    cs.LG 2025-08 unverdicted novelty 4.0 of 10

    The preprint's abstract claims a sparse softmax variant that masks non-competitive classes and accelerates training, but the provided body contains an unrelated chart-captioning paper and none of the claimed method.

  10. ChartSketcher: Reasoning with Multimodal Feedback and Reflection for Chart Understanding

    cs.CV 2025-05 conditional novelty 4.0 of 10

    ChartSketcher has a multimodal LLM sketch intermediate reasoning steps directly on chart images and feed those sketches back as visual feedback, improving chart QA accuracy over its base model.

Pith tools