Pith. sign in

REVIEW 9 cited by

ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.06598 v3 pith:MORWEZMG submitted 2025-01-11 cs.AI

Xuanle Zhao , Xianzhen Luo , Qi Shi , Chi Chen , Shuo Wang , Zhiyuan Liu , Maosong Sun This is my paper · ORCID
classification cs.AI
keywords chart-to-codecodechartchartcodergenerationchartslanguagemllms
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in chart understanding tasks. However, interpreting charts with textual descriptions often leads to information loss, as it fails to fully capture the dense information embedded in charts. In contrast, parsing charts into code provides lossless representations that can effectively contain all critical details. Although existing open-source MLLMs have achieved success in chart understanding tasks, they still face two major challenges when applied to chart-to-code tasks: (1) Low executability and poor restoration of chart details in the generated code and (2) Lack of large-scale and diverse training data. To address these challenges, we propose \textbf{ChartCoder}, the first dedicated chart-to-code MLLM, which leverages Code LLMs as the language backbone to enhance the executability of the generated code. Furthermore, we introduce \textbf{Chart2Code-160k}, the first large-scale and diverse dataset for chart-to-code generation, and propose the \textbf{Snippet-of-Thought (SoT)} method, which transforms direct chart-to-code generation data into step-by-step generation. Experiments demonstrate that ChartCoder, with only 7B parameters, surpasses existing open-source MLLMs on chart-to-code benchmarks, achieving superior chart restoration and code excitability. Our code is available at https://github.com/thunlp/ChartCoder.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Chart Specification: Structural Representations for Incentivizing VLM Reasoning in Chart-to-Code Generation

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A 7B VLM trained with a structured chart-specification reward beats larger and commercial models on chart-to-code benchmarks using only 3K-4K training samples.

  2. SVRepair: Structured Visual Reasoning for Automated Program Repair

    cs.SE 2026-02 conditional novelty 6.0 of 10

    A multimodal program-repair system that converts bug screenshots into semantic scene graphs and iteratively crops to bug regions, reporting 36.47% on SWE-Bench M, 38.02% on MMCode, and 95.12% on CodeVision.

  3. Visual Programmability: A Guide for Code-as-Thought in Chart Understanding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A vision-language model learns to dynamically switch between code-based and visual reasoning for chart questions, improving average accuracy by about one point over fixed strategies.

  4. Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback

    cs.CV 2025-07 conditional novelty 6.0 of 10

    RRVF trains an image-to-code MLLM using reinforcement learning with a render-and-compare visual feedback loop, and it outperforms supervised fine-tuning on chart and web benchmarks.

  5. Multilingual Multimodal Software Developer for Code Generation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A 7B vision-language model trained on synthetic diagram-to-code data outperforms several larger open-weight models on a new 10-language UML/flowchart code-generation benchmark.

  6. ChartGen: Scaling Chart Understanding Via Code-Guided Synthetic Chart Generation

    cs.HC 2025-05 conditional novelty 6.0 of 10

    A fully automated pipeline generates a 222.5K-pair synthetic chart dataset with 27 chart types and 11 plotting libraries, and a GPT-4o-judged benchmark shows current open-weights VLMs still underperform on chart-to-co...

  7. VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    Merging a coding LLM into a vision-language model via task vectors yields an open-source multimodal coder that reaches near-GPT-4o performance on the authors' new benchmark.

  8. Enhancing Multimodal Continual Instruction Tuning with BranchLoRA

    cs.CL 2025-05 conditional novelty 5.0 of 10

    BranchLoRA reduces catastrophic forgetting in multimodal continual instruction tuning by using a shared LoRA matrix, task-specific branches, frozen experts, and learned task keys.

  9. Information-Theoretic Complementary Prompts for Improved Continual Text Classification

    cs.CL 2025-05 conditional novelty 4.0 of 10

    InfoComp learns per-task private prompts and one shared prompt with parameter-alignment and SimSiam-style losses, reporting consistent accuracy gains over prior state-of-the-art continual text classification methods o...

Pith tools