Pith. sign in

REVIEW 5 cited by

TableLlama: Towards Open Large Generalist Models for Tables

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.09206 v3 pith:MURZEZHH submitted 2023-11-15 cs.CL cs.AIcs.DB

Tianshu Zhang , Xiang Yue , Yifei Li , Huan Sun This is my paper · ORCID
classification cs.CLcs.AIcs.DB
keywords tablesmodeltasksgeneralistmodelsopen-sourcetablellamatowards
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Semi-structured tables are ubiquitous. There has been a variety of tasks that aim to automatically interpret, augment, and query tables. Current methods often require pretraining on tables or special model architecture design, are restricted to specific table types, or have simplifying assumptions about tables and tasks. This paper makes the first step towards developing open-source large language models (LLMs) as generalists for a diversity of table-based tasks. Towards that end, we construct TableInstruct, a new dataset with a variety of realistic tables and tasks, for instruction tuning and evaluating LLMs. We further develop the first open-source generalist model for tables, TableLlama, by fine-tuning Llama 2 (7B) with LongLoRA to address the long context challenge. We experiment under both in-domain setting and out-of-domain setting. On 7 out of 8 in-domain tasks, TableLlama achieves comparable or better performance than the SOTA for each task, despite the latter often has task-specific design. On 6 out-of-domain datasets, it achieves 5-44 absolute point gains compared with the base model, showing that training on TableInstruct enhances the model's generalizability. We open-source our dataset and trained model to boost future work on developing open generalist models for tables.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Utilizing Training Data to Improve LLM Reasoning for Tabular Understanding

    cs.LG 2025-08 conditional novelty 6.0 of 10

    LRTab retrieves error-avoiding prompt conditions learned from incorrect chain-of-thought traces on training tables to improve LLM tabular reasoning, achieving modest gains on WikiTQ and TabFact.

  2. MTabVQA: Evaluating Multi-Tabular Reasoning of Language Models in Visual Space

    cs.CV 2025-06 conditional novelty 6.0 of 10

    MTabVQA is a new visual multi-table question answering benchmark, and fine-tuning VLMs on its instruction set improves their accuracy on it.

  3. Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Table-r1 combines a layout-transformation self-supervised task and a mix-paradigm GRPO stage so 7B/8B models outperform other small-model table reasoners and approach GPT-4o-level accuracy.

  4. TableMind: An Autonomous Programmatic Agent for Tool-Augmented Table Reasoning

    cs.AI 2025-09 conditional novelty 5.0 of 10

    TableMind, a two-stage SFT-plus-RL agent trained on an 8B model, reports state-of-the-art results on three table reasoning benchmarks.

  5. TabReason: A Reinforcement Learning-Enhanced Reasoning LLM for Explainable Tabular Data Prediction

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Applying GRPO reinforcement learning with format and accuracy rewards to a 1.5B LLM yields high weighted F1 on financial tabular benchmarks, but near-zero MCC on imbalanced datasets and unvalidated explanations.

Pith tools