REVIEW 8 cited by
TableBench: A Comprehensive and Complex Benchmark for Table Question Answering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent advancements in Large Language Models (LLMs) have markedly enhanced the interpretation and processing of tabular data, introducing previously unimaginable capabilities. Despite these achievements, LLMs still encounter significant challenges when applied in industrial scenarios, particularly due to the increased complexity of reasoning required with real-world tabular data, underscoring a notable disparity between academic benchmarks and practical applications. To address this discrepancy, we conduct a detailed investigation into the application of tabular data in industrial scenarios and propose a comprehensive and complex benchmark TableBench, including 18 fields within four major categories of table question answering (TableQA) capabilities. Furthermore, we introduce TableLLM, trained on our meticulously constructed training set TableInstruct, achieving comparable performance with GPT-3.5. Massive experiments conducted on TableBench indicate that both open-source and proprietary LLMs still have significant room for improvement to meet real-world demands, where the most advanced model, GPT-4, achieves only a modest score compared to humans.
Forward citations
Cited by 8 Pith papers
-
SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner
SWE-Flow synthesizes incremental, test-driven development tasks from real GitHub projects and shows that fine-tuning Qwen2.5-Coder-32B-Instruct on them improves performance on the resulting SWE-Flow-Bench benchmark.
-
IFEvalCode: Controlled Code Generation
A 1,620-sample, 8-language, Chinese/English benchmark separates code correctness from instruction-following and shows instruction compliance is far lower than correctness across 40+ LLMs.
-
AraTable: Benchmarking LLMs' Reasoning and Understanding of Arabic Tabular Data
AraTable is the first Arabic tabular QA benchmark; its experiments show LLMs are much weaker at reasoning over Arabic tables than at direct lookup.
-
Coding Triangle: How Does Large Language Model Understand Code?
The Coding Triangle framework evaluates LLMs on editorials, code, and test cases, revealing that models are self-consistent yet lack diversity and that model mixtures improve robustness.
-
Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data
A new benchmark, TableEval, with 3017 tables in five formats, shows LLMs are robust to table representation but perform worse on scientific tables, with the caveat that the domain gap is confounded by task difficulty.
-
StructCoh: Structured Contrastive Learning for Context-Aware Text Semantic Matching
StructCoh, a graph-enhanced contrastive learning framework for text semantic matching, reportedly outperforms prior methods on legal and plagiarism benchmarks, but the reported results are not reproducible from the pa...
-
TableZoomer: A Collaborative Agent Framework for Large-scale Table Question Answering
TableZoomer compresses tables into schemas, zooms to query-relevant regions, and executes generated Python to answer questions, lifting Qwen3-8B accuracy by 19.34 points on DataBench and 25 points on TableBench Fact Checking.
-
ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering
ChartMind is a new bilingual chart QA benchmark, and ChartLLM's structured context extraction yields higher scores than three existing prompting paradigms in the paper's evaluations.
Discussion (0). Sign in to comment.