Pith. sign in

REVIEW 17 cited by

TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2105.07624 v2 pith:YU4ATALK submitted 2021-05-17 cs.CL cs.AI

Fengbin Zhu , Wenqiang Lei , Youcheng Huang , Chao Wang , Shuo Zhang , Jiancheng Lv , Fuli Feng , Tat-Seng Chua This is my paper
classification cs.CLcs.AI
keywords datahybridtat-qareasoningtabulartextualansweranswering
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Hybrid data combining both tabular and textual content (e.g., financial reports) are quite pervasive in the real world. However, Question Answering (QA) over such hybrid data is largely neglected in existing research. In this work, we extract samples from real financial reports to build a new large-scale QA dataset containing both Tabular And Textual data, named TAT-QA, where numerical reasoning is usually required to infer the answer, such as addition, subtraction, multiplication, division, counting, comparison/sorting, and the compositions. We further propose a novel QA model termed TAGOP, which is capable of reasoning over both tables and text. It adopts sequence tagging to extract relevant cells from the table along with relevant spans from the text to infer their semantics, and then applies symbolic reasoning over them with a set of aggregation operators to arrive at the final answer. TAGOPachieves 58.0% inF1, which is an 11.1% absolute increase over the previous best baseline model, according to our experiments on TAT-QA. But this result still lags far behind performance of expert human, i.e.90.8% in F1. It is demonstrated that our TAT-QA is very challenging and can serve as a benchmark for training and testing powerful QA models that address hybrid form data.

Discussion (0). Sign in to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use

    cs.AI 2026-03 conditional novelty 6.0 of 10

    FinToolBench couples 760 executable financial APIs with 295 tool-required queries and scores agents on execution success plus timeliness, intent, and domain compliance, with a finance-aware retrieval baseline (FATR).

  2. AlphaForgeBench: Benchmarking End-to-End Trading Strategy Design with Large Language Models

    q-fin.TR 2026-02 conditional novelty 6.0 of 10

    LLMs are unreliable when asked to emit buy/sell/hold actions, so this paper benchmarks them as code-writing quantitative researchers whose generated strategies are backtested deterministically.

  3. Efficacy of AI RAG Tools for Complex Information Extraction and Data Annotation Tasks: A Case Study Using Banks Public Disclosures

    cs.AI 2025-07 conditional novelty 6.0 of 10

    In a small three-person case study, a RAG tool cut bank-document annotation time by up to 90 percent, while interactive AI use showed higher agreement with an expert benchmark, though the interactive condition reused ...

  4. Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A grammar-guided evolutionary search that composes prompt edits outperformed PromptWizard, OPRO, and RL-Prompt on small LLMs across four domain-specific tasks.

  5. CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Multimodal context-learning benchmark CLBench-V separates grounding, information application, and knowledge acquisition; the best evaluated model scores 0.2847.

  6. When Tables Go Crazy: Evaluating Multimodal Models on French Financial Documents

    cs.CL 2026-02 conditional novelty 5.0 of 10

    A new French financial document benchmark shows vision-language models are strong at text/table extraction but brittle on charts and multi-turn dialogue, with accuracy converging near 50% in conversational settings.

  7. CF-RAG: A Dataset and Method for Carbon Footprint QA Using Retrieval-Augmented Generation

    cs.CL 2025-08 conditional novelty 5.0 of 10

    A fine-tuned Llama 3 model with a trained document critic and program-based reasoning beats GPT-4o and other baselines on a new carbon footprint QA benchmark.

  8. FRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Models

    cs.CL 2025-07 conditional novelty 5.0 of 10

    Fine-tuning small language models on synthetic financial errors yields high detection and editing scores, but the evaluation is limited to synthetic data from the same pipeline.

  9. Unlocking Speech Instruction Data Potential with Query Rewriting

    cs.AI 2025-07 conditional novelty 5.0 of 10

    A multi-LLM rewriting and multi-agent validation pipeline makes text-to-speech synthesized speech instruction data far more usable and improves downstream speech instruction following.

  10. What to Keep and What to Drop: Adaptive Table Filtering Framework

    cs.CL 2025-06 conditional novelty 5.0 of 10

    ATF prunes table columns and rows with LLM scoring plus retrieval, cutting cells by about 70% and improving out-of-domain TableQA accuracy, while hurting in-domain QA and fact verification.

  11. Multimodal Tabular Reasoning with Privileged Structured Information

    cs.LG 2025-06 conditional novelty 5.0 of 10

    An 8B multimodal LLM trained on 9k reasoning traces distilled from structured tables reaches state-of-the-art open-source accuracy on table-image question answering and fact verification.

  12. MMTABREAL: Real-World Benchmark for Multimodal Table Understanding

    cs.CV 2025-05 conditional novelty 5.0 of 10

    The paper releases a 500-table, 4,021-question benchmark of real-world multimodal tables and shows that leading vision-language models drop 20-40% in accuracy relative to earlier synthetic benchmarks.

  13. Realistic Evaluation of TabPFN v2 in Open Environments

    cs.LG 2025-05 conditional novelty 5.0 of 10

    TabPFN v2 underperforms tree-based models on most open-environment tabular tasks and is only preferable on small, covariate-shifted, class-balanced data.

  14. Hierarchical Reranking for Scalable Financial RAG System

    cs.IR 2026-07 reject novelty 4.0 of 10

    A finance-specific RAG pipeline combining table-to-JSON conversion, two-stage reranking, and long-context split-fusion reports NDCG@20=0.7918 and second place in the ICAIF '24 FinanceRAG challenge.

  15. On the Comprehensibility of Multi-structured Financial Documents using LLMs and Pre-processing Tools

    cs.IR 2025-06 conditional novelty 4.0 of 10

    Preprocessing financial PDFs into text, tables, and chart data with existing tools improves LLM question-answering accuracy over direct GPT-4o image input in a small private evaluation.

  16. Learning Sparsity for Effective and Efficient Music Performance Question Answering

    cs.SD 2025-06 conditional novelty 4.0 of 10

    Sparsify reports state-of-the-art accuracy on Music AVQA benchmarks by borrowing three existing sparsification techniques, cutting training time by 28% and retaining 70-80% of accuracy on a 25% data subset.

  17. PHLoRA: data-free Post-hoc Low-Rank Adapter extraction from full-rank checkpoint

    cs.LG 2025-09 conditional novelty 3.0 of 10

    PHLoRA extracts LoRA-compatible adapters from full-rank fine-tuned models via truncated SVD of the weight delta, matching full-rank performance on several benchmarks with no gradients or training data.

Pith tools