Pith. sign in

REVIEW 3 major objections 7 minor 101 references

Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Two-stage training lifts 8B models to LLM-level table reasoning

desk verdict Solid two-stage method for program-based table reasoning in small LMs; the AIT-QA numbers rest on a nonstandard split and need fixing, but the core on WTQ/TabFact/HiTab holds. read the letter →

arxiv 2506.06137 v1 pith:P6JIRSUA submitted 2025-06-06 cs.LG cs.CL

classification cs.LGcs.CL
keywords tablereasoningsmalllanguagemodelsprogram-basedlayouttransformationinferenceGRPOreinforcementlearningself-supervisedcodegeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that small language models (7B-8B) can be trained to reason over tables by generating executable Python programs, closing most of the gap to much larger models. Its proposed method, Table-r1, first teaches the model to understand varied table layouts through a self-supervised task and then sharpens its code-based reasoning with a reinforcement-learning objective. On WikiTQ, TabFact, HiTab, and AIT-QA, the trained models outperform earlier small-model methods and land near GPT-4o and DeepSeek results, with gains of at least 15 accuracy points over the base LLaMA-8B model. If this holds, high-accuracy table question answering no longer requires a massive deployment model.

What carries the argument

The load-bearing mechanism is Layout Transformation Inference (LTI): a self-supervised task where the model sees an original table t and a transformed table t′, generated by chaining one to three operations such as row swaps, column deletion, or transpose, and must emit an executable program that reconstructs t′ from t. This teaches robust header handling without manual annotation. The second mechanism is mix-paradigm GRPO, an extension of Group Relative Policy Optimization in which each completion can end in a <code_solution> block or a direct <answer>, and rewards guide the policy to prefer programs while retaining text answers as a fallback for layouts too complex to index. Together they target the paper's two stated failure modes: layout heterogeneity and reasoning inconsistency.

What would settle it

Re-run Table-r1 on AIT-QA with the original official split and compare directly with the published TAMA baseline; if the score falls below 89.2 or the margin over the one-shot P-TR baseline narrows sharply, the four-benchmark parity claim loses support. A second check is to retrain without cold-start distillation on one dataset; if the gain collapses, the teacher data rather than the two-stage pipeline is responsible.

Watch

Extended reading notes

Core claim

The central claim is that the table-reasoning gap is not fixed by model size: a purpose-built two-stage pipeline can let an 8B model rival a much larger one. Stage 1, Layout Transformation Inference, generates pairs of original and transformed tables and trains the model to write the transformation program, forcing it to treat headers as structural anchors and generalize across flat, hierarchical, and bi-directional layouts. Stage 2 applies a mix-paradigm variant of GRPO, with rewards for compilation correctness, answer correctness, and format adherence, that lets the model prefer code generation but fall back to direct text answers when a table is too complex to index reliably. On LLaMA-8B the recipe reaches 74.3 on WikiTQ, 84.4 on TabFact, 74.2 on HiTab, and 86.5 on AIT-QA, against one-shot program-based baselines of 55.3, 65.2, 49.2, and 48.8, and the ablations attribute most of the gain to the combination of cold-start distillation, GRPO, and the layout task rather than to any single component.

Load-bearing premise

The four-benchmark parity claim rests on AIT-QA results measured with a newly created random 8:2 question split rather than the benchmark's official split, so table-level overlap between training and test could inflate the reported numbers.

Editorial extensions

If this is right

  • Small-model table reasoning can improve by 15 to 38 accuracy points without changing the model size, which would lower the cost of deploying table question answering.
  • The self-supervised layout task provides a training signal that does not require human annotation, so the method can be applied to new table domains using only unlabeled tables.
  • Reward-based training with executable feedback transfers to table reasoning and adds accuracy on top of supervised fine-tuning, as the ablation study shows.
  • The model learns to prefer code for most questions but switch to direct answers on dense hierarchical layouts, suggesting hybrid strategies can be learned rather than hand-designed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: re-run AIT-QA on the official split; the paper's random 8:2 question split can put questions about the same table in both training and test, and published baselines were not measured on the new split.
  • The LTI task should transfer to other table-to-code problems, such as spreadsheet formula induction or data-cleaning program synthesis, where headers are the main structural challenge.
  • Since input tables are truncated to the first ten rows, the method's advantage on very large tables should be rechecked with full-length inputs before generalizing the claim.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes Table-r1, a two-stage training method for program-based table reasoning (P-TR) with small language models (LLaMA-8B and Qwen-7B). Stage 1 introduces a self-supervised task, Layout Transformation Inference (LTI), in which the model learns to generate programs that reconstruct a table from a layout-modified version; labels are auto-generated and rewritten by the base model to avoid distributional mismatch. Stage 2 applies a mix-paradigm variant of GRPO that rewards executable code answers while allowing text-based fallback, preceded by a cold-start SFT on LLM-distilled reasoning traces. Experiments on WTQ, TabFact, HiTab, and AIT-QA report consistent accuracy gains over the base models and claim competitiveness with LLM baselines, with ablations attributing the gains to LTI, cold start, GRPO, and the mixed paradigm. The paper also includes error analyses, training-difficulty analyses, and a NeurIPS-style limitations/checklist section.

Significance. If validated, the paper makes a useful empirical contribution: it demonstrates that 7B-8B models can substantially narrow the gap to much larger LLMs on table reasoning, it introduces an annotation-free self-supervised task for layout understanding, and it extends GRPO to a new domain with a plausible mixed-paradigm objective. The paper is thorough in its ablations, reports standard deviations on three runs, and includes an honest limitations section covering reward-weight tuning, model scale, and multimodal tables. The main caveat is the AIT-QA evaluation, which relies on a nonstandard random split and is not directly comparable to published baselines; this affects the breadth of the central claim. The overall idea is sound and the evidence for the core pipeline is strong on the other three benchmarks, but the AIT-QA issue must be resolved before the claims can be accepted as stated.

major comments (3)
  1. [§5.1, Table 2 (AIT-QA rows)] The AIT-QA evaluation uses a random 8:2 question-level split instead of the original dataset split, and the paper does not report the random seed, the exact train/test question counts, or any results on the official split. Because AIT-QA contains only 116 tables and 515 questions, a random question-level split is likely to place questions about the same table on both sides of the split; since the cold-start SFT and GRPO stages train on the training questions, the model can learn table-specific layouts and answer-relevant indices, inflating test accuracy through table-level leakage. The published TAMA baseline (89.2) was measured on the original split, so the comparison in Table 2 between Table-r1's 86.5/89.9 and TAMA's 89.2 is not apples-to-apples. The 'competitive with LLMs across four benchmarks' claim rests on this result, so the authors must either evaluate on the official split, provide split-robust evidence (e.g., a table-level split), or explicitly drop/qualify the AIT-QA claim.
  2. [Abstract vs. §5.2/Table 2] The abstract claims that Table-r1 'outperforms all SLM-based methods,' but Table 2 lists TAMA (LLaMA3.1-8B-Inst, T-TR) at 89.2 on AIT-QA, which is higher than Table-r1's reported 86.5 for LLaMA-8B. Even if the split difference explains the discrepancy, the paper does not acknowledge this in the text; as presented, the claim is internally inconsistent. The authors should reconcile the claim with the table, for example by reporting the official-split result for AIT-QA or by qualifying the sentence to 'outperforms all SLM-based methods on the benchmarks with standard evaluation splits' and stating the AIT-QA caveat explicitly.
  3. [§5.4, Table 3 (w/o all modules row)] The row labeled 'Table-r1 (w/o all modules)' is not an ablation of the trained pipeline; it is the untrained base model's one-shot P-TR performance, numerically identical to the E2E one-shot rows in Table 2. Labeling this row as 'w/o all modules' conflates removing all training modules with not performing any training, which can overstate the contribution of the individual modules when a reader compares the row to the other ablations. The individual ablations (-SSL, -Cold Start, -GRPO, -Mix) are informative and should remain, but this row should be renamed 'Base model' and the discussion in §5.4 should clarify that it is not an ablation of Table-r1.
minor comments (7)
  1. [§5.1] If the random AIT-QA split is kept, the paper should specify the random seed, the resulting train/test question counts, and the number of tables that appear in both splits, so that readers can assess the leakage risk.
  2. [§2.2 / References] The sentence 'The impressive performance of o1 [57]' cites reference [57], which is Marco-o1 rather than OpenAI o1; the citation should point to the appropriate source, e.g., [15] or a direct o1 reference.
  3. [References] References [55] and [56] are duplicate entries for 'Large language models are complex table parsers'; one entry should be removed or the two references should be differentiated.
  4. [Footnote 2 / Appx. C] The source-code URL in the footnote is not live, and the supplementary material referenced in the NeurIPS checklist is not present in the submitted text; provide a working link or a detailed appendix to support reproducibility.
  5. [Table 2 caption] The caption states that the E2E one-shot series share the same prompts but does not clarify whether the LLM baselines (GPT-4o, DeepSeek-v3, DeepSeek-r1) also use SC=5; this should be stated explicitly.
  6. [Table 3 header] The column header 'AiT-QA' is a typo; it should read 'AIT-QA'.
  7. [Appx. C] The implementation section says 'We train for 1 epoch' but also 'The model achieves its best performance at around 400 steps'; with the stated batch sizes and up to 5,000 distilled samples, 400 steps is more than one epoch, so clarify the training schedule or the effective number of epochs.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; training and evaluation signals are externally grounded.

full rationale

The paper's central claim that Table-r1 surpasses SLM baselines and approaches LLM performance is supported by accuracy measured against golden answers on external benchmarks (WTQ, TabFact, HiTab, AIT-QA) in Table 2. The RL stage uses rewards computed by executing generated code and comparing outputs with golden answers (Section 4.2.3, Eq. 7), so the optimization signal is not derived from the model's own predictions. The Stage-1 LTI task generates target tables via deterministic operations O and validates labels through a program executor (Eqs. 3-4), making the self-supervised supervision external to the downstream evaluation. Cold-start distillation uses a DeepSeek-v3 teacher on the training split only (Section 4.2.2, Appendix C), and GRPO optimizes with execution-derived rewards; no test-set answer is used as a training signal. The AIT-QA random 8:2 question-level split (Section 5.1) raises a legitimate benchmark-comparability concern, but it is not a circularity: the reported 86.5/89.9 values are still held-out accuracies against golden answers, not quantities defined by the fitting procedure. The paper's self-citations ([17], [18] by the first author) are ordinary related-work references and are not load-bearing for the method's derivation. No step was found where a prediction is defined in terms of a fitted input or where a load-bearing premise reduces to a self-citation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new entities such as particles, forces, dimensions, or conserved quantities are introduced. The free parameters are hand-chosen training choices, reward weights, the nonstandard AIT-QA split, and distillation scale, all of which affect the quantitative claims.

free parameters (5)
  • Reward function weights = strict_format 0.75; ans 1.5; comment_ratio 0.45; multiple_python_blocks_penalty 1.0; compilation 0.75…
    Max rewards in Table 6 are hand-chosen; Appx A.3 admits reward weighting is heavily reliant on empirical tuning.
  • AIT-QA train/test split = random 8:2 by question
    Section 5.1 introduces a random question-level split instead of the original dataset split, which may affect comparability and leak table-level information.
  • LTI operation sequence length = n sampled from 1 to 3
    Section 4.1.1 chooses a random number of operations per synthetic example; the range is a design choice.
  • Distilled cold-start sample count = up to 5,000 per dataset
    Appx C; no sensitivity analysis, and this component provides the largest accuracy gains.
  • LoRA ranks and RL hyperparameters = SFT rank 32, RL rank 64, temperature 0.85, batch 4x8x8, max tokens 2400, 1 epoch
    Appx C; chosen by hand without reported sensitivity analysis.
assumptions (4)
  • domain assumption Synthetic layout transformations (row/column swaps, deletions, transposes) learn skills that transfer to real heterogeneous table headers.
    Stage 1 relies on this transfer; only indirect evidence from ablations (Table 3, 4).
  • domain assumption Executing Python/pandas code on the full table is a faithful executor for benchmark answers.
    P-TR paradigm assumes E(p,t) matches expected answer semantics; standard in the field.
  • ad hoc to paper A random 8:2 question-level split of AIT-QA produces a valid and comparable evaluation.
    Section 5.1; not the original split, and same-table questions can appear on both sides.
  • domain assumption Text-based completions are valid unless proven otherwise in reward computation.
    Section 4.2.3 assigns default validity to text outputs, introducing a bias the authors say is counteracted by other reward terms.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models." pith.science (2026). https://pith.science/paper/P6JIRSUA

@misc{pith2026250606137,
  author       = {Pith},
  title        = {Pith review of: Table-r1: Self-supervised and Reinforcement Learning for Program-based Table Reasoning in Small Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P6JIRSUA}},
  note         = {Machine review of arXiv:2506.06137}
}
read the original abstract

Table reasoning (TR) requires structured reasoning over semi-structured tabular data and remains challenging, particularly for small language models (SLMs, e.g., LLaMA-8B) due to their limited capacity compared to large LMs (LLMs, e.g., GPT-4o). To narrow this gap, we explore program-based TR (P-TR), which circumvents key limitations of text-based TR (T-TR), notably in numerical reasoning, by generating executable programs. However, applying P-TR to SLMs introduces two challenges: (i) vulnerability to heterogeneity in table layouts, and (ii) inconsistency in reasoning due to limited code generation capability. We propose Table-r1, a two-stage P-TR method designed for SLMs. Stage 1 introduces an innovative self-supervised learning task, Layout Transformation Inference, to improve tabular layout generalization from a programmatic view. Stage 2 adopts a mix-paradigm variant of Group Relative Policy Optimization, enhancing P-TR consistency while allowing dynamic fallback to T-TR when needed. Experiments on four TR benchmarks demonstrate that Table-r1 outperforms all SLM-based methods, achieving at least a 15% accuracy improvement over the base model (LLaMA-8B) across all datasets and reaching performance competitive with LLMs.

Figures

Figures reproduced from arXiv: 2506.06137 by the authors.

Figure 1
Figure 1. (a) Two TR paradigms: the text-based directly generates answers, while the program-based [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of Table-r1. Stage 1 introduces an SSL task to improve layout understanding, with annotations auto-generated (bottom-left). Stage 2 applies RL, starting from teacher-guided cold start and followed by a mix-paradigm GRPO, enabling dynamic strategy selection. Reward functions provide numerical feedback based on outputs. external executor E to obtain the final answer: p = M(q, t), a′ = E(p, t). (2) In our meth… view at source ↗
Figure 3
Figure 3. Error type distribution. To analyze the limitations of the two paradigms (RQ1), we manually inspect 200 instance out￾puts each from WTQ and HiTab using Qwen2.5- 7B-Instruct (text-based) and Qwen2.5-Coder￾7B-Instruct (program-based). The program￾based model is restricted to the first 10 rows due to token limits, while the text-based model processes the entire table (truncated if exceed￾7 [PITH_FULL_IMAGE:figures/ful… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Comparison of SC=5 and Pass@16 accuracy across five model variants to assess the impact [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Training curves of Qwen-7B (after distillation) on the WTQ dataset sorted by difficulty [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Prompts for T-TR. E.2 Prompts for P-TR E.3 Prompts for the Mix-paradigm GRPO 16 [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Prompts for P-TR. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Prompts for the Mix-paradigm GRPO. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: In P-TR, errors occur due to misunderstandings of the table’s semantics. Specifically, the [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: In T-TR, errors arise from information extraction errors of the table. Specifically, the [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: In P-TR, errors occur when the generated code solution lacks thorough consideration. [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: An example where the P-TR method fails but the T-TR method succeeds, due to the table [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

101 extracted references · 41 canonical work pages

  1. [1]

    Large language models for mathematical reasoning: Progresses and challenges

    Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. In Neele Falk, Sara Papi, and Mike Zhang (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop , pp. 225–237, St. Julia...

  2. [2]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-V oss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Li...

  3. [3]

    Tablemaster: A recipe to advance table understanding with language models

    Lang Cao. Tablemaster: A recipe to advance table understanding with language models. arXiv preprint arXiv:2501.19378, 2025

  4. [4]

    Large language models are few(1)-shot table reasoners

    Wenhu Chen. Large language models are few(1)-shot table reasoners. ArXiv, abs/2210.06710,

  5. [5]

    Tabfact: A large-scale dataset for table-based fact verification

    Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. Tabfact: A large-scale dataset for table-based fact verification. In International Conference on Learning Representations, 2020

  6. [6]

    Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588, 2022

  7. [7]

    Hitab: A hierarchical table dataset for question answering and natural language generation

    Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. Hitab: A hierarchical table dataset for question answering and natural language generation. ArXiv, abs/2108.06712, 2021. URL https://api.semanticscholar. org/CorpusID:237091377

  8. [8]

    Nadkarni, Yushi Hu, Caiming Xiong, Dragomir R

    Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, R.K. Nadkarni, Yushi Hu, Caiming Xiong, Dragomir R. Radev, Marilyn Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. Binding language models in symbolic languages. ArXiv, abs/2210.02875, 2022. URL https://api.semanticscholar.org/CorpusID:252734772. 10

Show all 101 references
  1. [9]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei L...

  2. [10]

    Rethinking table instruction tuning

    Naihao Deng and Rada Mihalcea. Rethinking table instruction tuning. arXiv preprint arXiv:2501.14693, 2025

  3. [11]

    Pal: Program-aided language models

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. In International Conference on Machine Learning, pp. 10764–10799. PMLR, 2023

  4. [12]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

  5. [13]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

  6. [14]

    Tablelora: Low-rank adaptation on table structure understanding for large language models

    Xinyi He, Yihao Liu, Mengyu Zhou, Yeye He, Haoyu Dong, Shi Han, Zejian Yuan, and Dongmei Zhang. Tablelora: Low-rank adaptation on table structure understanding for large language models. arXiv preprint arXiv:2503.04396, 2025

  7. [15]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

  8. [16]

    Tree-of-table: Unleashing the power of llms for enhanced large-scale table understanding

    Deyi Ji, Lanyun Zhu, Siqi Gao, Peng Xu, Hongtao Lu, Jieping Ye, and Feng Zhao. Tree-of-table: Unleashing the power of llms for enhanced large-scale table understanding. arXiv preprint arXiv:2411.08516, 2024

  9. [17]

    Tabprompt: Graph- based pre-training and prompting for few-shot table understanding

    Rihui Jin, Jianan Wang, Wei Tan, Yongrui Chen, Guilin Qi, and Wang Hao. Tabprompt: Graph- based pre-training and prompting for few-shot table understanding. In Conference on Empirical Methods in Natural Language Processing , 2023. URL https://api.semanticscholar. org/CorpusID:...

  10. [18]

    Hegta: Leveraging heterogeneous graph-enhanced large language models for few-shot complex table understanding

    Rihui Jin, Yu Li, Guilin Qi, Nan Hu, Yuan-Fang Li, Jiaoyan Chen, Jianan Wang, Yongrui Chen, Dehai Min, and Sheng Bi. Hegta: Leveraging heterogeneous graph-enhanced large language models for few-shot complex table understanding. In Proceedings of the AAAI Conference on Artifici...

  11. [19]

    Ait-qa: Question answering dataset over complex tables in the airline industry

    Yannis Katsis, Saneem Chemmengath, Vishwajeet Kumar, Samarth Bharadwaj, Mustafa Canim, Michael Glass, Alfio Gliozzo, Feifei Pan, Jaydeep Sen, Karthik Sankaranarayanan, et al. Ait-qa: Question answering dataset over complex tables in the airline industry. NAACL-HLT 2022, pp. 305, 2022

  12. [20]

    Table-gpt: Table-tuned gpt for diverse table tasks

    Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. Table-gpt: Table-tuned gpt for diverse table tasks. arXiv preprint arXiv:2310.09263, 2023

  13. [21]

    GraphOT- TER: Evolving LLM-based graph reasoning for complex table question answering

    Qianlong Li, Chen Huang, Shuai Li, Yuanxin Xiang, Deng Xiong, and Wenqiang Lei. GraphOT- TER: Evolving LLM-based graph reasoning for complex table question answering. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert (e...

  14. [22]

    Rethinking tabular data understanding with large language models

    Tianyang Liu, Fei Wang, and Muhao Chen. Rethinking tabular data understanding with large language models. arXiv preprint arXiv:2312.16702, 2023

  15. [23]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783, 2025

  16. [24]

    Large language model for table processing: A survey

    Weizheng Lu, Jiaming Zhang, Jing Zhang, and Yueguo Chen. Large language model for table processing: A survey. arXiv preprint arXiv:2402.05121, 2024

  17. [25]

    Tart: An open- source tool-augmented framework for explainable table-based reasoning

    Xinyuan Lu, Liangming Pan, Yubo Ma, Preslav Nakov, and Min-Yen Kan. Tart: An open- source tool-augmented framework for explainable table-based reasoning. arXiv preprint arXiv:2409.11724, 2024

  18. [26]

    Potable: Programming standardly on table-based reasoning like a human analyst

    Qingyang Mao, Qi Liu, Zhi Li, Mingyue Cheng, Zheng Zhang, and Rui Li. Potable: Programming standardly on table-based reasoning like a human analyst. arXiv preprint arXiv:2412.04272, 2024

  19. [27]

    Show your work: Scratchpads for intermediate computation with language models

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114, 2021

  20. [28]

    Compositional semantic parsing on semi-structured tables

    Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In Annual Meeting of the Association for Computational Linguistics , 2015. URL https://api.semanticscholar.org/CorpusID:9027681

  21. [29]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024

  22. [30]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

  23. [31]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

  24. [32]

    Towards reasoning ability of small language models

    Gaurav Srivastava, Shuxiang Cao, and Xuan Wang. Towards reasoning ability of small language models. arXiv preprint arXiv:2502.11569, 2025. 12

  25. [33]

    Tablegpt2: A large multimodal model with tabular data integration

    Aofeng Su, Aowen Wang, Chao Ye, Chen Zhou, Ga Zhang, Gang Chen, Guangcheng Zhu, Haobo Wang, Haokai Xu, Hao Chen, et al. Tablegpt2: A large multimodal model with tabular data integration. arXiv preprint arXiv:2411.02059, 2024

  26. [34]

    Table meets llm: Can large language models understand structured table data? a benchmark and empirical study

    Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang. Table meets llm: Can large language models understand structured table data? a benchmark and empirical study. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pp. 645–654, 2024

  27. [35]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

  28. [36]

    Trove: Inducing verifiable and efficient toolboxes for solving programmatic tasks

    Zhiruo Wang, Daniel Fried, and Graham Neubig. Trove: Inducing verifiable and efficient toolboxes for solving programmatic tasks. arXiv preprint arXiv:2401.12869, 2024

  29. [37]

    Chain-of-table: Evolving tables in the reasoning chain for table understanding.arXiv preprint arXiv:2401.04398, 2024

    Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, et al. Chain-of-table: Evolving tables in the reasoning chain for table understanding.arXiv preprint arXiv:2401.04398, 2024

  30. [38]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

  31. [39]

    Continual learning for large language models: A survey

    Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: A survey. arXiv preprint arXiv:2402.01364, 2024

  32. [40]

    Table-llm-specialist: Language model specialists for tables using iterative generator- validator fine-tuning

    Junjie Xing, Yeye He, Mengyu Zhou, Haoyu Dong, Shi Han, Dongmei Zhang, and Surajit Chaudhuri. Table-llm-specialist: Language model specialists for tables using iterative generator- validator fine-tuning. arXiv preprint arXiv:2410.12164, 2024

  33. [41]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

  34. [42]

    Self-distillation bridges distribution gap in language model fine-tuning

    Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. Self-distillation bridges distribution gap in language model fine-tuning. arXiv preprint arXiv:2402.13669, 2024

  35. [43]

    Triples as the key: Structuring makes decomposition and verification easier in llm-based tableqa

    Zhen Yang, Ziwei Du, Minghan Zhang, Wei Du, Jie Chen, Zhen Duan, and Shu Zhao. Triples as the key: Structuring makes decomposition and verification easier in llm-based tableqa. In The Thirteenth International Conference on Learning Representations, 2025

  36. [44]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023

  37. [45]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in Neural Information Processing Systems, 36, 2024

  38. [46]

    Large language models are versatile decomposers: Decomposing evidence and questions for table-based rea- soning

    Yunhu Ye, Binyuan Hui, Min Yang, Binhua Li, Fei Huang, and Yongbin Li. Large language models are versatile decomposers: Decomposing evidence and questions for table-based rea- soning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in ...

  39. [47]

    Table-critic: A multi-agent framework for collaborative criticism and refinement in table reasoning

    Peiying Yu, Guoxin Chen, and Jingjing Wang. Table-critic: A multi-agent framework for collaborative criticism and refinement in table reasoning. arXiv preprint arXiv:2502.11799, 2025. 13

  40. [48]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

    Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837, 2025

  41. [49]

    Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild

    Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild. arXiv preprint arXiv:2503.18892, 2025

  42. [50]

    Alter: Augmentation for large-table-based reasoning

    Han Zhang, Yuheng Ma, and Hanfang Yang. Alter: Augmentation for large-table-based reasoning. arXiv preprint arXiv:2407.03061, 2024

  43. [51]

    Tablellama: Towards open large generalist models for tables

    Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. Tablellama: Towards open large generalist models for tables. ArXiv, abs/2311.09206, 2023. URL https://api.semanticscholar. org/CorpusID:265213406

  44. [52]

    A survey of table reasoning with large language models

    Xuanliang Zhang, Dingzirui Wang, Longxu Dou, Qingfu Zhu, and Wanxiang Che. A survey of table reasoning with large language models. arXiv preprint arXiv:2402.08259, 2024

  45. [53]

    Yunjia Zhang, Jordan Henkel, Avrilia Floratou, Joyce Cahoon, Shaleen Deep, and Jignesh M. Patel. Reactable: Enhancing react for table question answering. ArXiv, abs/2310.00815, 2023

  46. [54]

    e5: Zero-shot hierarchical table analysis using augmented llms via explain, extract, execute, exhibit and extrapolate

    Zhehao Zhang, Yan Gao, and Jian-Guang Lou. e5: Zero-shot hierarchical table analysis using augmented llms via explain, extract, execute, exhibit and extrapolate. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics...

  47. [56]

    Large language models are complex table parsers

    Bowen Zhao, Changkai Ji, Yuejie Zhang, Wen He, Yingwen Wang, Qing Wang, Rui Feng, and Xiaobo Zhang. Large language models are complex table parsers. arXiv preprint arXiv:2312.11521, 2023

  48. [57]

    Marco-o1: Towards open reasoning models for open-ended solutions

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-o1: Towards open reasoning models for open-ended solutions. arXiv preprint arXiv:2411.14405, 2024

  49. [58]

    Structlm: Towards building generalist models for structured knowledge grounding

    Alex Zhuang, Ge Zhang, Tianyu Zheng, Xinrun Du, Junjie Wang, Weiming Ren, Stephen W Huang, Jie Fu, Xiang Yue, and Wenhu Chen. Structlm: Towards building generalist models for structured knowledge grounding. arXiv preprint arXiv:2402.16671, 2024. 14 A Limitations

  50. [61]

    Although our method has been evaluated on four popular public benchmarks—covering various types of semi-structured tables (e.g., WTQ, TabFact)—it may not generalize well to other types of tables, such as multimodal tables

  51. [62]

    The SLMs used in this study refer to 7B or 8B-scale language models, and the proposed approach may not be directly applicable to smaller models such as 3B or 0.5B LMs

  52. [63]

    Year\n(int64)

    Currently, the reward weighting for different functions in reinforcement learning still resembles neural network hyperparameters, heavily reliant on empirical tuning and lacking automation. B Distinction Between Table Reasoning and Text-to-SQL While TR and Text-to-SQL (T2SQL) ...

  53. [64]

    REASONING PHASE: <reasoning>

  54. [65]

    Task Analysis: - Identify required columns/rows from the question - Note any needed operations (filtering, aggregation, etc.)

  55. [66]

    First locate (...) then calculate (...) finally compare (...)

    Planning: - Draft pseudocode or step-by-step logic - Example: "First locate (...) then calculate (...) finally compare (...)"

  56. [67]

    This approach works because (...)

    Validation: - Verify assumptions about data structure - Cross-check with table preview - Reflect: "This approach works because (...)"

  57. [68]

    Decision: - [✓] Code solution needed / [✓] Direct answer possible </reasoning>

  58. [69]

    </code_solution> === OPTION B: Direct Answer === <answer> Final Answer: [Concise statement without question repetition] </answer> === Critical Reminders ===

    EXECUTION PHASE: [CHOOSE ONE] === OPTION A: Code-Based Solution === <code_solution> python # Comment each step result = df.loc # Example print(result) # output the final answer. </code_solution> === OPTION B: Direct Answer === <answer> Final Answer: [Concise statement without ...

  59. [70]

    Always use tuple indexing: df.loc[row_tuple, col_tuple]

  60. [71]

    Never assume single-level headers/indices

  61. [72]

    Validate against edge cases (missing hierarchies, etc.)""" • Prompts (Mix) Figure 8: Prompts for the Mix-paradigm GRPO. 18 F Case Studies Input: Question: the mean depreciation rate for these selected machinery and equipment categories as a whole, weighted by their chained dol...

  62. [73]

    This value is located in the row `('rate', 'mean, all assets', '')` and the column `('2002 to 2010',)`

    Identify the mean depreciation rate for the period 2002 to 2010. This value is located in the row `('rate', 'mean, all assets', '')` and the column `('2002 to 2010',)`

  63. [74]

    This value is located in the row `('rate', 'mean, all assets', '')` and the column `('1985 to 2010',)`

    Identify the mean depreciation rate for the period 2002 to 2010. This value is located in the row `('rate', 'mean, all assets', '')` and the column `('1985 to 2010',)`

  64. [75]

    Calculate the difference between the two rates

  65. [76]

    Laps Led

    Calculate the percentage increase by dividing the difference by the 1985 to 2010 rate and then multiplying by 100. The percentage increase is given by the formula: \[ \text{Percentage Increase} = \left( \frac{\text{2002 to 2010 rate} - \text{1985 to 2010 rate}}{\text{1985 to 2...

  66. [77]

    **Sorting the Data**: We need to sort the DataFrame by the "Date" column to ensure that the earliest date is at the top

  67. [78]

    **Identifying the First Goal**: Once the data is sorted, the first row will contain the earliest date, and we can then extract the corresponding "Venue" from this row. **Code Solution**: ```python # Sort the DataFrame by the 'Date' column df_sorted = df.sort_values(by='Date') ...

  68. [79]

    - Determine the percentage they plan to claim a tax credit

    **Task Analysis**: - Identify the relevant data: donors between 35 and 54 years old whose household income was in the highest quintile. - Determine the percentage they plan to claim a tax credit. - Calculate the percentage points difference from the total percentage

  69. [80]

    - Find the column for 'top two quintiles' under the 'household income quintiles' level

    **Planning**: - Locate the row for age group '35 to 54 years' under the 'percentage' level. - Find the column for 'top two quintiles' under the 'household income quintiles' level. - Extract the percentage value from the DataFrame. - Calculate the difference from the total perc...

  70. [81]

    - Ensure the DataFrame structure and indexing are as expected

    **Validation**: - Verify that the row and column tuples are correct based on the table preview. - Ensure the DataFrame structure and indexing are as expected

  71. [82]

    **Decision**: - Code solution needed. </reasoning> <code_solution> ```python # Extract the percentage for donors between 35 and 54 years whose household income was in the highest quintile percentage = df.loc[('percentage', 'age group', '35 to 54 years'), ('household income qui...

  72. [83]

    **Task Analysis**: - Identify the relevant row: ('percentage', 'age group', '35 to 54 years') - Identify the relevant column: ('household income quintiles', 'top two quintiles') - Extract the value from the DataFrame at this location

  73. [84]

    - Extract the value from the DataFrame

    **Planning**: - Use tuple indexing to locate the specified row and column. - Extract the value from the DataFrame. - Since the question asks for the percentage points, we already have the required percentage directly from the DataFrame

  74. [85]

    This approach works because the DataFrame structure matches the expected hierarchical indices and the required data is directly accessible

    **Validation**: - Verify that the DataFrame has the expected structure with hierarchical indices. - Cross-check the table preview to ensure the row and column exist. - Reflect: "This approach works because the DataFrame structure matches the expected hierarchical indices and t...

  75. [86]

    **Decision**: - Direct answer possible </reasoning> <answer> Final Answer: 64.0 </answer> Golden Answer: 64.0 • Incorrect P-TR vs. correct T-TR on identical instance in HiTab Figure 12: An example where the P-TR method fails but the T-TR method succeeds, due to the table layou...

  76. [87]

    Guidelines: • The answer NA means that the abstract and introduction do not include the claims made in the paper

    Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope? Answer: [Yes] Justification: See Abstract and Section 1. Guidelines: • The answer NA means that the abstract and introduction do not include the cl...

  77. [88]

    Limitations

    Limitations Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: See Appx. A. Guidelines: • The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those a...

  78. [89]

    23 Guidelines: • The answer NA means that the paper does not include theoretical results

    Theory assumptions and proofs Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA] Justification: The paper does not include theoretical results. 23 Guidelines: • The answer NA means that the...

  79. [90]

    Experimental result reproducibility Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data...

  80. [91]

    Guidelines: • The answer NA means that paper does not include experiments requiring code

    Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: We provide code along with su...

  81. [92]

    C and supplemental material

    Experimental setting/details Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results? Answer: [Yes] Justification: See Section 5.1, Appx. C and su...

  82. [93]

    Guidelines: • The answer NA means that the paper does not include experiments

    Experiment statistical significance Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Standard deviation in Table 2. Guidelines: • The a...

  83. [94]

    Experiments compute resources Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experi- ments? Answer: [Yes] Justification: See Appx. C. Guidelines...

  84. [95]

    Guidelines: • The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics

    Code of ethics Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? Answer: [Yes] Justification: The research conducted in the paper conform, in every respect, with the NeurIPS...

  85. [96]

    Guidelines: • The answer NA means that there is no societal impact of the work performed

    Broader impacts Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed? Answer: [NA] Justification: There is no societal impact of the work performed. Guidelines: • The answer NA means that there is no soci...

  86. [97]

    Guidelines: • The answer NA means that the paper poses no such risks

    Safeguards Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses...

  87. [98]

    Guidelines: • The answer NA means that the paper does not use existing assets

    Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: See Section 5.1....

  88. [99]

    Guidelines: • The answer NA means that the paper does not release new assets

    New assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: The paper does not release new assets. Guidelines: • The answer NA means that the paper does not release new assets. ...

  89. [100]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Crowdsourcing and research with human subjects Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? A...

  90. [101]

    Guidelines: • The answer NA means that the paper does not involve crowdsourcing nor research with human subjects

    Institutional review board (IRB) approvals or equivalent for research with human subjects Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals...

  91. [102]

    Answer: [Yes] Justification: LLMs for reasoning and distillation

    Declaration of LLM usage Question: Does the paper describe the usage of LLMs if it is an important, original, or non- standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the...

  92. [2022]

    URL https://api.semanticscholar.org/CorpusID:252872943

  93. [2024]

    URL https://arxiv.org/abs/2412.19437

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.