Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

MATATA: Weakly Supervised End-to-End MAthematical Tool-Augmented Reasoning for Tabular Applications

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MATATA uses only the final answer as the training signal for every planning step and tool call, and reports that small open language models trained this way match or beat much larger and more heavily annotated systems on FinQA, TAT-QA…

desk verdict Genuinely useful weak-supervision recipe for tool-augmented tabular agents, but the per-tool label-propagation assumption needs evidence. read the letter →

arxiv 2411.18915 v5 pith:EV5CM2VV submitted 2024-11-28 cs.LG cs.CL

Vishnou Vinayagame , Gregory Senay , Luis Martí This is my paper · ORCID
classification cs.LGcs.CL
keywords weaksupervisiontool-augmentedreasoningtabularmathematicalsmalllanguagemodelsinstructiontuningKTOpreferenceoptimizationdocumentunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a multi-step, tool-using reasoning agent for tabular documents can be trained end-to-end with nothing but the final answer as supervision. MATATA starts from a lightly prompt-engineered model, keeps only the reasoning trajectories that happen to end in the right answer, fine-tunes the planner and each tool on those trajectories, then applies KTO preference optimization using both right and wrong trajectories. On FinQA, TAT-QA, and TabMWP, 3.8B and 8B open models trained this way reach scores that rival or beat frameworks built on GPT-4 teachers or human step-by-step annotations. If true, the result removes the main bottleneck to deploying capable tabular-reasoning agents: expensive intermediate labels.

What carries the argument

The load-bearing object is the trajectory $T^r_d$, a sequence of tool calls emitted by the planner; each tool's LoRA adapter is trained on the input-output pairs $\langle \text{input}_i, \text{output}_i\rangle$ collected from those trajectories, with the trajectory's final correctness as the only label. KTO turns this single final-answer signal into a per-trajectory preference signal, letting the model learn from failed trajectories as well as successes. The machinery also includes a shared tool set (row and column extractors, context extractor, program generator and executor, scale finder, answer extractor, and related tools) that lets one adapter be trained on examples drawn from several datasets at once.

What would settle it

Manually annotate the intermediate tool outputs on a random sample of validation trajectories and measure how often a tool call inside a correct-answer trajectory is wrong or a tool call inside a wrong-answer trajectory is right; if either is frequent, the weak label is not actually supervising individual tools. A direct ablation is to train the same two-stage pipeline with oracle step-level labels and check whether accuracy rises substantially.

Watch

Extended reading notes

Core claim

The paper's central claim is that a trajectory's final-answer correctness is a sufficient label for training every component of a tool-augmented reasoner. A MATATA instance consists of a planner $P$ and a set of tools $\{T_i\}$; the planner outputs a trajectory $T^r = \langle T_{\text{start}}, \ldots, T_{\text{end}}\rangle$, and the tools are composed to produce the predicted answer $\hat{a}$. Training data are formed by running the model on dataset instances and splitting trajectories only by whether $\hat{a}$ equals the gold answer $a_d$. In the instruction-tuning stage the correct trajectories are used; in the KTO stage both correct and incorrect trajectories get binary labels $\pm 1$ for the same planner and tool inputs. The paper reports that this weak signal yields FinQA accuracy 77.59, TAT-QA exact match 77.6, and TabMWP accuracy 98.13 on an 8B model, and that the two-stage gains are consistent across model sizes.

Load-bearing premise

The whole pipeline assumes that a trajectory ending in the correct final answer contains good intermediate tool calls worth imitating, and that a trajectory ending in a wrong answer contains only bad ones, even though the final-answer label says nothing directly about those intermediate steps.

Editorial extensions

If this is right

  • Training a competent tool-augmented tabular reasoner no longer requires a GPT-4 teacher or human step annotations; only final answers from existing datasets are needed.
  • A single shared tool set trained on combined datasets outperforms per-dataset training, so the framework should benefit from adding more tabular datasets without redesigning the tools.
  • Because the method also learns from wrong trajectories, underrepresented reasoning categories, such as count-on-tables questions on TAT-QA, gain the most in the preference-optimization stage.
  • Full training of one MATATA instance takes about 16 GPU-hours on one A100, which makes the capability reproducible at a small budget.
  • The accuracy jump from the prompt-engineered baseline is large and consistent across model sizes, suggesting the weak-supervision loop, not the initial prompts, is what carries the result.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the final-answer signal transfers as these results suggest, the same recipe could be applied to other tool-based settings where only outcomes are labeled, such as calculator-driven science problems, code execution, or database querying, but the paper does not test those settings.
  • The weak label is likely noisier for tools whose outputs are not directly tied to the final answer; long-term robustness may depend on how often wrong intermediate steps cancel out in otherwise correct trajectories.
  • Because KTO consumes all wrong trajectories rather than pairwise preferences, the distribution of negative examples changes as the model improves; this could be exploited deliberately as a curriculum, a point the paper leaves implicit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces MATATA, a two-stage training method for tool-augmented multi-step reasoning over tabular data. A prompt-engineered planner/tool agent (MPE) generates correct trajectories; these are used to LoRA instruction-tune per-tool and planner adapters (MIT); MIT then generates correct and incorrect trajectories used for KTO preference optimization. The final models MATATA-3.8B and MATATA-8B are evaluated on FinQA, TAT-QA, and TabMWP, reporting strong accuracies including 77.59 on FinQA and 98.13 on TabMWP, with per-stage gains shown in Table 4. The paper claims that this is the first framework to train tabular-reasoning agents with final-answer weak supervision, without GPT-4 teachers or human intermediate annotations.

Significance. If the empirical results are reproducible, MATATA would be a practically valuable result: it suggests that intermediate tool-level annotations are unnecessary for training competent SLM agents on tabular math reasoning, and it demonstrates cross-dataset sharing of tool adapters. Strengths include a clean two-stage pipeline, ablations over training stages and single-vs-all dataset training (Table 4), use of multiple SLM families, and detailed prompts in the appendix. The main weaknesses are the absence of tool-level evaluation of the weak-supervision mechanism, lack of error bars/multiple seeds, and missing details for KTO weights and dataset sizes, which currently leave the central mechanism under-evidenced.

major comments (3)
  1. [§3.2, Eqs. (7)–(13)] The method's key assumption is that a final-answer correct trajectory provides positive training signal for every tool call in that trajectory and a final-answer incorrect trajectory provides negative signal for every tool call. This is not validated. Correct final answers can occur despite bad intermediate tool calls (e.g., a Row/Column Lookup that drops needed information or a Program_Generator with compensating errors), and incorrect final answers can occur despite correct tool calls (e.g., only Answer_Extractor failing). The paper reports no tool-level accuracy, no label-noise analysis, and no oracle comparison. Since the claimed contribution is precisely that this weak-supervision mechanism trains each tool, please add per-tool evaluation on held-out trajectories, an ablation with oracle intermediate labels, or at least a manual error analysis to substantiate the mechanism and quantify the noise level.
  2. [Tables 3 and 4] All numbers appear to be from a single run without error bars or significance tests. Fine-tuning with LoRA and KTO is stochastic, and the headline claims (FinQA 77.59 vs TAT-LLM-70B 76.81; TAT-QA 77.6 vs TAT-LLM-13B 77.5) are small margins. Please report mean and standard deviation over at least 3 random seeds for the main MATATA configurations and for the prompt-engineered baseline, and state whether the differences are stable. Without this, the 'state-of-the-art' and per-stage improvement claims are not fully supported.
  3. [§4.1, Eqs. (12)–(14)] The KTO stage is described as using 'task-specific weights' to handle class imbalance, but the weights are not reported, and Table 5 reports only percentages of correct/incorrect trajectories, not the actual sizes of DIT+ and DIT− used as KTO training sets. Exact counts and the weight hyperparameters are necessary for reproducibility and to assess how much data each adapter sees. Please include them (e.g., a table with |DPE_i|, |DIT+_i|, |DIT−_i| per tool and dataset).
minor comments (5)
  1. [Abstract and §1] The phrases 'annotation-free' and 'end-to-end' are overstated. The method still requires 5 manually chosen in-context examples per tool (Appendix B) and final-answer labels from benchmarks, and training is performed per adapter on extracted input/output pairs rather than by backpropagating through the whole tool chain. Please qualify these terms, e.g., 'no intermediate human annotations' and 'weakly supervised per-module training'.
  2. [Table 3] The TAT-QA score for MATATA-8B (77.6) does not carry an asterisk, while Table 4 reports 77.81* on the golden test set. Please clarify which test set each reported TAT-QA number uses.
  3. [Table 1] The table uses inconsistent symbols (e.g., '/' marks) that make some entries hard to read; please use a consistent legend.
  4. [Appendix B] The prompt listings contain numerous spacing/OCR artifacts (e.g., 'qu es ti on', malformed table separators). Please typeset the prompts properly.
  5. [Reproducibility] Please state whether code and trained adapters will be released; no code link or data-generation script is currently provided.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the weak-supervision signal is the external final-answer correctness label and the evaluation is independent of training.

full rationale

MATATA's derivation chain is not circular. The weak-supervision labels in Eqs. (7)-(13) are the benchmark final answers a_d, which are external to the model and never produced by it. Phase 1 filters MPE trajectories by whether MPE's final answer matches a_d; Phase 3 filters MIT trajectories the same way. This is self-training/distillation, not a derivation of the target result from the method's own outputs. The per-tool datasets in Eqs. (8) and (12) propagate trajectory-level labels to tool-level examples, which is a heuristic assumption about label quality; if wrong it would degrade training, but it does not make the evaluation circular, because test accuracy is measured against independent final-answer labels. Model selection on validation sets is standard. The paper contains no load-bearing self-citations: references [9,15,33] support general KTO and self-improvement techniques and are not authored by the present authors, and no uniqueness theorem or ansatz is imported from prior work. The reported test numbers are not fitted inputs.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities. The training relies on four domain assumptions about the reliability of final-answer filtering, plus one standard comparability assumption. The free parameters are hyperparameters and prompt-engineering choices that are not centrally fitted to the test sets but are needed to reproduce the method.

free parameters (7)
  • LoRA rank r = 64
    Chosen for all adapters; not tuned per tool or dataset.
  • LoRA alpha = 32
    Chosen for all adapters.
  • LoRA dropout = 0.05
    Chosen for all adapters.
  • Learning rate = 1e-5
    Chosen for all adapters.
  • KTO beta = 0.1
    Default value from KTO, used for all adapters.
  • KTO class-imbalance task weights = not reported
    Applied to balance correct/incorrect trajectories; values omitted, so this is an unreported hand-set parameter.
  • Number of in-context examples per tool prompt = 5
    Hand-picked for Phase 1 prompts; same examples used for all models and datasets.
assumptions (4)
  • domain assumption Trajectories with a correct final answer provide valid supervision for every tool they invoke.
    Phase 2 (Eq. 8-9) trains each tool on input/output pairs from DPE only, which is filtered by final answer equality. This assumes no harmful intermediate mistakes in correct trajectories.
  • domain assumption Trajectories with an incorrect final answer are wholly negative for each tool they invoke.
    Phase 4 (Eq. 10-13) labels all tool calls in DIT- as negative for KTO. Some individual tool calls in a wrong trajectory may be useful, so this can inject noise.
  • domain assumption Final-answer exact match is a reliable weak supervision signal.
    Used throughout to split correct/incorrect; ignores partially correct reasoning and metric subtleties.
  • standard math The three public benchmarks are mutually comparable and their published baseline scores are trustworthy.
    The SOTA comparison assumes consistent evaluation; TAT-QA has leaderboard and golden test variants, noted in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MATATA: Weakly Supervised End-to-End MAthematical Tool-Augmented Reasoning for Tabular Applications." pith.science (2026). https://pith.science/paper/EV5CM2VV

@misc{pith2026241118915,
  author       = {Pith},
  title        = {Pith review of: MATATA: Weakly Supervised End-to-End MAthematical Tool-Augmented Reasoning for Tabular Applications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EV5CM2VV}},
  note         = {Machine review of arXiv:2411.18915}
}
read the original abstract

Business documents often contain substantial tabular and textual information with numerical values, requiring mathematical reasoning for effective document understanding. While Small Language Models (SLMs) still struggle at this task, tool-augmented multi-step agents perform better, at the cost of relying on closed-source or larger models, external data, or extensive prompt-engineering. This work introduces MATATA, a novel weakly supervised end-to-end approach to train multi-step reasoning language agents for document tabular applications. MATATA presents an annotation-free paradigm for each agent to enhance 3.8B/8B SLMs. During its two-stage training, MATATA uses the final outcome of the multi-step reasoning chain as weak supervision. This approach avoids having to individually supervise each intermediate agent in the reasoning chain. By employing an adaptive planner and shared tools across different datasets, MATATA shows robust performance. Experiments demonstrate that MATATA achieves state-of-the-art on FinQA, and on TAT-QA among reasoning methods based on open-source SLMs. Although being SLM-based, MATATA closely matches GPT-4-based frameworks on TabMWP. This novel weakly supervised approach enables training an end-to-end multi-step reasoning agent without intermediate supervision, supporting future developments of cost-effective powerful agentic systems.

Figures

Figures reproduced from arXiv: 2411.18915 by the authors.

Figure 1
Figure 1. Schematic representation of MATATA solving a problem instance ⟨q, c, t⟩. 1 Input is first sent to the MATATA Planner. 2 Planner then generates a tool use trajectory, Tr, that expresses the sequence of how the tools available to MATATA should be applied to solve the problem. After that, 3 input is sent to the first tool in Tr, and 5 the sequence of tools is applied to 5 produce the output of MATATA. multiple steps st… view at source ↗
Figure 2
Figure 2. Schematic representation of MATATA training phases. Phase 1 relies on a prompt-engineered MATATA (MPE) to generate a training dataset D PE containing tool-augmented reasoning trajectories (Trd) that yield correct answers. Phase 2 uses D PE for a weakly supervised LoRA instruction tuning over a pretrained LLM, pro￾ducing MIT. MIT is used during Phase 3 to generate D IT, that contains both correct and incorrect reason… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reason Before You Retrieve: Agentic Planning for Multi-modal RAG

    cs.AI 2026-06 reject novelty 5.0 of 10

    MM-R2 claims SOTA multimodal RAG accuracy on InfoSeek and Encyclopedic VQA via intent grounding plus a 10-topic KnowledgeMap, but its teacher trajectories leak the gold Wikipedia page and omit the image.

Reference graph

Works this paper leans on

38 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Vinayagame, G

    Abdin, M., al.: Phi-3 technical report: A highly capable language model locally on your phone (2024),https://arxiv.org/abs/2404.14219, mIT License 16 V. Vinayagame, G. Senay, and L. Martí

    arXiv 2024

  2. [2]

    Bi, J., Wu, Y., Xing, W., Wei, Z.: Enhancing the reasoning capabilities of small language models via solution guidance fine-tuning (2024), https://arxiv.org/ abs/2412.09906

    arXiv 2024

  3. [3]

    In: Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good

    Boerner, T.J., Deems, S., Furlani, T.R., Knuth, S.L., Towns, J.: ACCESS: Ad- vancing innovation: NSF’s advanced cyberinfrastructure coordination ecosystem: Services & support. In: Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good. p. 173–176. PEARC ’23, Association for Computing Machinery, New York, NY, USA (2023)....

    arXiv 2023

  4. [4]

    In: Proceedings of the 34th International Conference on Neural Information Processing Systems

    Brown, T.B., al.: Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS ’20, Curran Associates Inc., Red Hook, NY, USA (2020)

    work page 2020

  5. [5]

    In: The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024),https://openreview.net/forum?id=ZIpdu0cHYu

    Chen, S., Wang, Y., Wu, Y.F., Chen, Q.G., Xu, Z., Luo, W., Zhang, K., Zhang, L.: Advancing tool-augmented large language models: Integrating insights from errors in inference trees. In: The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024),https://openreview.net/forum?id=ZIpdu0cHYu

    work page 2024

  6. [6]

    Transactions on Machine Learning Research (2023)

    Chen, W., Ma, X., Wang, X., Cohen, W.W.: Program of thoughts prompting: Dis- entangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research (2023)

    work page 2023

  7. [7]

    Proceedings of EMNLP 2021 (2021), mIT License

    Chen, Z., Chen, W., Smiley, C., Shah, S., Borova, I., Langdon, D., Moussa, R., Beane,M.,Huang,T.H.,Routledge,B.,Wang,W.Y.:Finqa:Adatasetofnumerical reasoning over financial data. Proceedings of EMNLP 2021 (2021), mIT License

    work page 2021

  8. [8]

    Sustainable Computing: Informatics and Systems 38, 100857 (2023)

    Desislavov, R., Martńez-Plumed, F., Hernéndez-Orallo, J.: Trends in ai infer- ence energy consumption: Beyond the performance-vs-parameter laws of deep learning. Sustainable Computing: Informatics and Systems 38, 100857 (2023). https://doi.org/https://doi.org/10.1016/j.suscom.2023.100857, https:// www.sciencedirect.com/science/article/pii/S2210537923000124

    arXiv 2023

Show all 38 references
  1. [9]

    Ethayarajh,K.,Xu,W.,Muennighoff,N.,Jurafsky,D.,Kiela,D.:Kto:Modelalign- ment as prospect theoretic optimization (2024), https://arxiv.org/abs/2402. 01306

    2024

  2. [10]

    Fang, X., Xu, W., Tan, F.A., Zhang, J., Hu, Z., Qi, Y., Nickleach, S., Socolinsky, D., Sengamedu, S., Faloutsos, C.: Large language models(llms) on tabular data: Prediction, generation, and understanding – a survey (2024),https://arxiv.org/ abs/2402.17944

    2024 arXiv

  3. [11]

    Gou, Z., Shao, Z., Gong, Y., yelong shen, Yang, Y., Huang, M., Duan, N., Chen, W.:ToRA:Atool-integratedreasoningagentformathematicalproblemsolving.In: The Twelfth International Conference on Learning Representations (2024),https: //openreview.net/forum?id=Ep0TtjVoap

    2024

  4. [12]

    In: Practice and Experience in Advanced Research Computing 2021: Evolution Across All Di- mensions

    Hancock, D.Y., Fischer, J., Lowe, J.M., Snapp-Childs, W., Pierce, M., Marru, S., Coulter, J.E., Vaughn, M., Beck, B., Merchant, N., Skidmore, E., Jacobs, G.: Jetstream2: Accelerating cloud computing via Jetstream. In: Practice and Experience in Advanced Research Computing 2021...

    2021

  5. [13]

    In: First Conference on Language Modeling (2024), https://openreview.net/forum?id=stmqBSW2dV

    Hosseini, A., Yuan, X., Malkin, N., Courville, A., Sordoni, A., Agarwal, R.: V- STar: Training verifiers for self-taught reasoners. In: First Conference on Language Modeling (2024), https://openreview.net/forum?id=stmqBSW2dV

    2024

  6. [14]

    In: International Con- MATATA: Tool-Augmented Reasoning for Tabular Applications 17 ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf9

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Con- MATATA: Tool-Augmented Reasoning for Tabular Applications 17 ference on Learning Representations (2022),https://openr...

    2022

  7. [15]

    In: Bouamor, H., Pino, J., Bali, K

    Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., Han, J.: Large lan- guage models can self-improve. In: Bouamor, H., Pino, J., Bali, K. (eds.) Pro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. pp. 1051–1068. Association for Computat...

    2023 doi

  8. [16]

    Husom, E.J., Goknil, A., Shar, L.K., Sen, S.: The price of prompting: Profiling energy use in large language models inference (2024),https://arxiv.org/abs/ 2407.16893

    2024

  9. [17]

    Kim, J., Paranjape, B., Khot, T., Hajishirzi, H.: Husky: A unified, open-source language agent for multi-step reasoning (2024), https://arxiv.org/abs/2406. 06469

    2024

  10. [18]

    Laurençon, H., Marafioti, A., Sanh, V., Tronchon, L.: Building and better under- standing vision-language models: insights and future directions. (2024)

    2024

  11. [19]

    In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=HtqnVSCj3q

    Lu, P., Peng, B., Cheng, H., Galley, M., Chang, K.W., Wu, Y.N., Zhu, S.C., Gao, J.: Chameleon: Plug-and-play compositional reasoning with large language models. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://openreview.net/forum?id=HtqnVSCj3q

    2023

  12. [20]

    In: ICLR

    Lu, P., Qiu, L., Chang, K.W., Wu, Y.N., Zhu, S.C., Rajpurohit, T., Clark, P., Kalyan,A.:Dynamicpromptlearningviapolicygradientforsemi-structuredmath- ematical reasoning. In: ICLR. ICLR (2023),https://openreview.net/forum?id= DHyHRBwJUTN

    2023

  13. [21]

    Luccioni, S., Jernite, Y., Strubell, E.: Power hungry processing: Watts driving the cost of ai deployment? In: The 2024 ACM Conference on Fairness, Accountability, and Transparency. p. 85–99. FAccT ’24, ACM (Jun 2024).https://doi.org/10. 1145/3630106.3658542, http://dx.doi.org...

    2024

  14. [22]

    Mathew, M., Karatzas, D., Jawahar, C.V.: Docvqa: A dataset for vqa on document images (2021), https://arxiv.org/abs/2007.00398

    2021 arXiv

  15. [23]

    PeerJ Computer Science 3, e103(Jan2017)

    Meurer, A., Smith, C.P., Paprocki, M., Čertík, O., Kirpichev, S.B., Rocklin, M., Kumar, A., Ivanov, S., Moore, J.K., Singh, S., Rathnayake, T., Vig, S., Granger, B.E., Muller, R.P., Bonazzi, F., Gupta, H., Vats, S., Johansson, F., Pedregosa, F., Curry, M.J., Terrel, A.R., Rouč...

    doi

  16. [24]

    In: Proceedings of the 36th International Conference on Neural Information Processing Systems

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., Lowe, R.: Training language models to follo...

    2024

  17. [25]

    In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://arxiv.org/abs/2305.18290

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Di- rect preference optimization: Your language model is secretly a reward model. In: Thirty-seventh Conference on Neural Information Processing Systems (2023), https://arxiv.org/abs/2305.18290

    2023 arXiv

  18. [26]

    (eds.) Findings of the Association for Computational Linguistics: 18 V

    Srivastava, P., Malik, M., Gupta, V., Ganu, T., Roth, D.: Evaluating LLMs’ mathe- maticalreasoning infinancialdocument questionanswering.In:Ku,L.W.,Martins, A., Srikumar, V. (eds.) Findings of the Association for Computational Linguistics: 18 V. Vinayagame, G. Senay, and L. Ma...

    2024 doi

  19. [27]

    Team, M.A.: Un ministral, des ministraux (2024),https://mistral.ai/fr/news/ ministraux/, mistral Research License

    2024

  20. [28]

    Touvron, H., al.: Llama 2: Open foundation and fine-tuned chat models (2023), https://arxiv.org/abs/2307.09288

    2023 arXiv

  21. [29]

    Wang, T., Li, S., Lu, W.: Self-training with direct preference optimization improves chain-of-thought reasoning (2024),https://arxiv.org/abs/2407.18248

    2024 arXiv

  22. [30]

    NIPS ’22, Curran Associates Inc., Red Hook, NY, USA (2024)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E.H., Le, Q.V., Zhou, D.: Chain-of-thought prompting elicits reasoning in large language models.In:Proceedingsofthe36thInternationalConferenceonNeuralInformation Processing Systems. NIPS ’22, Curran Associ...

    2024

  23. [32]

    In: Duh, K., Gomez, H., Bethard, S

    You, W., Yin, S., Zhao, X., Ji, Z., Zhong, G., Bai, J.: MuMath: Multi-perspective data augmentation for mathematical reasoning in large language models. In: Duh, K., Gomez, H., Bethard, S. (eds.) Findings of the Association for Computational Linguistics: NAACL 2024. pp. 2932–2...

    2024 doi

  24. [33]

    In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K

    Zelikman, E., Wu, Y., Mu, J., Goodman, N.: STar: Bootstrapping reasoning with reasoning. In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (eds.) Advances in Neural Information Processing Systems (2022),https://openreview.net/forum? id=_3ELRdg2sgI

    2022

  25. [34]

    Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wu, F., Wang, G.: Instruction tuning for large language models: A survey (2024), https://arxiv.org/abs/2308.10792

    2024

  26. [35]

    In: Proceedings of the 30th ACM International Conference on Multimedia

    Zhu, F., Lei, W., Feng, F., Wang, C., Zhang, H., Chua, T.S.: Towards com- plex document understanding by discrete reasoning. In: Proceedings of the 30th ACM International Conference on Multimedia. p. 4857–4866. MM ’22, ACM (Oct 2022). https://doi.org/10.1145/3503161.3548422, h...

    2022

  27. [36]

    Zhu, F., Lei, W., Huang, Y., Wang, C., Zhang, S., Lv, J., Feng, F., Chua, T.S.: Tat- qa: A question answering benchmark on a hybrid of tabular and textual content in finance (2021), https://arxiv.org/abs/2105.07624, creative Commons (CC BY) Attribution 4.0 International

    2021 arXiv

  28. [37]

    Zhu, F., Liu, Z., Feng, F., Wang, C., Li, M., Chua, T.S.: Tat-llm: A specialized language model for discrete reasoning over tabular and textual data (2024),https: //arxiv.org/abs/2401.13223

    2024 arXiv

  29. [38]

    Performance on training data of MATATA-3.8B and MATATA-8B over dif- ferent configurations of the training sets (in % points)

    Zmigrod,R.,Wang,D.,Sibue,M.,Pei,Y.,Babkin,P.,Brugere,I.,Liu,X.,Navarro, N., Papadimitriou, A., Watson, W., Ma, Z., Nourbakhsh, A., Shah, S.: Buddie: A business document dataset for multi-task information extraction (2024),https: //arxiv.org/abs/2404.04003 MATATA: Tool-Augmente...

    2024 arXiv

  30. [39]

    stem " ( the first digit or digits ) and a

    Correct examples from the prompt-engineered model are used for Instruction Tuning, while correct and incorrect examples from Instruction Tuning are used for KTO preference optimization. B Prompts Used in Phase 1 Prompts used in Phase 1 are inspired from Chameleon [19] and TAT-...

    2018

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.