{"id":"4b007a0a-39ba-428d-b2fb-07c7719b133b","arxiv_id":"2411.18915","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"MATATA uses final-answer-only weak supervision with instruction tuning and KTO preference optimization to train tool-augmented SLM agents that beat or match much larger models on FinQA, TAT-QA, and TabMWP.","lead":"MATATA trains small language models (3.8B and 8B) to answer math questions about business tables by having a planner call specialized tools, using only the final answer as the training signal instead of step-by-step human annotations. If the results hold, it offers a cheaper, more private way to build document-understanding agents without GPT-4 teachers.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Trajectory-level final-answer labels are propagated to every tool call (Eqs. 8, 12); no evidence shows per-tool adapters learn correct behavior, so the weak-supervision mechanism may be training on mislabeled intermediates.","rationale":"The reader's weakest assumption is the same one I find most load-bearing: the paper's weak-supervision construction assigns the trajectory-level binary outcome to every intermediate tool call without checking tool-level correctness. The central claim is not just that MATATA obtains high scores; it is that this validates an annotation-free, end-to-end way to train tool agents with final-answer-only labels. That claim stands or falls on whether final-answer correctness is a usable proxy for per-tool correctness. If the proxy is poor, the method may still work in practice, but the stated mechanism is unsupported and the results could be explained by the few most frequently used tools or by shortcut learning rather than by weak supervision of the whole chain. Other concerns (missing code, single-run results, loose use of 'annotation-free'/'end-to-end') are real but secondary; they affect reproducibility and framing, not the internal validity of the training signal. The staged ablations in Table 4 do show consistent gains from IT and KTO, which is independent support that the weak-supervision pipeline has some signal; that is why I do not push to reject. But without a tool-level noise audit, the core mechanism remains a black box. The proposed sampling/annotation test directly measures the mislabel rates and, with the cleaned-label retraining follow-up, would show whether the noise is consequential.","tokens_in":41469,"tokens_out":8945,"duration_ms":90611,"concrete_test":"Sample 100 correct trajectories from DPE (Phase 1) and 100 incorrect trajectories from DIT- (Phase 3) for FinQA (or TAT-QA). For every tool call in each trajectory, have two independent annotators (human experts or a strong LLM with access to the question/table/gold answer but not to the trajectory's final-answer label) judge whether the tool's output is correct, incorrect, or unverifiable relative to the gold reasoning. Compute the false-positive rate (tool calls labeled incorrect in correct trajectories) and false-negative rate (tool calls labeled correct in incorrect trajectories). If either rate exceeds, say, 20%, the central assumption is factually violated; a follow-up retraining of the affected tool adapters on cleaned labels would confirm whether the violations change test accuracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise, stated in Eqs. (7)-(13), is that a trajectory whose final answer is correct can be used as a positive training example for every tool call in that trajectory, and a trajectory whose final answer is wrong as a negative example for every tool call. This propagation is not justified: a correct final answer can be obtained even if an intermediate Row/Column Lookup drops needed rows or columns, an extractor returns irrelevant spans, or a program contains compensating errors; conversely, a wrong final answer can occur after perfectly correct tool calls (e.g., only the Answer_Extractor fails). If such mislabeled tool-level examples are frequent, the per-tool LoRA adapters are trained on noisy labels, and the gains over the prompt-engineered baseline in Table 4 may be driven by the strongest tools (Program_Generator/Answer_Extractor) or by distributional artifacts rather than by the claimed weak-supervision mechanism. The paper reports no tool-level accuracy or label-noise analysis, and the language 'end-to-end weak supervision' depends on this unverified equivalence between trajectory-level and tool-level correctness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MATATA, a two-stage training method for tool-augmented multi-step reasoning over tabular data. A prompt-engineered planner/tool agent (MPE) generates correct trajectories; these are used to LoRA instruction-tune per-tool and planner adapters (MIT); MIT then generates correct and incorrect trajectories used for KTO preference optimization. The final models MATATA-3.8B and MATATA-8B are evaluated on FinQA, TAT-QA, and TabMWP, reporting strong accuracies including 77.59 on FinQA and 98.13 on TabMWP, with per-stage gains shown in Table 4. The paper claims that this is the first framework to train tabular-reasoning agents with final-answer weak supervision, without GPT-4 teachers or human intermediate annotations.","tokens_in":41760,"tokens_out":6814,"duration_ms":61712,"significance":"If the empirical results are reproducible, MATATA would be a practically valuable result: it suggests that intermediate tool-level annotations are unnecessary for training competent SLM agents on tabular math reasoning, and it demonstrates cross-dataset sharing of tool adapters. Strengths include a clean two-stage pipeline, ablations over training stages and single-vs-all dataset training (Table 4), use of multiple SLM families, and detailed prompts in the appendix. The main weaknesses are the absence of tool-level evaluation of the weak-supervision mechanism, lack of error bars/multiple seeds, and missing details for KTO weights and dataset sizes, which currently leave the central mechanism under-evidenced.","major_comments":[{"comment":"The method's key assumption is that a final-answer correct trajectory provides positive training signal for every tool call in that trajectory and a final-answer incorrect trajectory provides negative signal for every tool call. This is not validated. Correct final answers can occur despite bad intermediate tool calls (e.g., a Row/Column Lookup that drops needed information or a Program_Generator with compensating errors), and incorrect final answers can occur despite correct tool calls (e.g., only Answer_Extractor failing). The paper reports no tool-level accuracy, no label-noise analysis, and no oracle comparison. Since the claimed contribution is precisely that this weak-supervision mechanism trains each tool, please add per-tool evaluation on held-out trajectories, an ablation with oracle intermediate labels, or at least a manual error analysis to substantiate the mechanism and quantify the noise level.","section":"§3.2, Eqs. (7)–(13)"},{"comment":"All numbers appear to be from a single run without error bars or significance tests. Fine-tuning with LoRA and KTO is stochastic, and the headline claims (FinQA 77.59 vs TAT-LLM-70B 76.81; TAT-QA 77.6 vs TAT-LLM-13B 77.5) are small margins. Please report mean and standard deviation over at least 3 random seeds for the main MATATA configurations and for the prompt-engineered baseline, and state whether the differences are stable. Without this, the 'state-of-the-art' and per-stage improvement claims are not fully supported.","section":"Tables 3 and 4"},{"comment":"The KTO stage is described as using 'task-specific weights' to handle class imbalance, but the weights are not reported, and Table 5 reports only percentages of correct/incorrect trajectories, not the actual sizes of DIT+ and DIT− used as KTO training sets. Exact counts and the weight hyperparameters are necessary for reproducibility and to assess how much data each adapter sees. Please include them (e.g., a table with |DPE_i|, |DIT+_i|, |DIT−_i| per tool and dataset).","section":"§4.1, Eqs. (12)–(14)"}],"minor_comments":[{"comment":"The phrases 'annotation-free' and 'end-to-end' are overstated. The method still requires 5 manually chosen in-context examples per tool (Appendix B) and final-answer labels from benchmarks, and training is performed per adapter on extracted input/output pairs rather than by backpropagating through the whole tool chain. Please qualify these terms, e.g., 'no intermediate human annotations' and 'weakly supervised per-module training'.","section":"Abstract and §1"},{"comment":"The TAT-QA score for MATATA-8B (77.6) does not carry an asterisk, while Table 4 reports 77.81* on the golden test set. Please clarify which test set each reported TAT-QA number uses.","section":"Table 3"},{"comment":"The table uses inconsistent symbols (e.g., '/' marks) that make some entries hard to read; please use a consistent legend.","section":"Table 1"},{"comment":"The prompt listings contain numerous spacing/OCR artifacts (e.g., 'qu es ti on', malformed table separators). Please typeset the prompts properly.","section":"Appendix B"},{"comment":"Please state whether code and trained adapters will be released; no code link or data-generation script is currently provided.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"This is a solid empirical paper whose main claim—that final-answer weak supervision can train tool-augmented tabular reasoning agents—is plausible and potentially important. The essential missing evidence is an evaluation of whether the per-tool adapters actually learn correct tool behavior rather than benefiting from trajectory-level artifacts. If the authors add tool-level diagnostics and multi-seed results, I would expect the paper to be acceptable. The 'annotation-free' framing should also be softened. No concerns about novelty or authorship beyond the above."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nThe headline: this is a solid, clearly-written empirical paper that shows you can train a multi-step tool-using agent on 3.8B/8B SLMs using only final-answer labels, with results that are competitive with or better than GPT-4-based and human-annotated systems on FinQA, TAT-QA, and TabMWP. The two-stage recipe—instruction tuning on self-generated correct trajectories followed by KTO on both correct and incorrect ones, with separate LoRA adapters per tool and the planner—is a real combination that I haven't seen applied to tabular agents before. The experimental design is careful: they compare single-dataset vs all-dataset training, show consistent gains at each stage, and the reported numbers are plausible.\n\nThe soft spots are mostly about what's not shown. No code, no seeds, no error bars; everything is single-run. The \"annotation-free\" claim overreaches—they still hand-write 5 in-context examples per tool prompt, which is minimal but not zero. More substantively, the load-bearing assumption is that a trajectory's final-answer correctness can be pushed down to every tool call in that trajectory (Eqs. 8 and 12). That's not obviously true: a correct final answer can be produced despite a bad lookup or extractor, and an incorrect final answer can happen after correct tool calls. The paper reports no tool-level accuracy or label-noise analysis. This could mean the per-tool adapters are trained on mislabeled examples, and the gains might come mostly from the strongest tools or from distributional artifacts rather than from the weak-supervision mechanism itself.\n\nThat said, this is a testable concern, not a fatal one. The gains over the prompt-engineered baseline are large and consistent, the two stages each contribute, and the method is well-specified enough to reproduce once code is available. The paper is honest about its lineage (STaR, Husky, ToRA, KTO) and doesn't overclaim the theoretical novelty.\n\nWho is this for? Anyone working on tool-augmented LLM agents, weak supervision, or tabular math reasoning. It deserves a serious referee: the results are important enough to verify, and the weak-supervision noise issue is exactly the kind of thing a good reviewer can push on. My recommendation is to send it to review with a request for code, multiple seeds, and at least a basic analysis of per-tool label quality (e.g., comparing a few hand-labeled tool outputs against the propagated labels). If that holds up, this is a strong paper.\n\nBest,\n[Your name]","headline":"Genuinely useful weak-supervision recipe for tool-augmented tabular agents, but the per-tool label-propagation assumption needs evidence.","tokens_in":42251,"tokens_out":3962,"would_cite":true,"duration_ms":31075,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MATATA uses only the final answer as the training signal for every planning step and tool call, and reports that small open language models trained this way match or beat much larger and more heavily annotated systems on FinQA, TAT-QA…","keywords":["weak supervision","tool-augmented reasoning","tabular mathematical reasoning","small language models","instruction tuning","KTO preference optimization","document understanding"],"falsifier":"Manually annotate the intermediate tool outputs on a random sample of validation trajectories and measure how often a tool call inside a correct-answer trajectory is wrong or a tool call inside a wrong-answer trajectory is right; if either is frequent, the weak label is not actually supervising individual tools. A direct ablation is to train the same two-stage pipeline with oracle step-level labels and check whether accuracy rises substantially.","tokens_in":41313,"feed_emoji":"📊","tokens_out":6558,"duration_ms":56605,"temperature":0.7,"pith_summary":"This paper tries to establish that a multi-step, tool-using reasoning agent for tabular documents can be trained end-to-end with nothing but the final answer as supervision. MATATA starts from a lightly prompt-engineered model, keeps only the reasoning trajectories that happen to end in the right answer, fine-tunes the planner and each tool on those trajectories, then applies KTO preference optimization using both right and wrong trajectories. On FinQA, TAT-QA, and TabMWP, 3.8B and 8B open models trained this way reach scores that rival or beat frameworks built on GPT-4 teachers or human step-by-step annotations. If true, the result removes the main bottleneck to deploying capable tabular-reasoning agents: expensive intermediate labels.","feed_headline":"Final answers alone push 8B models to top table-math scores","feed_subtitle":"A two-stage self-improvement loop trains 3.8B/8B tabular agents with no intermediate annotations, rivaling GPT-4 pipelines.","key_machinery":"The load-bearing object is the trajectory $T^r_d$, a sequence of tool calls emitted by the planner; each tool's LoRA adapter is trained on the input-output pairs $\\langle \\text{input}_i, \\text{output}_i\\rangle$ collected from those trajectories, with the trajectory's final correctness as the only label. KTO turns this single final-answer signal into a per-trajectory preference signal, letting the model learn from failed trajectories as well as successes. The machinery also includes a shared tool set (row and column extractors, context extractor, program generator and executor, scale finder, answer extractor, and related tools) that lets one adapter be trained on examples drawn from several datasets at once.","core_discovery":"The paper's central claim is that a trajectory's final-answer correctness is a sufficient label for training every component of a tool-augmented reasoner. A MATATA instance consists of a planner $P$ and a set of tools $\\{T_i\\}$; the planner outputs a trajectory $T^r = \\langle T_{\\text{start}}, \\ldots, T_{\\text{end}}\\rangle$, and the tools are composed to produce the predicted answer $\\hat{a}$. Training data are formed by running the model on dataset instances and splitting trajectories only by whether $\\hat{a}$ equals the gold answer $a_d$. In the instruction-tuning stage the correct trajectories are used; in the KTO stage both correct and incorrect trajectories get binary labels $\\pm 1$ for the same planner and tool inputs. The paper reports that this weak signal yields FinQA accuracy 77.59, TAT-QA exact match 77.6, and TabMWP accuracy 98.13 on an 8B model, and that the two-stage gains are consistent across model sizes.","pith_inferences":["If the final-answer signal transfers as these results suggest, the same recipe could be applied to other tool-based settings where only outcomes are labeled, such as calculator-driven science problems, code execution, or database querying, but the paper does not test those settings.","The weak label is likely noisier for tools whose outputs are not directly tied to the final answer; long-term robustness may depend on how often wrong intermediate steps cancel out in otherwise correct trajectories.","Because KTO consumes all wrong trajectories rather than pairwise preferences, the distribution of negative examples changes as the model improves; this could be exploited deliberately as a curriculum, a point the paper leaves implicit."],"forward_implications":["Training a competent tool-augmented tabular reasoner no longer requires a GPT-4 teacher or human step annotations; only final answers from existing datasets are needed.","A single shared tool set trained on combined datasets outperforms per-dataset training, so the framework should benefit from adding more tabular datasets without redesigning the tools.","Because the method also learns from wrong trajectories, underrepresented reasoning categories, such as count-on-tables questions on TAT-QA, gain the most in the preference-optimization stage.","Full training of one MATATA instance takes about 16 GPU-hours on one A100, which makes the capability reproducible at a small budget.","The accuracy jump from the prompt-engineered baseline is large and consistent across model sizes, suggesting the weak-supervision loop, not the initial prompts, is what carries the result."],"supporting_citations":[{"why":"The Husky baseline that MATATA most directly competes against; establishes the GPT-4-teacher fine-tuning paradigm for tool agents.","marker":"[17]"},{"why":"ToRA, the tool-integrated reasoning agent baseline used on TabMWP that also relies on generated annotations.","marker":"[11]"},{"why":"TAT-LLM, the human-expert-annotated stepwise baseline whose FinQA and TAT-QA scores bound what weak supervision must approach.","marker":"[37]"},{"why":"Introduces KTO, the binary-signal preference optimization used in Phase 4.","marker":"[9]"},{"why":"Supplies the self-improvement premise that a model can learn from its own generated correct trajectories.","marker":"[15]"},{"why":"LoRA, the adapter method used to train each planner and tool without full model fine-tuning.","marker":"[14]"},{"why":"FinQA dataset and its FinQANet baseline.","marker":"[7]"},{"why":"TAT-QA dataset and TagOP baseline, the benchmark where MATATA reports its best open-SLM result.","marker":"[36]"},{"why":"TabMWP dataset, the benchmark used for the near-GPT-4 comparison.","marker":"[20]"}],"fun_headline_variants":["Final-answer labels alone train 8B tool-use reasoners","No intermediate labels: 8B model rivals GPT-4 on table math","Single weak signal trains multi-step reasoning agents","Final answer as sole supervision: 8B agents top FinQA","Weakly-supervised end-to-end: 8B tabular agents match GPT-4"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole pipeline assumes that a trajectory ending in the correct final answer contains good intermediate tool calls worth imitating, and that a trajectory ending in a wrong answer contains only bad ones, even though the final-answer label says nothing directly about those intermediate steps.","fun_headline_variants_meta":{"raw":{"variants":["Final-answer labels alone train 8B tool-use reasoners","No intermediate labels: 8B model rivals GPT-4 on table math","Single weak signal trains multi-step reasoning agents","Final answer as sole supervision: 8B agents top FinQA","Weakly-supervised end-to-end: 8B tabular agents match GPT-4"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000877,"raw_usage":{"total_tokens":3820,"prompt_tokens":1000,"completion_tokens":2820,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":2727}},"tokens_in":616,"tokens_out":2820,"duration_ms":19134,"temperature":1.0,"reasoning_tokens":2727,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:44:59.696481+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Manually annotate the intermediate tool outputs on a random sample of validation trajectories and measure how often a tool call inside a correct-answer trajectory is wrong or a tool call inside a wrong-answer trajectory is right; if either is frequent, the weak label is not actually supervising individual tools. A direct ablation is to train the same two-stage pipeline with oracle step-level labels and check whether accuracy rises substantially.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Husky baseline that MATATA most directly competes against; establishes the GPT-4-teacher fine-tuning paradigm for tool agents."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ToRA, the tool-integrated reasoning agent baseline used on TabMWP that also relies on generated annotations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces KTO, the binary-signal preference optimization used in Phase 4."},{"cited_title":"In: International Con- MATATA: Tool-Augmented Reasoning for Tabular Applications 17 ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf9","cited_arxiv_id":null,"evidence_quote":"LoRA, the adapter method used to train each planner and tool without full model fine-tuning."},{"cited_title":"Proceedings of EMNLP 2021 (2021), mIT License","cited_arxiv_id":null,"evidence_quote":"FinQA dataset and its FinQANet baseline."},{"cited_title":"In: ICLR","cited_arxiv_id":null,"evidence_quote":"TabMWP dataset, the benchmark used for the near-GPT-4 comparison."}],"review_version":1}