{"id":"943121f5-1957-49d1-9d08-d0276cb1bb8c","arxiv_id":"2412.15495","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"TL-Training, a task-feature-based training framework, lets a 7B CodeLLaMA-2 model reach competitive tool-use performance using only 1,217 training trajectories.","lead":"A 7B language model fine-tuned on just 1,217 tool-use examples with a three-part training recipe matches or beats much larger models on several tool-calling benchmarks. The recipe filters bad training trajectories, up-weights critical tokens, and adds a reward mechanism for reinforcement learning.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ToolEyes multi-turn result is not trustworthy until train/test disjointness is shown; training data is built from the ToolEyes tool collection and overlap with its 382 test queries is never addressed.","rationale":"I agree with the reader's weakest_assumption. The central claim explicitly includes ToolEyes among the four test sets where TL-CodeLLaMA-2 matches or surpasses leading LLMs, and the paper's own Appendix B states that the 1,217 training trajectories were created using ToolEyes as the tool foundation. No sentence in the paper reports that these artificially created user requirements were checked for overlap with the 382 ToolEyes test queries. This is a missing-support issue, not an accusation of bad faith: the overlap could be zero, but the paper cannot be verified as it stands. The single-turn results are less threatened because the training set is multi-turn and the other test sets are separate suites, though the absence of error bars remains a secondary weakness. The appendix theorems are tautological restatements and provide no independent evidence, but they are not the main risk. The correct disposition is to keep the verdict CONDITIONAL: if the overlap check comes back clean, the multi-turn claim is substantially supported; if overlap is found, the multi-turn result must be re-evaluated or removed from the central claim.","tokens_in":17872,"tokens_out":5212,"duration_ms":44085,"concrete_test":"Obtain the released training set (GitHub) and ToolEyes test set. For each of the 1,217 training queries, compute exact normalized-string match and embedding cosine similarity (e.g., Sentence-BERT or GPT-embedding) against all 382 test queries; also compare tool-call sequences in generated trajectories against test solution traces. Report the number of train/test pairs above a pre-registered threshold (e.g., cosine > 0.9 or exact match after lowercasing/whitespace normalization). If any overlap is found, remove or regenerate those training examples, retrain TL-CodeLLaMA-2, and recompute Table 4; if DE/CE/VA degrade materially, the multi-turn result was memorization. If zero overlap at a conservative threshold and a random sample of near-duplicates is human-verified, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5.1 and Appendix B state that the 1,217 training trajectories are 'artificially create[d]' using ToolEyes as the tool foundation, and Table 2 lists the same ToolEyes benchmark (382 multi-turn cases) as the multi-turn test. The paper never reports any filtering, deduplication, or disjointness check between the generated training queries/trajectories and the ToolEyes test queries. If even a modest fraction of the 382 test queries appear in training data—exactly, in paraphrase, or through the same tool-call sequences—the headline ToolEyes numbers (DE 0.82, CE 4.84, VA 77.75) and the claim of matching/surpassing closed-source models on multi-turn tool use are inflated by memorization. The risk is amplified by the PPO stage, which reuses 1,194 of the same trajectories and optimizes a reward function built from the same error taxonomy, so the model could be fitted to ToolEyes-style tasks rather than learning general tool use. This does not impugn the single-turn results on ToolAlpaca, RoTBench, or BFCL-v3, since those test sets use different task distributions, but the multi-turn component of the central claim is load-bearing and currently unsupported without an overlap analysis.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TL-Training, a three-component framework for tool-use fine-tuning of LLMs: masking erroneous interaction trajectories during SFT (MAE), adaptively up-weighting key tokens during SFT (PKT), and applying PPO with an error-category-based reward (IRM). The authors train CodeLLaMA-2-7B on 1,217 GPT-4o-generated trajectories and evaluate on ToolAlpaca, RoTBench, BFCL-v3, and ToolEyes, reporting that the resulting model matches or surpasses several open- and closed-source baselines, improves robustness on noisy tool environments, and preserves general task performance. The central claim is that a compact, task-feature-aware training set plus the proposed loss and reward design suffices to reach state-of-the-art tool-use performance with a 7B model.","tokens_in":18059,"tokens_out":5242,"duration_ms":43308,"significance":"If the results hold, the paper offers a practical and data-efficient alternative to large-scale SFT for tool use, with the notable strengths of a compact training set, public code and data, an explicit error taxonomy, and an ablation design covering each proposed component. The single-turn results on ToolAlpaca, RoTBench, and BFCL-v3 are more credible because those test sets use different task distributions from the training set. However, the multi-turn ToolEyes comparison, which is load-bearing for the claim of matching closed-source models on multi-turn tool use, is currently weakened by the lack of reported train/test disjointness and by small test sizes with single-run evaluations. The reward function is also constructed from the same error categories used in the evaluation metrics, so the multi-turn conclusions require additional evidence of generalization.","major_comments":[{"comment":"The ToolEyes multi-turn result is not supported as a generalization result unless the training and test sets are known to be disjoint. Appendix B states that the 1,217 training trajectories are 'artificially create[d]' using ToolEyes as the tool foundation, while Table 2 lists the same ToolEyes benchmark (382 multi-turn cases) as the test set. The paper never reports a query-level or tool-level overlap check, deduplication, or any exclusion of ToolEyes test instances from the generated training data. If even a modest fraction of the 382 test queries appear in the training set, exactly or through similar tool-call sequences, the Table 4 numbers (DE 0.82, CE 4.84, VA 77.75) and the claim of matching GPT-4o on multi-turn tool use are inflated by memorization rather than reflecting genuine tool-use ability. Please provide an explicit overlap analysis between the 1,217 training trajectories and the 382 ToolEyes test cases, or evaluate the multi-turn claim on a held-out set built from different tools.","section":"§5.1, Appendix B, Table 4"},{"comment":"The reward function in Section 4.3 is built from the same error taxonomy that defines the evaluation metrics in Section 5.3: tool hallucination, parameter hallucination, missing or redundant parameters, and content filling issues. The PPO stage is therefore trained on a proxy that is closely aligned with the benchmark scoring rules, so the large gains from the IRM component in Table 5 (e.g., ToolAlpaca CF from 42.98 to 57.02 when IRM is added) may partly reflect fitting the model to the benchmark rather than improving general tool-use competence. This concern is especially relevant because the same trajectory set used for SFT is reused for PPO. To support the claim that the reward mechanism improves general tool use, please report performance on a held-out tool suite whose error categories or tool distributions differ from the reward taxonomy, or explicitly analyze how much of the gain is attributable to metric-aligned reward shaping.","section":"§4.3, §5.3"},{"comment":"Several headline comparisons rest on small absolute differences over very small test sets: RoTBench has 105 test instances, ToolAlpaca 114, BFCL-v3 239, and ToolEyes 382. For example, on RoTBench CF, TL-CodeLLaMA-2 (42.86) is 7.62 points above the average, which corresponds to about 8 of the 105 examples. With a single evaluation run using greedy decoding and no confidence intervals, bootstrap estimates, or multiple seeds, the paper's claim that the method 'matches or surpasses' state-of-the-art models is stronger than the reported evidence supports. Please add variance estimates, statistical tests, or multiple-seed training results for the main comparisons and the ablation table.","section":"Tables 2, 3, 5"}],"minor_comments":[{"comment":"The KL coefficient beta in the PPO objective in Section 4.3 is not reported in Section 5.4, and no sensitivity analysis is given for wmax (set to 9) or for the reward weights in Section 4.3, so the configuration is incompletely specified.","section":"§5.4"},{"comment":"The Section 4 footnote claims 'theoretical proofs of the effectiveness' are provided, but Theorems A.1 and A.2 restate the loss definitions and show that gradients scale with weights, which is true by construction; they do not establish that the proposed changes improve downstream tool-use performance. The theoretical-justification claim should be softened or replaced with a substantive analysis.","section":"Appendix A, footnote 1"},{"comment":"There are minor language errors: 'maintaines' in Section 6.1 should be 'maintains', and 'a reward mechanism tailed for tool use' in Section 8 should be 'tailored for tool use'.","section":"§6.1, §8"}],"recommendation":"major_revision","confidential_remarks":"The ToolEyes overlap issue is the main gate for this paper. If the authors can demonstrate train/test disjointness, or if they re-evaluate the multi-turn claim on a held-out set, the paper could be publishable after revision; the single-turn results are more credible and would survive the overlap concern."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The gist: this paper offers a genuinely efficient training recipe for tool-use LLMs. On 1,217 GPT-4o-generated trajectories, a 7B CodeLLaMA-2 model beats most open baselines and even GPT-4-turbo on several single-turn metrics. The three components—masking erroneous trajectories, reweighting key tokens, and a category-based reward for PPO—are individually familiar, but the combination and the accompanying analysis of error types are new. The ablation table supports each component's contribution, and the single-turn results on ToolAlpaca, RoTBench, and BFCL-v3 look solid enough to take seriously.\n\nThe soft spots are real but localized. First, the multi-turn ToolEyes result is burdened by an unaddressed train/test overlap risk: the training data is built using ToolEyes as the tool foundation, and ToolEyes is also the multi-turn test. The paper never says whether the 1,217 training queries are disjoint from the 382 test queries. This matters because the multi-turn claim (second-best error rate behind GPT-4o) is headline material. If there is any leakage, that number is inflated. The single-turn results are unaffected, but the authors need to release or describe the data generation and show disjointness.\n\nSecond, the appendix 'proofs' are tautological restatements of the loss definitions. They don't prove effectiveness and should be cut or reframed as intuition. Third, all tables are single-run; with test sets as small as 105 examples, a few points of difference could be noise. The authors should provide variance or at least flag this. The reward design is aligned with the evaluation categories, which is a reasonable choice but means the RL gains could partly reflect overfitting to the benchmark's taxonomy; the fact that single-turn results generalize across three different benchmarks mitigates this.\n\nNone of this undermines the core idea. The paper is a solid empirical contribution to data-efficient tool-use training, and the central claim—that a compact dataset plus task-aware losses can rival large-scale training—is plausible and worth testing. I'd send it to a serious peer review, with referees asked to push on the ToolEyes overlap and variance reporting.","headline":"Data-efficient tool-use training framework with strong single-turn results; the multi-turn ToolEyes claim needs a train/test overlap check and the appendix proofs are tautological.","tokens_in":18660,"tokens_out":5006,"would_cite":true,"duration_ms":38649,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a three-part task-feature training recipe lets a 7B model match or surpass leading open- and closed-source LLMs on tool-use benchmarks using only 1,217 training examples.","keywords":["tool learning","large language models","supervised fine-tuning","reinforcement learning","proximal policy optimization","token weighting","data efficiency","function calling"],"falsifier":"Retrain TL-CodeLLaMA-2 on 1,217 trajectories built from a disjoint set of tools (not the ToolEyes suite) and run the same four test sets; if the ToolEyes multi-turn scores (DE 0.82, CE 4.84, VA 77.75) drop sharply while single-turn scores hold, the original multi-turn result was inflated by training-test overlap. Alternatively, compute the fraction of ToolEyes test queries whose tool names or API-call patterns appear in the training set; a non-negligible overlap would settle the concern.","tokens_in":17591,"feed_emoji":"🔧","tokens_out":5762,"duration_ms":44526,"temperature":0.7,"pith_summary":"TL-Training is a three-part recipe for teaching a large language model to call external tools, and the paper claims that the recipe lets a 7-billion-parameter model trained on only 1,217 interaction traces match or beat much larger open- and closed-source models on four tool-use benchmarks. The first part inspects tool feedback to detect erroneous tool calls inside training trajectories and blocks those steps from gradient updates. The second reweights the training loss so that tokens that are most diagnostic for choosing the right tool, such as the first token of a tool name, get larger gradient contribution. The third defines a reward function keyed to a small taxonomy of tool-call errors and tunes the model with PPO. If the claim holds, data-efficient, task-aware training can substitute for massive tool-use datasets.","feed_headline":"Small model trained on 1,217 examples tops tool-use benchmarks","feed_subtitle":"Error masking, key-token weighting, and error-based rewards close the gap with GPT-4.","key_machinery":"The load-bearing mechanism is a training pipeline with three training modifications. The first, adverse-effects mitigation (MAE), parses tool feedback such as 'name x is not defined' or 'missing required positional argument' to detect erroneous steps and excludes them from the SFT loss, preventing the model from learning bad calls. The second, key-token prioritization (PKT), classifies tokens of each tool name into key and non-key sets (key tokens include the first token and any token whose prefix is shared with another tool) and multiplies their SFT loss weight by $w = \\mathrm{CLIP}(|NK_i|/|K_i|, 1, w_{\\max})$, so the model pays more gradient attention where tool names are easy to confuse. The third, the reward mechanism, maps a generated tool call to one of five categories—parse failure, tool hallucination, wrong tool, parameter issues, content filling—with rewards from $-2$ to $1$, and optimizes the model with PPO against a KL penalty that keeps it near the SFT model. Each component is ablated and each contributes; the full stack outperforms standard SFT on the four test sets.","core_discovery":"The paper's central discovery is that tool-use training is bottlenecked not by data quantity but by task-specific characteristics that standard SFT ignores, and that correcting for these characteristics unlocks state-of-the-art behavior from a small model. Analyzing existing tool-use LLMs, the authors find that (i) about 17% of the trajectories in an existing training set contain tool-calling errors, and training on them teaches the model to repeat those errors; (ii) wrong tool selections often differ from correct ones only in an early token, so the first tokens and shared-prefix tokens carry disproportionate importance; and (iii) tool-call errors collapse into a small set of categories visible in tool feedback. TL-Training operationalizes these insights: MAE masks erroneous interaction paths from back-propagation, PKT upweights key tokens by a clipped ratio of non-key to key tokens, and IRM assigns graded rewards for different error types during PPO. Trained on 1,217 GPT-4o-generated multi-turn trajectories over ToolEyes tools, the resulting 7B model, TL-CodeLLaMA-2, matches or surpasses GPT-4-class and specialized tool-use models on ToolAlpaca, RoTBench, BFCL-v3, and ToolEyes, while improving robustness to noisy tool descriptions and preserving general task performance.","pith_inferences":["If the training-test overlap concern is resolved, the same three components should transfer to other structured-generation tasks such as SQL generation, API calls, and robotic control, where a small taxonomy of errors is available from environment feedback.","The key-token weighting rule is parameter-light and could be adapted to other domains by defining key tokens from confusion statistics rather than tool-name prefixes; this is a natural testable extension the paper does not pursue.","Because the reward function is hand-crafted from a fixed error taxonomy, a learned reward model could capture rare error types; the paper notes it does not train one, so this is an open extension.","The claim that compact task-aware data can substitute for large datasets suggests that dataset quality engineering may matter more than scale for tool use, and the paper's ablation structure gives a template for testing that hypothesis."],"forward_implications":["A 7B model with 1,217 training trajectories can match or beat much larger and heavily trained models on single-turn tool selection, parameter identification, and content filling across ToolAlpaca, RoTBench, and BFCL-v3.","On multi-turn tool use, TL-CodeLLaMA-2 achieves a low total error rate on ToolEyes and the highest valid-answer rate among all compared open-source models, while remaining competitive with GPT-4-class closed models.","The model is more robust than a model specifically optimized for noisy tool environments, reaching or exceeding RoTLLaMA on RoTBench's noisy splits without targeted noise augmentation.","General knowledge and code/math capabilities are preserved (MMLU, GSM8K, HumanEval) after tool-use fine-tuning, with slight gains on math and code, so the method does not trade away general ability.","Each of the three components contributes to the final performance in ablations; masking erroneous trajectories alone cuts the multi-turn tool-call error rate by about one-third."],"supporting_citations":[{"why":"Supplies ToolLLaMA, the large-scale SFT baseline whose performance bottleneck motivates the method.","marker":"(Qin et al., 2024)"},{"why":"Supplies RoTBench and RoTLLaMA; the 17% training-data error statistic and the noisy-environment comparison come from this line.","marker":"(Ye et al., 2024b)"},{"why":"Supplies ToolAlpaca, one of the single-turn test sets used for evaluation.","marker":"(Tang et al., 2023)"},{"why":"Supplies BFCL-v3, the function-calling benchmark used as another single-turn test set.","marker":"(Patil et al., 2024)"},{"why":"Supplies ToolEyes, both the tool foundation for constructing the training data and the multi-turn test set with its evaluation metrics.","marker":"(Ye et al., 2025)"},{"why":"Supplies CodeLLaMA-2-7B, the base model that is fine-tuned into TL-CodeLLaMA-2.","marker":"(Rozière et al., 2023)"},{"why":"Supplies the PPO algorithm used in the reinforcement-learning stage.","marker":"(Schulman et al., 2017)"}],"fun_headline_variants":["7B model trained on 1,217 examples matches GPT-4 on tool-use benchmarks","1,217 examples: TL-Training 7B model rivals GPT-4 in tool use","Task-feature training turns 1,217 examples into GPT-4-level tool use","Error masking and rewards: 1,217 examples achieve tool-use parity with GPT-4","How to train a 7B tool-use model with 1,217 samples to match GPT-4"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The multi-turn result on ToolEyes rests on the assumption that the 1,217 training trajectories constructed from ToolEyes tools do not overlap with the ToolEyes test queries; if the same tools or usage patterns appear in both, the reported gains partly reflect memorization rather than generalization, and the paper does not state the disjointness.","fun_headline_variants_meta":{"raw":{"variants":["7B model trained on 1,217 examples matches GPT-4 on tool-use benchmarks","1,217 examples: TL-Training 7B model rivals GPT-4 in tool use","Task-feature training turns 1,217 examples into GPT-4-level tool use","Error masking and rewards: 1,217 examples achieve tool-use parity with GPT-4","How to train a 7B tool-use model with 1,217 samples to match GPT-4"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000419,"raw_usage":{"total_tokens":2219,"prompt_tokens":1067,"completion_tokens":1152,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":683,"completion_tokens_details":{"reasoning_tokens":1033}},"tokens_in":683,"tokens_out":1152,"duration_ms":9357,"temperature":1.0,"reasoning_tokens":1033,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T11:22:21.642031+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain TL-CodeLLaMA-2 on 1,217 trajectories built from a disjoint set of tools (not the ToolEyes suite) and run the same four test sets; if the ToolEyes multi-turn scores (DE 0.82, CE 4.84, VA 77.75) drop sharply while single-turn scores hold, the original multi-turn result was inflated by training-test overlap. Alternatively, compute the fraction of ToolEyes test queries whose tool names or API-call patterns appear in the training set; a non-negligible overlap would settle the concern.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies ToolLLaMA, the large-scale SFT baseline whose performance bottleneck motivates the method."}],"review_version":1}