{"id":"f9f2e761-5c86-4162-9019-330b89cdb302","arxiv_id":"2412.17365","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"IterIT iteratively re-scores instruction samples during fine-tuning and greedily selects a small, diverse, high-complexity subset each epoch.","lead":"Instruction tuning can work better with less data if the data is chosen well. This paper proposes a selection method that repeatedly re-scores training samples during fine-tuning, using the model's own difficulty and diversity estimates, and claims it beats full-data training with only 5% of data per epoch.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Alpaca-GPT4 is run with b=0.0 even though Sec. 5.4 reserves b=0.0 for task-specific data; the missing ablation leaves the headline gain on that dataset open to a hyperparameter-tuning explanation.","rationale":"The paper's central contribution is that iterative model-updated complexity plus response diversity beats full SFT and static baselines at 5% data per epoch. The most load-bearing assumption is that the reported advantage is due to that mechanism and not to benchmark-specific tuning. The b choice is the clearest place where this could break: it is a free parameter, set differently for one dataset without following the paper's own guidance, and no ablation covers that dataset. The reader's weakest_assumption identified the same issue; my read agrees. A single changed-configuration rerun would settle whether the concern lands. My verdict remains CONDITIONAL, same as the reader; no change in final recommendation.","tokens_in":15234,"tokens_out":11781,"duration_ms":114366,"concrete_test":"Re-run the Alpaca-GPT4 condition in Table 1 with b=0.1 instead of 0.0, keeping M=0.05N, a=3, batch size 32, and 3 epochs, and repeat with 3-5 random seeds for IterIT and GraphFilter. Compute the seven-benchmark average and MixEval. If IterIT with b=0.1 falls below GraphFilter's AVG of 61.45 or below Longest's MixEval of 40.60, the headline improvement on Alpaca-GPT4 is not robust to a hyperparameter choice the paper itself recommends for general instruction data; if it stays above both, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Sec. 4.1 rests on Table 1, including Alpaca-GPT4 (AVG 62.20, MixEval 40.15). Sec. 3.4 sets b=0.0 for Alpaca-GPT4 while using b=0.1 for the other general instruction datasets, and Sec. 5.4 states b=0.0 is recommended only for task-specific scenarios where reserved words are undefined or shared. Alpaca-GPT4 is a general instruction dataset, so the stated rule does not cover this choice and no dataset-specific justification is given. Since b changes the greedy diversity update (Eq. 7) and therefore the selected subset, the Alpaca-GPT4 result could reflect post-hoc selection of b rather than the proposed algorithm. This matters for the strongest claim: on Alpaca-GPT4, IterIT loses to Longest on 5 of the 9 reported columns (TruthfulQA, HumanEval, ARC, HellaSwag, MixEval), so 'beats Longest on most metrics' is already not true there, and the entire case rests on the average score. Without an ablation at b=0.1 or repeated seeds, the robustness of that average is unverified. This is the weakest link in the empirical support.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IterIT, an iterative data selection method for instruction tuning. It combines an IFD-based complexity score that is re-computed after each fine-tuning epoch on a coarse-to-fine candidate pool, with a response-level TF-IDF diversity score that is greedily updated via a weight-decay parameter b. On LLaMA-3-8B and Qwen-2.5-7B, using Alpaca, Alpaca-GPT4, WizardLM, Dolly, and CodeAlpaca, the authors report that selecting 5% of the data per epoch yields consistent improvements over full-data SFT (Vanilla) and over static baselines including Longest, Deita, Superfiltering, and GraphFilter, across seven benchmarks plus MixEval.","tokens_in":15564,"tokens_out":8684,"duration_ms":72012,"significance":"If the findings are robust, IterIT is a practically useful contribution: it provides a model-aware, iterative selection procedure that jointly optimizes complexity and response diversity, and it is evaluated under a unified re-implementation of baselines on multiple datasets and two backbone models. The paper includes useful ablations (iteration on/off, diversity component, hyper-parameter sensitivity) and promises to release code. The central premise—that model-specific complexity should be updated during fine-tuning rather than computed once—is interesting and supported by the reported 55.31% turnover of top samples after one epoch. However, the current empirical evidence contains several load-bearing gaps that must be addressed before the claims can be accepted.","major_comments":[{"comment":"The choice of b=0.0 for Alpaca-GPT4 is inconsistent with the paper's own recommendation. Sec. 3.4 sets b=0.0 for Alpaca-GPT4, while Sec. 5.4 states that b=0.0 is recommended only for task-specific scenarios where reserved words are undefined or shared. Alpaca-GPT4 is a general instruction dataset, so the stated rule does not cover the choice, and no dataset-specific justification is given. Because b controls the greedy diversity update in Eq. (7) and therefore directly affects the selected subset, the Alpaca-GPT4 results in Table 1 (AVG 62.20 vs. GraphFilter 61.45) could reflect per-dataset tuning rather than the algorithm's intrinsic behavior. Please provide an ablation of b on Alpaca-GPT4 (e.g., b=0.1) or a data-driven argument for why b=0.0 is appropriate for this dataset.","section":"Sec. 3.4 / Sec. 5.4"},{"comment":"The Qwen-2.5-7B results for Alpaca-GPT4 and WizardLM appear to be swapped between Table 3 and Table 7 in Appendix B. For example, the Vanilla AVG/MixEval values for Alpaca-GPT4 are 68.96/40.15 in Table 3 but 69.17/38.20 in Table 7, and the corresponding WizardLM values are exchanged in the same way for all three methods. This is a factual inconsistency in a table that directly supports the generalization claim in Sec. 4.3. Please correct the tables and verify that the underlying experiments are reported consistently.","section":"Table 3 / Appendix B"},{"comment":"The paper reports only single-run point estimates without error bars, multiple seeds, or statistical significance tests. Many of the reported gains are small—for instance, WizardLM AVG 60.29 vs. Longest 59.75 (Table 1), Dolly AVG 58.82 vs. Longest 59.14 (Table 6), and Qwen Alpaca AVG 66.43 vs. Longest 66.49 (Table 3), the latter being a loss on the average. Without variance estimates or repeated runs, the claimed 'consistent improvements' are not empirically established. Please report at least three seeds with mean and standard deviation, or otherwise justify that the differences exceed run-to-run noise.","section":"Section 4 results"},{"comment":"The statement that IterIT 'beats the strongest rule-based approach, Longest, on most of the metrics' is not supported by the tables across all datasets. On Alpaca-GPT4 (Table 1), IterIT loses to Longest on TruthfulQA, HumanEval, ARC, HellaSwag, and MixEval (5 of 9 columns). On Dolly (Table 6), IterIT loses on MMLU, TruthfulQA, BBH, HellaSwag, and the AVG itself. The claim should be qualified to 'on the aggregate average in most datasets' or should be reported per-dataset with exact win/loss counts.","section":"Sec. 4.1"},{"comment":"The text claims that setting b=0.1 improves GSM8K, BBH, and HumanEval by 2.50%, 2.59%, and 4.76%, respectively, compared with b=0.0. However, these per-benchmark numbers are not shown in Fig. 4(c) (which plots only the average score) or in any table. Because the choice of b is central to the method and the paper already reports per-task results elsewhere, please provide the full per-benchmark table for the b ablation on Alpaca, or explicitly state where these numbers come from.","section":"Sec. 5.4"}],"minor_comments":[{"comment":"The output line reads 'return M_{θ_{T−1}}'; after looping over T epochs, the final model should be M_{θ_T}. Please correct the subscript.","section":"Algorithm 1"},{"comment":"The column header 'A VG' appears with a space in several tables; this should be 'AVG'.","section":"Tables 1, 6, 7"},{"comment":"The caption writes 'I TER IT' with an extra space. Please fix the typo.","section":"Figure 1"},{"comment":"The sentence 'data selection methods can easily achieve significant improvement in a single task when the selected data is closer to the distribution of a specific group of test data' is a speculative claim that is not substantiated by any controlled experiment in the paper. Please either remove it or support it with evidence.","section":"Sec. 4.1"},{"comment":"The appendix table for Qwen-2.5-7B should be cross-checked with Table 3; in addition to the dataset-row swap noted above, several MixEval values are identical between datasets (e.g., 40.15 appears for both Alpaca-GPT4 Vanilla and WizardLM Vanilla in different tables), which is suspicious and should be verified.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper includes MixEval (Ni et al., 2024) as an evaluation benchmark, and two of the present authors (Jinjie Ni and Yang You) are co-authors of that work. This is not a technical flaw, but given the paper's reliance on MixEval results in several tables, the authors should explicitly acknowledge the overlap in the evaluation section for transparency. The swapped rows between Table 3 and Appendix Table 7 suggest a preparation error that should be caught during revision; I would recommend the editor ask for a full numerical verification of all reported tables. The central idea is worth publishing if the empirical gaps are addressed with additional seeds and the b-hyperparameter justification is fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the iterative re-scoring of a model-specific complexity score during SFT, plus response-level TF-IDF diversity with weight decay, is a real departure from the static-selection baselines (Deita, Superfiltering, GraphFilter). The ablations back the two design choices: removing iteration consistently hurts, and response-based diversity beats instruction-based or no diversity. The Qwen and CodeAlpaca results also help.\n\nThat's the good news. The empirical case is shakier than the text admits. Table 1 shows IterIT loses to Longest on 5 of 9 columns on Alpaca-GPT4 (TruthfulQA, HumanEval, ARC, HellaSwag, MixEval), so 'beats Longest on most metrics' is false there; the win is carried by the average. And that average rests on b=0.0 for Alpaca-GPT4, while Sec 5.4 recommends b=0.0 only for task-specific data with shared reserved words. Alpaca-GPT4 is general instruction data, so the choice looks post-hoc rather than principled. Since b changes the greedy diversity update, this is the load-bearing weak point. An ablation at b=0.1 on Alpaca-GPT4 would settle it.\n\nAlso, all results are single-run, no error bars, and the margins are often 1-2 points. At that noise level, 'consistent improvements' is an overstatement. The paper promises code on GitHub but I don't see a release in the text besides the abstract's 'will be available'; given the field's reproducibility norms, that should be a condition.\n\nThe circularity flags are minor: MixEval is co-authored by two of the authors, but the benchmark is public and the other benchmarks are standard. The per-dataset b is a bigger issue.\n\nNet: this is a serious, novel method for a busy subarea, with good ablations, but the headline claim needs tempering and the b choice needs justification or ablation. I'd send it to review, not desk-reject it. The right outcome is a revision with error bars, a b ablation, and code release.","headline":"A plausible new iterative data-selection method with good ablations, but the consistency claim overstates the evidence and the Alpaca-GPT4 b=0.0 choice looks post-hoc.","tokens_in":16070,"tokens_out":2443,"would_cite":false,"duration_ms":21680,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training an LLM on only 5% of an instruction dataset per epoch, with scores recomputed as the model changes, out-performs full-data fine-tuning on average benchmark scores.","keywords":["iterative data selection","instruction tuning","complexity-diversity tradeoff","IFD score","TF-IDF diversity","model-data collaboration","LLM post-training","data quality"],"falsifier":"A decisive test is to rerun IterIT on Alpaca-GPT4 with $b=0.1$ (the value the paper recommends for multi-task general data) and check whether the average score still surpasses Longest and GraphFilter. If the gain disappears or reverses, the Alpaca-GPT4 result is an artifact of choosing $b$ after seeing the test set.","tokens_in":15059,"feed_emoji":"🎯","tokens_out":7964,"duration_ms":64625,"temperature":0.7,"pith_summary":"The paper argues that the right subset of instruction-tuning data is not a fixed precomputed set but something that should be re-derived as the model changes. It introduces IterIT, an algorithm that before each fine-tuning epoch re-scores a candidate pool with a model-specific complexity metric (the instruction-following-difficulty ratio) and a response-based diversity metric with weight decay, then greedily picks the highest-scoring samples. On Alpaca, Alpaca-GPT4, WizardLM, and Dolly, models trained on just 5% of the data per epoch under this schedule outperform full-data fine-tuning and edge out the strongest rule-based baseline (Longest) on most average metrics. If true, the finding makes data selection a live, low-cost component of LLM post-training rather than a one-shot preprocessing step.","feed_headline":"Iterative data selection beats full-data fine-tuning at 5% per epoch","feed_subtitle":"Recomputing scores during fine-tuning lets 5% of data beat full-dataset SFT.","key_machinery":"The central object is the composite score $S_i = S^{COM}_{i,\\theta_t} \\times S^{DIV}_i$, where $S^{COM}_{i,\\theta_t}$ is the ratio of the perplexity of response $Y_i$ conditioned on instruction $X_i$ to the perplexity of $Y_i$ alone (the IFD score, a model-specific complexity measure), and $S^{DIV}_i$ is the sum of TF-IDF weights of n-grams in $Y_i$ with a decay factor $\\alpha_g$ that multiplies by $b$ each time a sample containing n-gram $g$ is selected. The algorithm operates in epochs: at epoch $t$ it computes $S^{COM}$ for the current model over the retained candidate pool (the top $a \\times M$ samples from the initial ranking), drops samples with $S^{COM} \\ge 1$, and greedily fills the epoch's $M$-sample budget by repeatedly picking the largest $S_i$ and decaying $\\alpha$ for the chosen n-grams. This turns data selection into a model-data feedback loop instead of a one-time filtering step.","core_discovery":"IterIT demonstrates that updating data-selection scores during fine-tuning improves instruction-tuned LLMs. The paper reports that re-ranking a coarse-to-fine candidate pool with the instruction-following-difficulty (IFD) score after every epoch — then greedily selecting samples that maximize the product of IFD complexity and a response-derived TF-IDF diversity score with decaying weights — beats Vanilla full-data SFT on average across seven benchmarks plus MixEval on four instruction datasets, and beats Longest, the strongest rule-based baseline, on most metrics, while using only 5% of the data at each epoch. The method also generalizes to CodeAlpaca code generation and to a different backbone, Qwen-2.5-7B.","pith_inferences":["Editorially, the same iterative re-scoring loop could be applied to later post-training stages such as preference optimization, where the model's reward or policy changes across epochs and static preference data may quickly become stale.","The finding that short-response samples remain valuable (the lower quartile of lengths is much lower than Longest's) suggests that sample value is relative to the model's current competence frontier, which shifts during training; a length-based static ranking cannot track that frontier.","One testable extension is to swap the TF-IDF diversity score for an embedding-based semantic diversity measure inside the same loop; if gains persist, the method would be robust to the lexical proxy, and if not, the lexical coverage mechanism is doing the work.","Also implicit is a recipe for noisy or multi-source synthetic data: iterative ranking with a model-specific difficulty filter (the $S^{COM} \\ge 1$ cutoff) may serve as a general de-noising step for post-training corpora beyond instruction tuning."],"forward_implications":["Instruction-tuned models improve when the selection metric is recomputed with the partially trained model, not a fixed pretrained scorer.","Training on roughly 5% of the dataset per epoch can exceed full-data SFT on average multi-benchmark performance, cutting per-epoch gradient cost.","Response-based diversity, not instruction-topic diversity, is what makes the diversity term effective in this pipeline.","The approach carries over to code instruction tuning and to a second backbone model (Qwen-2.5-7B), suggesting the mechanism is not dataset- or architecture-specific.","The 'long is more' heuristic is strong but incomplete: IterIT's selected sets overlap Longest's by only about 50% Jaccard similarity while outperforming it on most metrics."],"supporting_citations":[{"why":"Supplies the IFD complexity score (ratio of conditional to prior response perplexity) that IterIT uses as its model-specific complexity measure.","marker":"(Li et al., 2024b)"},{"why":"Supplies the TF-IDF-based diversity score on instructions that IterIT adapts to responses, also defining the GraphFilter baseline.","marker":"(Wu et al., 2024a)"},{"why":"Introduces the 'long is more' baseline (Longest) that IterIT must beat, showing that simple length-based selection is a tough competitor.","marker":"(Zhao et al., 2024)"},{"why":"Co-defines the Longest baseline and argues for longest-response selection, anchoring the main rule-based comparison.","marker":"(Shen, 2024)"},{"why":"Deita, a learned-scorer baseline that represents the static 'score once, select once' approach that IterIT improves on.","marker":"(Liu et al., 2024b)"},{"why":"Provides the superficial-alignment finding that a few carefully chosen samples can suffice, motivating the whole data-selection direction.","marker":"(Zhou et al., 2024)"},{"why":"The Alpaca dataset, a primary testbed for the instruction-tuning experiments.","marker":"(Taori et al., 2023)"},{"why":"The WizardLM dataset of evolved instructions, testing the method on a more complex, synthetic-data regime.","marker":"(Xu et al., 2023)"}],"fun_headline_variants":["IterIT: 5% data, re-ranked each epoch, beats full fine-tuning","Dynamic complexity scores let 5% of data outdo full SFT","Iterative re-ranking: 5% data outperforms full dataset","Model-aware selection: 5% per epoch beats full tuning","Update data scores every epoch; 5% beats full fine-tune"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the weight-decay coefficient $b$ (set to 0.1 for most datasets and 0.0 for Alpaca-GPT4) is chosen by the stated rule for sharing of reserved words, not by looking at the test benchmarks; if $b=0.0$ was tuned on Alpaca-GPT4 results, the reported gains there may not replicate.","fun_headline_variants_meta":{"raw":{"variants":["IterIT: 5% data, re-ranked each epoch, beats full fine-tuning","Dynamic complexity scores let 5% of data outdo full SFT","Iterative re-ranking: 5% data outperforms full dataset","Model-aware selection: 5% per epoch beats full tuning","Update data scores every epoch; 5% beats full fine-tune"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001045,"raw_usage":{"total_tokens":4359,"prompt_tokens":879,"completion_tokens":3480,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":3384}},"tokens_in":495,"tokens_out":3480,"duration_ms":27719,"temperature":1.0,"reasoning_tokens":3384,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:32:50.596715+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test is to rerun IterIT on Alpaca-GPT4 with $b=0.1$ (the value the paper recommends for multi-task general data) and check whether the average score still surpasses Longest and GraphFilter. If the gain disappears or reverses, the Alpaca-GPT4 result is an artifact of choosing $b$ after seeing the test set.","supporting_citations":[],"review_version":1}