{"id":"83228f52-0f1e-4bc3-89db-5de04eb3c8d3","arxiv_id":"2411.14497","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A multi-agent LLM framework that generates, selects, and evolves instruction-tuning data improves downstream instruction-following by about 12% over Evol-Instruct baselines.","lead":"The paper proposes Star-Agents, a multi-agent framework that rewrites instruction-tuning data with several LLMs, scores each candidate with a dual-model difficulty metric, and evolves which agents get used. On Pythia-1B and Llama-2-7B, fine-tuned models trained on the optimized data beat the Evol-Instruct baseline by about 12% on average across MT-bench, Vicuna-bench, and WizardLM testset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dual-model selection depends on an unverified IFD monotonicity assumption (Eq. 4–5); if it fails, the 'tailored complexity' scores are not meaningful and the reported 12%/40% gains may not transfer. The large model used for IFD is also unspecified.","rationale":"The paper's central claim is empirical: LLMs fine-tuned on Star-Agents-optimized data outperform the same LLMs fine-tuned on Evol-Instruct data, with roughly 12% average improvement and up to 40% on Fermi tasks. The reported results are internally consistent, with gains at two model scales, ablation results in Table 3, and a comparison against IFD and random selection in Table 4. Those results give real support to the claim, conditional on reproducibility. However, the framework's distinctive design is the dual-model difficulty criterion, and that criterion relies on an assumption about IFD monotonicity across model sizes that is neither derived nor experimentally checked. This is exactly the reader's weakest assumption, and it is load-bearing: if the assumption is false for a substantial fraction of candidates, then the selection score does not measure what Figure 3 and the text claim, and the method's success would have to be explained by some other factor (e.g., the LLM quality filter or the evolution mechanism). The missing specification of the large model makes the assumption untestable from the paper alone, and the absence of error bars or multiple seeds means the reported margins cannot distinguish a robust effect from run-to-run variance. A direct computational check of IFD monotonicity and its correlation with downstream utility would settle whether the concern lands. Since the reader already conditioned acceptance on verification and clarification, this stress-test does not change the verdict; it sharpens the specific condition that must be verified.","tokens_in":16152,"tokens_out":6641,"duration_ms":65447,"concrete_test":"Compute IFD scores with Pythia-1B and a fixed, disclosed large model (e.g., Llama-2-7B or Qwen-14B-Chat) on 1,000 candidate samples drawn from the Star-Agents generation step. Report (a) the fraction of samples with IFD_large < IFD_small, and (b) the per-sample correlation between (IFD_small − IFD_large) and the downstream improvement obtained by adding that sample to the fine-tuning set. If the fraction is not close to 1 or the correlation is not positive and significant, Eq. 5's tailored-difficulty interpretation fails and the central mechanism is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of Star-Agents is the dual-model evaluation in Section 3.2. Equation 5 scores a candidate by the normalized gap IFD_small(I,R) − IFD_large(I,R), and the paper interprets a large gap as evidence that the sample is difficult for the target model but still within reach of a stronger model. This interpretation rests entirely on the unproven statement after Eq. 4: 'We assume that for the same sample, stronger model yields a smaller IFD score.' IFD is a ratio of two perplexities (conditional over unconditional); a stronger model typically reduces both numerator and denominator, so the ratio need not decrease monotonically with model strength. No evidence for the assumption is provided, and the paper never states which model computes IFD_large: Section 4.1 lists generator LLMs but not the evaluator. If monotonicity fails on a meaningful fraction of candidates, then high πdual can select samples that are not in the intended difficulty corridor, and the claimed advantage of the selection method—and its transferability to other target models—is unsupported. The empirical tables show consistent gains, but they are single runs with no error bars or seeds, so they cannot by themselves arbitrate this assumption. This is the most load-bearing concern because the dual-model score is the component that distinguishes Star-Agents from generic multi-agent generation plus LLM filtering.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Star-Agents, a framework for automatically constructing instruction-tuning data tailored to a target LLM. It has three stages: (i) generating diverse candidate instruction-response pairs by sampling 'agent-pairs' from a pool of LLMs; (ii) scoring each candidate with π = πllm · πdual, where πllm is an LLM-judge quality comparison against a base sample and πdual is the normalized gap between the IFD difficulty scores of the target (small) model and a larger model; and (iii) evolving the agent-pair sampling probabilities based on these scores, with an Instruction Memory Bank for task-specific retrieval. The authors fine-tune Pythia-1B and Llama-2-7B on the Evol-Instruct dataset and a Star-Agents-refined version ('Star Instruct'), and report that the latter improves average scores on MT-Bench, Vicuna-bench, and WizardLM testset by about 12% (absolute ~0.45 points), with up to 40% relative gains on Fermi tasks. Ablations isolate the contributions of diversity, selection, and evolution, and compare the selection metric against IFD and random selection.","tokens_in":16446,"tokens_out":5716,"duration_ms":51171,"significance":"If the reported gains hold, Star-Agents provides a practical, automatic alternative to manual and single-model data curation, and its multi-agent generation and evolutionary selection are plausible contributors. The paper includes a reasonable ablation structure, comparisons against IFD and random baselines, and a small Open LLM Leaderboard check, which are commendable. However, the central dual-model scoring mechanism rests on an unverified monotonicity assumption about IFD, and the empirical evidence consists of single runs without variance reporting. The core idea is interesting and likely reproducible, but the evidence as presented is not sufficient to establish the 12%/40% claims.","major_comments":[{"comment":"The dual-model score πdual is the core novelty, yet it rests on the unproven assumption stated after Eq. (4) that 'stronger model yields a smaller IFD score'. Since IFD is a ratio of conditional to unconditional perplexity, and model strength typically reduces both, monotonicity is not guaranteed. No empirical evidence is provided for this assumption, and the paper never specifies which model serves as the 'large' model for IFD_large (Section 4.1 lists generator LLMs but not the evaluator). If the assumption fails on a meaningful fraction of candidates, the selected samples will not lie in the intended difficulty corridor, and the claimed advantage of the dual-model metric over plain IFD is unsupported. Please verify the assumption (e.g., report the share of samples where IFD_large < IFD_small across model pairs) and identify the large model.","section":"Section 3.2, Eqs. (4)-(5)"},{"comment":"All reported improvements are based on single fine-tuning runs and a single GPT-4 judging pass; no error bars, multiple seeds, or variance estimates are provided. The average gains (e.g., 4.17 vs 3.73 in Table 2) are small relative to the known variability of LLM-as-a-judge scores, so the headline claims of 12% and 40% are not yet statistically grounded. Please report means and standard deviations over at least three seeds for the main comparisons and include a human-evaluation subset or judge-agreement statistics.","section":"Tables 2-4, Section 4.2"},{"comment":"The IFD and Random baselines are underspecified. It is not clear which candidate pool they select from (the same Star-Agents-generated D(Si) or the original Evol-Instruct data), how many samples are selected per seed (one per seed, as Eq. (1) suggests, or a filtered subset), and what the final dataset sizes are. Without this information, Table 4 cannot be interpreted as a controlled comparison of selection metrics. Please describe the exact selection protocol for each baseline and report the resulting dataset sizes.","section":"Section 4.1 and Table 4"}],"minor_comments":[{"comment":"The word 'Finaly' should be 'Finally'.","section":"Abstract"},{"comment":"Equation (1) writes arg max over π(D(Si)) rather than π(d), and Eq. (5) uses 'max_{1≤i≤m}' without defining the index set; please fix the notation.","section":"Eq. (1) and Eq. (5)"},{"comment":"The paper states '10 agent-pairs were employed' but does not specify the values of M, β, l, and n from Section 3; please provide these hyperparameters for reproducibility.","section":"Section 4.1"},{"comment":"There are several typos: 'Performence' in Figure 2, 'Evolutiuon' in Table 3, 'Imapct' in Table 4, 'comparsion' in Section 4.1, 'socres' in Section 3.3, and 'Evol-Instuct' in Related Work.","section":"Throughout"},{"comment":"The computational cost section confuses MACs and FLOPs, and the '5 of 10 LLMs' reduction is not directly tied to the agent-pair sampling described in Section 3.1; please clarify the calculation.","section":"Appendix A.5"}],"recommendation":"major_revision","confidential_remarks":"The central IFD monotonicity assumption is empirically testable and the missing details on baselines and seeds are obtainable, so a major revision is appropriate rather than rejection. The paper's contribution is interesting, but the current evidence does not yet support the quantitative claims at the advertised level."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper reports a clean, consistent win — roughly 12% average improvement on two model families over strong baselines, with an ablation that attributes the gain to the selection and evolution modules. That is worth a referee's time. The real open questions are about reproducibility, not about whether the method probably works.\n\nWhat is actually new: the combination of agent-pair sampling for generating instruction-response pairs, the dual-model IFD gap as a selection score, and the evolutionary update of sampling probabilities. Each piece has precursors — CAMEL, Evol-Instruct, Cherry LLM, DEITA — but the specific package is not in prior work. The empirical story is credible: Table 2 shows consistent gains across three benchmarks, Table 3 shows the selection module matters more than diversity alone, and Table 4 shows the dual-model selection beats vanilla IFD and random selection. That last comparison is the right one and it lands the way the authors claim.\n\nSoft spots, in proportion. The biggest is a missing specification: the paper never says which model computes IFD_large. The stress-test note worries that the monotonicity assumption in Eq. 4–5 is unproven. I think that concern is fair but not fatal. For a stronger model, conditioning on an instruction should reduce response perplexity more than it reduces unconditional perplexity, so the IFD ratio usually goes down — but \"usually\" is not \"always,\" and the paper should either prove it, test it empirically on a few model pairs, or at least report the gap distribution. Right now it is an assertion. That is a fixable problem. The absence of error bars and seeds is more annoying; the 40% Fermi gain could be noise, though the average 12% across three benchmarks is harder to dismiss. No code/data at submission is another practical barrier, and the computational cost section is sloppy (MACs vs FLOPs). None of this sinks the core claim; all of it should be addressed in revision.\n\nWho is this for: anyone building automatic instruction-data pipelines for small or medium models. It is not a breakthrough, but it is a useful data point and a solid baseline for future work. I would send it to peer review and push for a conditional accept: require code, specify the large evaluator, add seeds, and include a simple sanity check of the IFD monotonicity assumption. If the authors can do that, the paper will be a reasonable contribution.","headline":"Consistent gains and a sensible ablation make this worth refereeing; the unverified IFD monotonicity and missing implementation details are fixable.","tokens_in":17006,"tokens_out":3120,"would_cite":true,"duration_ms":72777,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning on data curated by Star-Agents—a multi-LLM generation and dual-model selection pipeline—improves instruction-following scores by about 12% over Evol-Instruct data, with up to 40% gains on Fermi tasks.","keywords":["instruction tuning","data optimization","multi-agent LLMs","dual-model evaluation","IFD difficulty metric","data selection","instruction-following benchmarks","LLM agents"],"falsifier":"Compute IFD scores for a large fixed set of instruction-response pairs using a 1B and a 7B model, and count the fraction of pairs where the 7B model's IFD is not below the 1B model's; a non-negligible violation rate (say, above 5%) would show the assumption behind the dual-model gap is not reliable. A second check is to replace the large evaluator with another large model of similar size but different training and observe whether the selected data, and the fine-tuned model's scores, change materially.","tokens_in":15937,"feed_emoji":"🤖","tokens_out":9119,"duration_ms":77006,"temperature":0.7,"pith_summary":"This paper claims that the quality of instruction-tuning data can be improved automatically, without human annotation, by having multiple LLM agents rewrite a seed dataset and then selecting the best samples with a dual-model difficulty filter. The authors report that Pythia-1B and Llama-2-7B fine-tuned on this optimized 'Star Instruct' data beat the same models fine-tuned on Evol-Instruct data by an average of about 12% on MT-bench, Vicuna-bench, and the WizardLM testset, with up to 40% improvement on Fermi tasks. If confirmed, this gives a low-cost recipe for squeezing better instruction-following performance out of small models, using data that is neither too easy nor too hard for them.","feed_headline":"Multi-agent data selection boosts instruction-tuned LLMs by 12%","feed_subtitle":"Star-Agents tailors data difficulty to each target model and beats Evol-Instruct on three benchmarks.","key_machinery":"The load-bearing object is the dual-model evaluation score $\\pi = \\pi_{llm} \\cdot \\pi_{dual}$, where $\\pi_{dual}$ is the normalized difference between the IFD (Instruction-Following Difficulty) scores of the small target model and a larger model, and $\\pi_{llm}$ is a judge score comparing each candidate sample against a base sample. IFD measures how much easier a response is to generate when the instruction is provided than when it is not; the paper assumes a stronger model always gives a lower IFD for the same sample. The score is used both to select the best sample per seed instruction and, through an update rule, to evolve the sampling probability of each agent-pair, so that generator pairs that produce high-scoring data are used more often.","core_discovery":"The paper claims that an automated data-optimization loop can replace hand-crafted complexity escalation for instruction tuning. Starting from the Evol-Instruct seed set, Star-Agents has multiple LLM agents rewrite instructions and responses in pairs, scores each candidate with a dual-model evaluation that combines an IFD-based difficulty gap (small target model versus a larger model) with an LLM judge's quality verdict, and then evolves the sampling probabilities of the agent pairs toward those that produce high-scoring data. The selected 'Star Instruct' data, when used to fine-tune Pythia-1B and Llama-2-7B, yields an average relative improvement of about 12% over the same models fine-tuned on Evol-Instruct data across MT-bench, Vicuna-bench, and the WizardLM testset, with gains up to 40% on Fermi tasks and smaller but consistent gains on a public leaderboard.","pith_inferences":["The dual-model IFD gap is a general-purpose difficulty signal that could be applied beyond instruction tuning, for instance to pretraining data filtering or curriculum learning for any small model paired with a capable teacher.","The monotonicity assumption linking model strength to IFD is probably the most fragile link: if the large evaluator is not a capable generalist on the task distribution, the ranking could invert, so the method may need re-calibration for each target/large pair.","The paper's own limitations note that multi-turn and domain-specific instruction sets were not tested; a direct next experiment is to check whether the same pipeline improves performance on multi-turn dialogue data.","The agent-pair evolution resembles an online learning algorithm over a finite set of generators; analyzing its regret or connecting it to adversarial bandit methods could provide convergence guarantees the current heuristic lacks."],"forward_implications":["Instruction-tuning data can be selected or generated automatically for a specific target model, reducing the need for expensive human annotation and manual prompt design.","Small models (around 1B parameters) benefit more from data of intermediate, tailored difficulty than from uniformly harder data; the paper shows Evol-Instruct's complex samples can actually hurt Pythia-1B relative to Star Instruct.","The evolution rule concentrates generation budget on a few effective agent pairs (e.g., Mistral-ChatGPT reaches about 30% sampling probability while Phi2-ChatGPT decays to near zero), so the framework becomes cheaper over iterations.","The gains are not confined to the three headline benchmarks: the paper's appendix reports higher average scores on a public leaderboard for Llama-2-7B trained on Star Instruct compared with Evol-Instruct (56.29 vs 55.11)."],"supporting_citations":[{"why":"Supplies the Evol-Instruct seed dataset and the WizardLM testset used as a benchmark and baseline.","marker":"[44]"},{"why":"Provides the IFD metric that the dual-model evaluation is built on.","marker":"[20]"},{"why":"Defines the MT-bench and Vicuna-bench evaluation protocol with GPT-4 as judge.","marker":"[54]"},{"why":"Pythia, the 1B target model used in the fine-tuning experiments.","marker":"[2]"},{"why":"Llama-2, the 7B target model used in the fine-tuning experiments.","marker":"[35]"},{"why":"Selective Reflection-Tuning, a strong 7B baseline that Star-Agents surpasses.","marker":"[17]"}],"fun_headline_variants":["Star-Agents auto-tunes instruction data, lifting LLM scores by 12%","Multi-LLM agent loop refines data: 12% avg gain for fine-tuned LLMs","AI agents optimize instruction data: up to 40% gain on Fermi tasks","Automated data evolution with agents boosts instruction-tuned LLMs 12%","Star-Agents: LLM agents craft better instruction data, +12% average"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The selection rule works only if a stronger model always assigns a lower IFD score than the weaker model to the same instruction-response pair, so the gap between the two scores marks samples that are hard for the small model but still learnable; if that ordering fails, the chosen data may not be the most beneficial for the target model.","fun_headline_variants_meta":{"raw":{"variants":["Star-Agents auto-tunes instruction data, lifting LLM scores by 12%","Multi-LLM agent loop refines data: 12% avg gain for fine-tuned LLMs","AI agents optimize instruction data: up to 40% gain on Fermi tasks","Automated data evolution with agents boosts instruction-tuned LLMs 12%","Star-Agents: LLM agents craft better instruction data, +12% average"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000998,"raw_usage":{"total_tokens":4220,"prompt_tokens":935,"completion_tokens":3285,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":3173}},"tokens_in":551,"tokens_out":3285,"duration_ms":22541,"temperature":1.0,"reasoning_tokens":3173,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:52:28.178956+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute IFD scores for a large fixed set of instruction-response pairs using a 1B and a 7B model, and count the fraction of pairs where the 7B model's IFD is not below the 1B model's; a non-negligible violation rate (say, above 5%) would show the assumption behind the dual-model gap is not reliable. A second check is to replace the large evaluator with another large model of similar size but different training and observe whether the selected data, and the fine-tuned model's scores, change materially.","supporting_citations":[{"cited_title":"[[A]]” if assistant A is the bset, “[[B]]","cited_arxiv_id":null,"evidence_quote":"Defines the MT-bench and Vicuna-bench evaluation protocol with GPT-4 as judge."}],"review_version":1}