{"id":"ee0deb51-0ed5-4d20-abbc-5463e88f1ffd","arxiv_id":"2505.00358","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"R&B repartitions training data into embedding-based semantic clusters and reweights them online via a gradient Gram matrix, matching or exceeding prior data-mixing methods at under 0.1% extra compute.","lead":"R&B is a training recipe that splits training data into semantic clusters and continuously rebalances how often each cluster is sampled, using gradients already computed during training. It claims to match or beat existing data-mixing methods while adding almost no extra compute.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Empirical gains may be inflated by selecting cluster count m* on the evaluation set; the reported R&B results need a held-out cluster-selection protocol.","rationale":"After reading the paper, the strongest empirical claims are plausible but rest on a protocol gap. The reader's verdict CONDITIONAL is appropriate, but my load-bearing concern differs from the reader's stated weakest assumption: the embedding-gradient assumption is real, yet even if it holds, the reported gains could be an artifact of selecting k on the evaluation curves. Conversely, if a held-out selection protocol preserves the gains, the method has genuine support. I therefore recommend keeping CONDITIONAL: the paper should be accepted only if the authors supply a selection protocol that does not use the reported eval loss, plus error bars across seeds. I do not recommend REJECT because the method is concretely specified, the overhead model is explicit and reproducible, and the extended Dolly run (Fig. 4) shows a stable albeit small advantage even on original categories.","tokens_in":20511,"tokens_out":4539,"duration_ms":49149,"concrete_test":"Re-run the natural-language comparisons in Table 1 with m* chosen by a protocol that does not use the reported evaluation loss: for example, maximize silhouette score on training embeddings, use a fixed k (e.g., 20), or select k on an inner split of the training set while holding out the reported test partition. Then report the final evaluation loss for each baseline at that same m*. If R&B's margin over Skill-It/Aioli/DGA shrinks or reverses under this held-out selection, the headline claim is not supported. A cheaper variant: on Dolly-15k, repeat the Fig. 2 k-sweep but split the 15k examples into clustering-selection and evaluation partitions, and check whether the green-star k chosen on the selection split still beats the original 8 categories on the untouched split.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's headline result is that R&B matches or exceeds prior mixing methods at negligible overhead (Tables 1–3). The Regroup component depends on choosing the number of clusters m*; Table 1 uses m*=30, 100, and 7 for Sup-NatInst, Sup-NatInst test, and Dolly, and Table 2 uses m*=10 for S1. Section 3.2 describes a sweep over k and identifies a \"sweet spot\" from the top-row evaluation-loss curves in Fig. 2, with the green star marking the best k. Nothing in the text indicates that the m* values used in Tables 1–3 were selected without consulting the evaluation loss that is later reported. Algorithm 2 nominally selects m* via a clustering metric computed on training data, but that metric is not used in the reported protocol. If m* is chosen as the minimizer of the reported eval loss, the comparison is selection-on-the-test-set: Regroup will look better than fixed human domains by construction, and the baselines are not given the same k-selection freedom. The silhouette-based alternative is not a substitute here: the bottom-row R² values are 0.047, 0.173, 0.482, and 0.895, and the paper does not use silhouette to pick the final m*. This experimental gap directly affects the central claim; it is separate from the theory assumption that embeddings mimic gradients (App. B.1.2), which is a mechanism-level concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes R&B, a two-stage data-mixing method: Regroup repartitions training data by k-means on ModernBERT embeddings, and Balance reweights the resulting domains online using a Gram matrix of per-domain gradient inner products accumulated during ordinary training, with a softmax update driven by evaluation-set proportions. The authors give a regret bound for stable clusterings, a first-order derivation of the update rule, a FLOP cost model, and experiments on Dolly-15k, Super-NaturalInstructions (ID/OOD), S1-Reasoning, and DataComp CLIP training. The central claims are that semantic clusters outperform human-defined skill categories and that R&B matches or beats prior mixing methods with 0.01%-scale overhead.","tokens_in":20796,"tokens_out":6125,"duration_ms":59651,"significance":"If the empirical claims survive scrutiny, R&B is a useful contribution: the gradient-accumulation trick avoids separate evaluation passes, the m^2 scaling is much cheaper than methods that evaluate on large evaluation sets, and the theoretical framing (regret bound, NTK-style Gram matrix) is a reasonable scaffold. The experiments span natural language, reasoning, and multimodal training, and the ablations that separate Regroup from Balance are a strength. At present, however, the m* selection protocol and the headline table's domain confound are load-bearing gaps.","major_comments":[{"comment":"Figure 2's top row sweeps k and marks a 'sweet spot' (green star) on evaluation-loss curves, and Tables 1 and 2 then use m* = 30, 100, 7, and 10 for the reported head-to-head results. The text does not describe a held-out procedure for choosing these values, so the Regroup comparisons may be selected on the same evaluation loss that is later reported, which would inflate R&B's advantage; baselines are not given equivalent k-selection freedom. Algorithm 2 proposes selecting m* with a clustering metric, but that metric is not used in the reported protocol, and the silhouette-to-loss correlations in Figure 2's bottom row (R^2 = 0.047, 0.173, 0.482, 0.895) are too weak on three of the four datasets to serve as a validated substitute. Please either select m* on a held-out portion of the evaluation set, use and validate Algorithm 2's metric, or report sensitivity to m* and show that the headline gains persist.","section":"Section 3.2, Figure 2, Tables 1-2"},{"comment":"Table 1 compares R&B on regrouped domains (m* = 30, 100, 7) against Skill-It, Aioli, and DGA on their original domains (m = 38, 60, 8), without stating this in the table or its caption. The performance gap therefore conflates the Regroup component with the Balance component; the ablation table in Figure 3 does control for this, but the paper's headline table does not. Please make Table 1 a same-domain comparison (or explicitly present the Figure 3 ablation as the performance evidence and keep Table 1 for overhead), and state in the caption which domain partition each row uses.","section":"Table 1"},{"comment":"The theoretical mechanism assumes that ModernBERT embedding clusters 'mimic the gradients' of the model being trained (Appendix B.1.2), but the only empirical support offered is the silhouette-to-loss correlation, which is negligible for Dolly-15k (R^2 = 0.047), weak for Super-NatInst (R^2 = 0.173), and moderate for Super-NatInst Test (R^2 = 0.482), and the paper does not use silhouette to select m*. As written, the claim that semantic clustering is generally superior to human-defined domains rests on a small number of training runs whose cluster counts were selected from the evaluation curves in Figure 2. Please add a direct check (e.g., compare k-means cluster alignment with per-cluster gradient inner products during a probe run) or substantially soften the mechanism claim and restrict it to the datasets where the effect is replicated.","section":"Section 3.2 / Appendix B.1.2"}],"minor_comments":[{"comment":"The title contains a typo ('B alancing') and Section 4.1 says 'three distinct three natural-language data settings'; please fix both.","section":"Title and Section 4.1"},{"comment":"Table 1's overhead entries for Skill-It and Aioli on Sup-NatInst test are formatted as '6x107%' and '7x106%'; use proper superscripts and state whether these are relative overhead percentages or multiplicative factors.","section":"Table 1"},{"comment":"Algorithm 3 ends with an empty 'Return' statement; return the trained model or remove the line.","section":"Algorithm 3"},{"comment":"In Table 6, 'num layersto track=1, lamda=3' has typos and should name the layer-selection procedure; also clarify why R&B on original NI-OOD produces a NaN in Gp, since the footnote says so but the main text should state this limitation.","section":"Table 6"},{"comment":"The text claims R&B reaches convergence with only 20% of the training steps needed by other methods, but Figure 3 does not show a convergence threshold or quantify how this percentage was computed; please add the threshold or revise the claim.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the headline performance gains partly reflect selection on the evaluation set. The paper is otherwise within scope for a CS journal and the fixes are feasible; I would not reject on the current evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"R&B is worth engaging with. The core idea—replace human-defined domains with k-means clusters of embeddings, then reweight on the fly using a Gram matrix of per-domain gradients—is genuinely new and practically appealing. The compute cost is tiny because they reuse gradients from the main training pass and only interact with the evaluation set via its proportions, not its losses or gradients. That is a real innovation.\n\nThe experiments, though small, are diverse: instruction tuning, reasoning, and CLIP. The ablation in Fig. 3 shows regrouping helps most methods on most datasets. Balance alone improves over stratified on original domains (e.g., Sup-NatInst 2.520 vs. 2.591), which suggests the core mechanism has value even apart from the clustering.\n\nThe largest problem is m* selection. Section 3.2 reports a sweep over k and marks a “sweet spot” in the evaluation-loss curves; Table 1 then uses exactly those m* values. No held-out protocol is described. If m* is the minimizer of the eval loss, the Regroup gains are partially selection artifacts, and the baselines don’t get the same freedom. Algorithm 2 says a clustering metric selects k, but the paper doesn’t show that this metric was used for the headline results—and the reported silhouette correlations (R² 0.047–0.895) are too weak on three of four datasets to support it. This is not fatal to Balance alone, but it directly affects the strongest claim that semantic regrouping beats human domains. I agree with the stress-test note here.\n\nSecond, the theory covers a corner-of-the-simplex update, while the algorithm uses the softmax with entropy regularization. The appendix acknowledges this, but the formal justification for the actual update is thin. The “embeddings mimic gradients” assumption is stated explicitly, but the evidence for it is the weak silhouette correlation; the mechanism is plausible but not established.\n\nMinor concerns: no error bars, small models (125M and 0.5B), short training (2k steps). The OOD limitation on Sup-NatInst test is disclosed, but it means the method’s applicability to truly out-of-domain evaluation is limited.\n\nOverall, this paper has a good idea and an honest presentation. The m* issue is fixable with a proper held-out selection or by showing the clustering metric picks similar k. The theory could be tightened to match the implemented update. I’d support peer review: the contribution is novel and the efficiency claim is worth checking. But I would not cite the empirical gains as established until the selection protocol is clarified.","headline":"R&B is a genuinely new and efficient data-mixing framework, but the cluster-count selection appears tuned on the eval set and the theory doesn't quite cover the implemented update; worth peer review with major revisions.","tokens_in":21341,"tokens_out":3718,"would_cite":true,"duration_ms":37483,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"R&B claims that repartitioning training data by semantic similarity and rebalancing domain weights from already-computed gradients delivers state-of-the-art data mixing at 0.01% extra compute.","keywords":["data mixing","domain regrouping","semantic clustering","gradient balancing","Gram matrix","online data selection","efficient training","foundation models"],"falsifier":"A checkpoint experiment: fix a trained model and the learned clusters, compute the Gram matrix $G$, and measure the one-step loss decrease of the softmax-weighted mixture versus uniform sampling and versus the best single cluster. R&B's derivation requires the softmax-weighted step to decrease loss at least as much as any fixed proportion for a small learning rate; a measured violation would falsify the balancing mechanism.","tokens_in":20322,"feed_emoji":"⚖️","tokens_out":10100,"duration_ms":96268,"temperature":0.7,"pith_summary":"R&B is a claim about two separate bottlenecks in data mixing: what counts as a domain, and how much it costs to reweight domains. The paper argues that human-defined categories like 'open QA' or 'summarization' leave useful structure on the table, and that repartitioning data by embedding similarity into finer-grained clusters improves evaluation loss. It then argues that reweighting can be made nearly free by reading off domain gradients accumulated during ordinary training, forming their Gram matrix, and applying a softmax update, instead of paying for evaluation passes. The empirical claim is that this combination matches or beats existing data-mixing strategies on five datasets while adding about 0.01% extra compute, more than 99% less than the baselines. If true, data mixing stops being a separate, expensive optimization and becomes a byproduct of training itself.","feed_headline":"Gradient reuse makes data mixing 99% cheaper","feed_subtitle":"Semantic regrouping plus gradient reweighting beats costly baselines and cuts evaluation FLOPs by over 99%.","key_machinery":"The load-bearing object is the Gram matrix $G_{ij}(t) = \\nabla L(\\theta_t; D_i)^\\top \\nabla L(\\theta_t; D_j)$, the matrix of inner products between the gradients of each domain on the current model. Each round, R&B accumulates final-layer per-domain gradients from ordinary backward passes, forms $G$, and sets the next sampling distribution to $\\mathrm{softmax}(\\lambda G p / \\|G p\\|_2)$, where $p$ holds the evaluation-domain proportions. The derivation treats the loss as locally linear in the SGD step, so the per-step loss decrease is $\\eta p^\\top G p'$; maximizing this with a cross-entropy regularizer yields the softmax update. The other half, Regroup, uses k-means on ModernBERT embeddings plus a silhouette-score sweep to choose the number of clusters, and a regret bound (Lemma 1) says that low-radius, well-separated clusters make the mixing objective stable.","core_discovery":"The paper's central claim is that predefined, human-assigned data categories are the wrong units for data mixing, and that the right units are finer-grained domains obtained by semantically clustering the training data. On top of that, it claims that the optimal mixture over those domains can be tracked online at almost no extra cost: instead of running evaluation forward passes or separate gradient computations, R&B reuses per-example gradients that normal backprop already produces, aggregates them per domain, forms the Gram matrix $G$, and reweights via $\\mathrm{softmax}(\\lambda G p / \\|Gp\\|_2)$. The paper reports that this two-stage recipe matches or improves on Skill-It, Aioli, and DGA across Dolly-15k, Super-NaturalInstructions, S1-reasoning, and DataComp CLIP training while adding only about 0.01% compute overhead. It also proves a regret bound showing that compact, well-separated clusters reduce the worst-case penalty for bundling data into skills, which is the theoretical reason regrouping helps.","pith_inferences":["Not tested here: R&B's update could be run over sliding windows of a single corpus, turning mixture optimization into a fully local, online data-selection rule rather than a global one.","If the silhouette-score heuristic generalizes, practitioners could choose the number of clusters without training any model, making R&B a preprocessing step plus a free training-time add-on.","The paper's evidence that topic-based clusters beat task-based categories on instruction data suggests a transfer claim worth testing: topic groupings should be more stable across model sizes and architectures than task groupings, because they track content rather than annotation labels.","Because the Gram matrix is a per-domain neural tangent kernel, it could also flag redundant or conflicting domains mid-training: domains with persistently low alignment to the evaluation mixture are candidates for downweighting or removal."],"forward_implications":["Because R&B's overhead scales with the square of the number of domains rather than with evaluation-set size, models can be mixed over hundreds of fine-grained clusters, exactly the regime where prior methods become prohibitively expensive.","Regrouping alone, before any balancing, improves stratified sampling on three of the four language datasets tested, so the definition of a domain matters as much as the weighting.","The full R&B pipeline reaches lower evaluation loss than the strongest baselines on both NaturalInstructions benchmarks, and on the 40,000-step Dolly run it keeps a consistent advantage through training.","On multimodal CLIP training the advantage grows with domain count: with 50 domains R&B beats stratified sampling by 3.27% relative average score, suggesting the method pays off when semantic structure is fine-grained.","R&B reaches convergence with roughly 20% of the training steps needed by comparison methods, so efficient mixing also means faster training."],"supporting_citations":[{"why":"The closest prior domain-reweighting method; R&B contrasts its softmax windowed update with this method's multiplicative-weights accumulation.","marker":"[1]"},{"why":"Supplies the skills-graph data-mixing baseline that R&B is compared against on both performance and compute overhead.","marker":"[3]"},{"why":"Supplies the online data-mixing baseline whose per-domain evaluation cost R&B avoids.","marker":"[4]"},{"why":"Supplies the Dolly-15k instruction-tuning dataset whose human categories are shown to be suboptimal for mixing.","marker":"[7]"},{"why":"Supplies the ModernBERT embedding model used to create the semantic clusters in Regroup.","marker":"[23]"},{"why":"Supplies the Super-NaturalInstructions in-domain and out-of-domain benchmarks used for evaluation.","marker":"[24]"},{"why":"Supplies the S1-reasoning dataset used to test R&B beyond language modeling.","marker":"[25]"},{"why":"Supplies the silhouette score used to select cluster counts and to argue that well-separated clusters predict performance.","marker":"[26]"},{"why":"Supplies the dynamic gradient alignment baseline; R&B derives its update from the same gradient-alignment idea without extra evaluation passes.","marker":"[27]"},{"why":"Supplies the per-example gradient decomposition that lets R&B collect per-domain gradients from a single backward pass.","marker":"[35]"}],"fun_headline_variants":["R&B: Reuse gradients, cut data mixing cost 99%","Semantic regrouping fuels cheap data mixing","No extra evals: gradient reuse for data blend","R&B: Finer domains, near-zero overhead mixing","Gradient-based balancing trims data mix compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method depends on the assumption that data points grouped by embedding similarity will also affect the model in similar ways during training, so that reweighting those groups actually reweights what the model learns.","fun_headline_variants_meta":{"raw":{"variants":["R&B: Reuse gradients, cut data mixing cost 99%","Semantic regrouping fuels cheap data mixing","No extra evals: gradient reuse for data blend","R&B: Finer domains, near-zero overhead mixing","Gradient-based balancing trims data mix compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000174,"raw_usage":{"total_tokens":1290,"prompt_tokens":958,"completion_tokens":332,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":255}},"tokens_in":574,"tokens_out":332,"duration_ms":3744,"temperature":1.0,"reasoning_tokens":255,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:44:37.365738+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A checkpoint experiment: fix a trained model and the learned clusters, compute the Gram matrix $G$, and measure the one-step loss decrease of the softmax-weighted mixture versus uniform sampling and versus the best single cluster. R&B's derivation requires the softmax-weighted step to decrease loss at least as much as any fixed proportion for a small learning rate; a measured violation would falsify the balancing mechanism.","supporting_citations":[{"cited_title":"Free Dolly: Introducing the World’s First Truly Open Instruction-Tuned LLM","cited_arxiv_id":null,"evidence_quote":"Supplies the Dolly-15k instruction-tuning dataset whose human categories are shown to be suboptimal for mixing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the silhouette score used to select cluster counts and to argue that well-separated clusters predict performance."}],"review_version":1}