{"id":"c4a8540e-2ff4-40ad-9535-7be3e341e309","arxiv_id":"2506.14702","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding a rich taxonomy of training-time markers with marker dropout improves controllable and long-tail generation in an instruction-tuned LLM, with the largest gains on rare tasks like code repair.","lead":"Researchers at Cohere trained a 7B instruction-tuned model with detailed labels, called markers, attached to training examples, and then let the model infer or receive those labels at inference time. The approach raised win rates against a no-marker baseline by 5.7% overall and by more on rare domains and tasks, and sharply improved length and language following.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing control: the fixed-marker results compare a marker-augmented test prompt against a no-marker baseline, so the reported gains conflate training-time marker conditioning with test-time prompt enrichment.","rationale":"The reader's conditional verdict is reasonable, but I would sharpen the condition. The paper's strongest evidence (abstract, Fig. 5, Tables 4 and 6) always compares marker-augmented prompts against a no-marker baseline. Equation (2) and Section 3.2 set up the comparison as training-with-markers vs. training-without-markers, yet the fixed-marker experiments also change the test-time prompt. This is a confound: the observed win-rate and pass@1 lifts could come from the test-time metadata alone. The paper includes useful controls elsewhere: Table 5 reports paired significance tests for MT, Table 10 checks marker prediction accuracy under dropout, and Section 5.1 shows an external annotator can reproduce length gains. These strengthen the plausibility that markers are usable, but they do not isolate the training-time mechanism. The reader's concern about LLM-assigned marker accuracy is related but secondary; even perfectly accurate markers would not establish that training-time conditioning, rather than test-time prompt augmentation, is responsible. I therefore recommend keeping the CONDITIONAL verdict pending the inference-only marker control.","tokens_in":23120,"tokens_out":6845,"duration_ms":74438,"concrete_test":"Run a 2x2 comparison on the CodeRepair (HumanEvalPack) and AlpacaEval-LI sets: (A) Baseline, no template; (B) Baseline, with the exact marker template appended at inference (e.g., <task>CodeFix</task>, <length_tokens>199</length_tokens>); (C) TreasureMarked(fixed); (D) a model trained identically but with markers randomly permuted across training examples. If B closes most of the C-vs-A gap, the training-time marker objective is not the causal driver; if D matches C, the specific marker semantics are not load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that training-time markers drive the reported gains. In every fixed-marker result (CodeRepair +14.1% in Fig. 5, length violation 36.58% to 1.25% in Table 4, language control +10.98% in Table 6), the test prompt is augmented with the marker template while the Baseline prompt is not. Section 3.2 defines Baseline as trained on the same data without markers, but no condition appends the marker template to Baseline at inference. Appending '<task>CodeFix</task>' or '<length_tokens>199</length_tokens>' is itself a strong test-time hint, so the comparison cannot separate the training-time marker objective from the inference-time injection of metadata. The quality lever in Fig. 4 compounds this: it is evaluated with the same reward model that produced the <quality> labels, so that result partly measures self-consistency with the RM rather than transferable quality control. A model trained with shuffled markers and a baseline given the same test-time template would resolve whether the taxonomy, the marker objective, or merely the added prompt metadata causes the gains.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes training-time 'treasure markers,' a taxonomy of ninety markers across categories such as length, quality, domain, task, and language, added to instruction-tuning prompts and completions, with dropout to keep them optional at inference. On a proprietary 7B model fine-tuned on 2.7M examples, the authors report that a model trained with markers improves ArenaHard win rate by 5.7% over a no-marker baseline and by 9.1% on domains underrepresented in training data; fixed-marker variants also improve length-constraint following, code repair and translation pass rates, multilingual language control, and machine-translation scores. The paper includes an on-the-fly annotation ablation with Command A and a dropout ablation showing that markers can be inferred at inference time.","tokens_in":23307,"tokens_out":10276,"duration_ms":102118,"significance":"If the reported effects replicate, this is a useful and low-cost training-side intervention: it gives users control levers for quality, length, and language without requiring prompt engineering, and it appears to benefit long-tail tasks disproportionately. The manuscript's strengths include the use of external evaluation benchmarks (Arena-Hard, WMT24++, Language Confusion Benchmark, HumanEvalPack), a machine-translation table with paired significance tests, explicit dropout ablations, and a thorough related-work section. The main caveats are the prompt-enrichment confound in the fixed-marker comparisons, the circular reward-model evaluation of the quality lever, and missing uncertainty quantification for the headline win-rate claims. The general idea of control codes is not new, but the breadth of the taxonomy and its application to general instruction-following are a meaningful step beyond prior single-attribute markers.","major_comments":[{"comment":"The fixed-marker comparisons lack a marker-only control. In these conditions the TreasureMarked(fixed) test prompts contain a marker template that the Baseline prompts do not contain, so the reported gains (e.g., 14.1% for CodeRepair, 36.58% to 1.25% length violation, +10.98% language pass rate) conflate training-time marker conditioning with inference-time injection of metadata. For Table 4 the two conditions even differ in how the constraint is expressed: the natural-language length instruction is removed and replaced by <length_tokens>, so the comparison is not a clean ablation. Please add a control in which the Baseline model receives the identical marker template at inference, and ideally a shuffled-marker condition, to isolate the effect of the training-time objective.","section":"§4.2.1 (Fig. 5), §4.3 (Table 4), §4.5 (Table 6)"},{"comment":"It is not stated whether generated marker lists are stripped before evaluation. Because markers are prepended to completions during training and generation-side dropout is never applied, TreasureMarked models will often emit a <MARKER_LIST> preamble even when the prompt contains no markers. If these preambles are retained in the evaluated outputs, they are an uncontrolled difference from Baseline outputs and could affect both the GPT-4o judge and token-count-based length metrics. Please state explicitly what post-processing is applied to model outputs, and if markers are not stripped, repeat the main evaluations with stripped outputs.","section":"§2.1, §3.2"},{"comment":"The quality-control result is evaluated with the same reward model that produced the <quality> and <quality_bucket> labels. This circularity means that the 48.21% to 56.5% win-rate curve largely measures the model's ability to mirror the RM's preferences rather than a transferable notion of generation quality. Please re-evaluate the quality lever with an independent judge, such as GPT-4o or human ratings, or with a reward model held out from the labeling process.","section":"§4.1.1, Fig. 4"},{"comment":"The headline win-rate numbers are point estimates without error bars or significance tests. Given the finite prompt set and a single judge, the +5.7% overall and +9.1% long-tail differences on Arena-Hard need bootstrap confidence intervals or per-prompt paired tests; the same applies to the quality-lever curve in Fig. 4. The WMT section (Table 5) correctly reports significance tests, so adding equivalent uncertainty quantification to the other evaluations would make the results more comparable and more convincing.","section":"§4.1, Fig. 3, Fig. 4"},{"comment":"The accuracy of LLM-assigned markers is not validated. The paper states that <domain>, <task>, and <format> are tagged by Command R+ whenever dataset metadata is unavailable, but no annotation agreement, human-validation rate, or consistency check is reported. Since the training objective in Eq. (2) conditions on these labels, and the long-tail code results depend on task labels such as CodeFix and CodeTranslation, noisy or inconsistent labels could undermine the learned associations. Please report annotation quality, for example agreement with a human-annotated subset or a second annotation pass, and preferably include a shuffled-marker control for the main experiments.","section":"§3.1"}],"minor_comments":[{"comment":"The dataset name is misspelled as 'AlpcaEval' in both tables; it should be 'AlpacaEval'.","section":"Tables 3, 7"},{"comment":"The caption says 'Bold differences are significant at p≤0.05,' but no bold formatting is visible in the table; please format the significant entries accordingly.","section":"Table 5"},{"comment":"The row labels '0_50', '50_50', and '70_50' are not defined; please state in the caption that the first value is the dataset-level dropout percentage and the second is the sample-level dropout percentage.","section":"Table 10"},{"comment":"The caption contains a stray space in 'T reasureMarked'; please fix the typo.","section":"Fig. 2 caption"},{"comment":"The phrase 'absolute improvements of 35.3%' should be 'a 35.3 percentage-point improvement' because the violation rate changes from 36.58% to 1.25%; the current wording could be read as a relative improvement.","section":"§4.3"},{"comment":"The notation 'dropout(m_d)' is used without a formal definition; please define it as a stochastic masking operator applied independently to each training example.","section":"Eq. (2)"}],"recommendation":"major_revision","confidential_remarks":"The experiments rely on a proprietary base model, a proprietary reward model, and an internal 2.7M-example instruction set, and no data, markers, or model weights are released. This limits independent verification but is not by itself disqualifying for an industry paper. The fixed-marker confound is the central concern; if the authors add a marker-only baseline and an independent quality evaluation, the paper's main claims would be substantially stronger. I support major revision rather than rejection because the TreasureMarked (inferred) results on Arena-Hard and the significant MT improvements suggest a real effect."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid, practically motivated paper on training-time markers for instruction-tuned models. The main new thing is the breadth of the taxonomy (90 markers across length, quality, domain, task, language, format) combined with dual dropout so the model can infer markers when they're absent. That combination is genuinely useful, and the long-tail effect on rare code tasks is the strongest result.\n\nWhat the paper does well: the baseline is well-defined (same data, no markers), the evaluation covers external benchmarks (ArenaHard with GPT-4o, HumanEvalPack, WMT, Language Confusion), and the on-the-fly annotation ablation using Command A is a nice practical addition. The inferred-marker condition (TreasureMarked, no markers in the prompt) already gives a 5.7% win-rate lift and a 9.1% gain on underrepresented domains; that comparison is clean and supports the training-time attribution. The dropout ablation is informative.\n\nWhere the soft spots are, in order of size:\n\n1. The fixed-marker results (length control at 1.25% violation, language control +10.98%, and the headline CodeRepair +14.1%) compare a marker-augmented prompt against a baseline prompt with no marker template. That conflates training-time marker conditioning with test-time prompt enrichment. The stress-test is right that a baseline with the same template (or shuffled markers) is missing. The paper partially mitigates this with the inferred-marker condition, but the fixed-marker numbers are the ones that get quoted.\n\n2. The quality-control result (Figure 4) is internally circular: the same proprietary reward model created the <quality> labels and evaluates the win rates. This is disclosed, but it means that result mostly shows self-consistency with the RM, not transferable quality control.\n\n3. No code, data, or model artifacts are released, so the key numbers are not independently checkable. For a paper whose selling point is a training recipe, that's a real gap.\n\n4. Headline win rates lack error bars or significance tests, though the MT results do report significance.\n\nNone of these are fatal. The core long-tail claim survives scrutiny because the inferred-marker condition is clean. The paper deserves a serious referee; the fixes are straightforward: add a baseline-with-template control, evaluate quality with an external judge, and release artifacts. I'd recommend engagement.","headline":"Training-time markers with dual dropout are a real, useful idea, and the long-tail gains hold up through the inferred-marker condition; the fixed-marker results need a baseline-with-template control before the headline numbers are trustworthy.","tokens_in":23849,"tokens_out":3149,"would_cite":true,"duration_ms":30501,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training-time markers give users control and lift long-tail performance.","keywords":["training-time markers","long-tail performance","instruction tuning","controllable generation","marker dropout","multilingual","code repair","length control"],"falsifier":"One concrete test would be to compute marker prediction accuracy on a held-out set with human-verified tags, or to re-train the model with markers deliberately corrupted on a subset of the data and check whether the claimed improvements in long-tail performance disappear.","tokens_in":22918,"feed_emoji":"🏷️","tokens_out":1198,"duration_ms":13353,"temperature":0.7,"pith_summary":"This paper argues that the gap between what a model was trained on and what a user asks for at inference time can be closed by attaching a rich set of training-time markers to instruction-tuning data. The authors claim that conditioning generation on a taxonomy of tags, from domain and task to length and quality, improves win rates on open-ended generation and yields disproportionately large gains on rare, long-tail tasks. They also show that marker dropout makes the tags optional, so a model can infer them at inference time or accept them when explicitly provided.","feed_headline":"Tags during training lift rare-task performance by 14%.","feed_subtitle":"A taxonomy of 90 markers lets models infer user intent and control length, quality, and language.","key_machinery":"The central machinery is a taxonomy of training-time markers formatted in a standardized template and placed in both the prompt and the completion, so the model learns associations between these textual attributes and the resulting generation. Each training example carries a list of markers such as <domain>, <task>, <length_bucket>, <language>, <quality_bucket>, and <code_type>, and the training objective conditions the completion on the markers while dropout on the prompt side forces the model to learn to infer missing markers at inference time.","core_discovery":"The paper's central claim is that adding up to 90 structured training markers across 13 categories to instruction-finetuning data, and letting the model both see and generate these markers, creates controllable generation that performs better on underrepresented use cases. In experiments, the TreasureMarked model improves ArenaHard win rates by 5.7% over a baseline, with 9.1% gains in domains that appear in less than 5% of training data and up to 14.1% relative gains on rare subtasks like CodeRepair. Marker dropout is key: without it the model becomes overly dependent on explicit tags, but with 50% dataset-level dropout it learns to infer markers at inference time, making the tags optional while still gaining control over length, quality, and language.","pith_inferences":["Using an external annotator at inference time to generate markers could serve as a practical deployment pattern, trading one extra model call for improved accuracy on rare use cases.","The same taxonomy-based conditioning approach could be applied to other domains, such as specialized reasoning or safety-relevant generation, where rare attributes are worth steering explicitly.","The quality and consistency of the LLM-assigned markers is the principal source of uncertainty; if the tags mislabel examples, the model will learn weaker associations.","Future work could test whether the benefits persist when markers are assigned by a weaker or cheaper model, or when the taxonomy is expanded beyond the current 23 languages."],"forward_implications":["Users can control generation attributes at inference time by setting or inferring markers, reducing reliance on fragile prompt engineering.","Long-tail tasks that appear rarely in the training data stand to gain disproportionately from marker conditioning.","Explicit length markers nearly eliminate length-constraint violations (from 36.58% to 1.25% in the fixed setting).","Language control improves across 23 languages, with an average absolute gain of 10.98% on a language-confusion benchmark.","A second model can annotate an incoming prompt on-the-fly, extending the benefits of markers to unseen user requests."],"supporting_citations":[{"why":"Supplies the Arena-Hard-Auto benchmark used to measure open-ended generation win rates.","marker":"Li et al. 2024"},{"why":"Provides the AlpacaEval-LI dataset used to evaluate length-constraint following and generation quality.","marker":"Yuan et al. 2024"},{"why":"Provides the HumanEvalPack dataset used to evaluate code generation, repair, and translation.","marker":"Muennighoff et al. 2023"},{"why":"Supplies the m-Arena Hard multilingual evaluation set used for marker prediction accuracy and quality control.","marker":"Dang et al. 2024"},{"why":"Provides the Language Confusion Benchmark used to measure language-control improvements.","marker":"Marchisio et al. 2024"},{"why":"Supplies the WMT'24++ test sets used for machine translation evaluation.","marker":"Deutsch et al. 2025"},{"why":"Provides the XCOMET-XL metric used to score translation quality.","marker":"Colombo et al. 2023"},{"why":"Supplies Command A, the model used in the on-the-fly marker annotation ablation.","marker":"Cohere et al. 2025"}],"fun_headline_variants":["Training-time markers lift rare-task AI performance by 14%","90 training tags give AI control over rare, underrepresented tasks","Markers during training boost AI on long-tail tasks by up to 14%","Training markers improve AI on rare tasks—up to 14% gain","Add training tags to get up to 14% better on rare tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the LLM- and dataset-derived markers assigned to the training data are accurate and consistently defined, so that the model learns genuine associations between the tags and the desired output properties.","fun_headline_variants_meta":{"raw":{"variants":["Training-time markers lift rare-task AI performance by 14%","90 training tags give AI control over rare, underrepresented tasks","Markers during training boost AI on long-tail tasks by up to 14%","Training markers improve AI on rare tasks—up to 14% gain","Add training tags to get up to 14% better on rare tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001166,"raw_usage":{"total_tokens":4845,"prompt_tokens":987,"completion_tokens":3858,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":603,"completion_tokens_details":{"reasoning_tokens":3765}},"tokens_in":603,"tokens_out":3858,"duration_ms":28515,"temperature":1.0,"reasoning_tokens":3765,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:47:45.409838+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete test would be to compute marker prediction accuracy on a held-out set with human-verified tags, or to re-train the model with markers deliberately corrupted on a subset of the data and check whether the claimed improvements in long-tail performance disappear.","supporting_citations":[{"cited_title":"xcomet: Transparent machine translation evaluation through fine-grained error detection","cited_arxiv_id":null,"evidence_quote":"Provides the XCOMET-XL metric used to score translation quality."}],"review_version":2}