{"id":"bf7816c9-711f-4317-aaca-dccc50d6206c","arxiv_id":"2507.00214","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Training a generative classifier to output an LLM-generated reasoning before the emotion label improved emotion classification accuracy from 49.7% to 58.4% on the dair-ai/emotion test set.","lead":"This paper trains a 1B-parameter language model to generate an explanation before predicting an emotion, and reports an 8.7 percentage point accuracy gain over a baseline that only predicts the label. The approach could make small classifiers both more accurate and more interpretable, but the comparison lacks important controls.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 8.7-point gain is attributed to reasoning content, but no control isolates semantic content from sequence-length/format effects; a shuffled-reasoning control would test this.","rationale":"The reader's weakest assumption — that the accuracy gain could be due to longer output sequences or extra training tokens rather than reasoning content — is precisely the most load-bearing concern. The paper's abstract and conclusion repeatedly attribute the improvement to 'explicit reasoning training' and 'the benefit of explicit reasoning training,' so the causal interpretation is central, not incidental. Without a control for sequence length or input-independent text, the observed gain could be an artifact of the baseline being artificially weak: an instruct-tuned model trained to emit only a bare label may underperform its natural text-generation behavior. This is not a minor technicality; it determines whether the paper demonstrates 'reasoning-infused learning' or merely 'longer-output learning.' A shuffled-reasoning control is a clean, low-cost experiment that would settle the matter. I do not see evidence of internal inconsistency or fraud; the authors are appropriately tentative in the limitations section, acknowledging dependence on reasoning quality and class imbalance. The missing control is an experimental design gap, not a correctness error. Therefore the reader's CONDITIONAL verdict remains appropriate: accept only after the control is added. No verdict change is needed from my pass, as I agree with the reader's identified assumption and proposed direction.","tokens_in":7250,"tokens_out":2772,"duration_ms":35417,"concrete_test":"Fine-tune a third model, Classifier Q->R_shuffled+A, on the same dair-ai/emotion training set, where for each example the target sequence is the generated reasoning R from a randomly selected different training example followed by the correct label. This preserves output length, style, and label distribution while destroying the semantic alignment between reasoning and input. Compare its test accuracy to Q->RA (58.4%) and Q->A (49.7%). If the shuffled model matches Q->RA (~58%), the improvement is not due to reasoning content. If it drops toward Q->A, content matters. As a secondary check, also train Q->Filler+A with a fixed generic filler sentence (e.g., 'Here is my reasoning.') to isolate length/format alone. Repeating each condition over 3 seeds with McNemar's test would additionally provide variance and a proper paired comparison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that training a generative classifier to output (Reasoning, Label) improves emotion classification accuracy by 8.7 points over (Label), and that this improvement stems from the semantic content of the reasoning. The comparison Q->RA vs. Q->A varies several factors at once: output sequence length, output format (a long text vs. a single token), and alignment with the instruction-tuned base model's pretraining objective. Llama-3.2-1B-Instruct is trained to generate text; forcing it to emit only 'joy' is an unnatural task that may artificially depress the baseline. There is no control condition that outputs non-reasoning filler text (or input-independent generic reasoning) before the label, so the 8.7-point gain cannot be causally attributed to reasoning content. A controlled comparison with a shuffled-reasoning target (same reasonings, but randomly reassigned to different inputs) would separate length/format effects from the effect of reasoning actually being about the input. Because the paper's entire contribution rests on this attribution, this missing control is the most load-bearing weakness. The reader identified the same issue. Additionally, the lack of multiple training runs means the reported single-run accuracy has no variance estimate, and the two-proportion z-test does not account for paired test data; however, the confound is more fundamental because it directly undermines the interpretation of the improvement, not just its statistical stability.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-stage reasoning-infused learning framework. In Stage 1, a Llama-3.2-1B-Instruct model is fine-tuned on a general-purpose reasoning dataset (syvai/reasoning-gen) to create a reasoning generator (Llama-R-Gen). In Stage 2, this generator is used offline to augment the dair-ai/emotion training set, producing (text, reasoning+label) targets. A downstream generative classifier (Q->RA) is then fine-tuned to emit reasoning followed by the emotion label. On the emotion test set, the proposed Q->RA model achieves 58.4% accuracy versus 49.7% for a baseline Q->A model that outputs only the label, an 8.7-point gain. The paper also reports per-emotion results, showing large gains for sadness and fear but a collapse for the minority 'surprise' class, which is honestly discussed in the limitations.","tokens_in":7463,"tokens_out":4668,"duration_ms":51836,"significance":"If the causal attribution to reasoning content is validated, this is a useful and practical contribution: it demonstrates that a general reasoning generator can be transferred to a new domain to create richer training signals, and it releases the augmented datasets for further study. The paper is clearly written, transparent about the surprise-class failure, and provides a reproducible-seeming pipeline. However, the central claim that the accuracy gain comes from the semantic content of the reasoning is not yet established, because the comparison varies output length, output format, and task naturalness simultaneously. The significance of the work is therefore conditional on additional control experiments that isolate the effect of reasoning content.","major_comments":[{"comment":"The 8.7 percentage point improvement of Classifier Q->RA over Classifier Q->A is attributed to the reasoning content, but the two conditions differ in several correlated factors: output sequence length, output format (a long text versus a single token), and alignment with the instruction-tuned base model's generative objective. For Llama-3.2-1B-Instruct, generating only a single label is an unnatural task and may artificially depress the baseline. Because there is no control condition that outputs non-reasoning filler text before the label, and no shuffled-reasoning condition in which the same reasonings are randomly reassigned to different inputs, the gain cannot be causally assigned to reasoning that is about the input. This missing control is load-bearing for the paper's central claim of a benefit from explicit reasoning training.","section":"§5.2, Table 3"},{"comment":"The statistical support for the headline result is weakened by the lack of multiple training runs: each model was fine-tuned once, so no variance estimate accompanies the 58.4% versus 49.7% figures. Moreover, the two-proportion z-test treats the two models' predictions as independent samples, although both are evaluated on the same 2,000 test instances. A paired test such as McNemar's test, or a report of mean and standard deviation over several seeds, is needed to substantiate the claimed statistical significance.","section":"§4.3 and §5.2"},{"comment":"The reasoning generation step conditions on the gold answer and produces reasonings whose faithfulness is uneven, as the paper's own Case 2 shows: the model predicts 'fear' correctly but generates a justification for 'joy'. The paper acknowledges this issue in the discussion, but it does not quantify the quality, input-dependence, or faithfulness of the generated reasonings across the training set. Since the proposed method's benefit is argued to derive from the reasoning's semantic content, the manuscript should either measure reasoning quality or include the shuffled-reasoning control from the first major comment to demonstrate that content, rather than the mere presence of extra text, is what drives the improvement.","section":"§3.2 and Table 2, Case 2"}],"minor_comments":[{"comment":"There is a typo in the section heading: 'T raining Data for Reasoning Generation' should be 'Training Data for Reasoning Generation', and later in the same section 'thesyvai/reasoning-gen' is missing a space.","section":"§3.1"},{"comment":"The decoding parameters for the offline reasoning generation step (temperature, top-p, max new tokens, repetition penalty, etc.) are not reported anywhere, which makes the data augmentation step non-reproducible. Please add these to the appendix.","section":"§3.2 and Appendix A"},{"comment":"The sentence 'Prior work has explored jointly training models to predict labels and generate explanations' cites Wiegreffe and Pinter [12], which is a review of datasets for explainable NLP; please cite the original works, such as Rajani et al. [7] and Camburu et al. [3], in this sentence as well.","section":"§2, Related Work"},{"comment":"The GPT-4.1 zero-shot baseline is not a strong comparison for the proposed approach, since it is not fine-tuned; reporting the 26.4-point gap over it may overstate the method's advantage. Consider repositioning this baseline as a reference point rather than a primary comparison.","section":"§5.2, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's main contribution is conditional on demonstrating that reasoning content, not output length or format, causes the improvement. The suggested shuffled-reasoning control is standard and should be feasible within the scope of this paper. The authors already release the augmented datasets, which is a strength; adding the control and proper paired significance testing would make the paper acceptable. The use of a dataset created by the same authors (syvai/reasoning-gen) is not itself a problem, but the paper should specify its provenance and split more clearly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a useful empirical study of an idea that is not new. The specific result—an 8.7-point accuracy gain on dair-ai/emotion from training a generative classifier to emit (reasoning, label) instead of just (label)—is not in the cited literature, and the authors are honest about their limitations. The two-stage split, where a general reasoning generator trained on math/code/science is applied offline to build the downstream set, is a mild twist on 'Distilling Step-by-Step!' and STaR. Releasing the augmented datasets is a concrete contribution.\n\nThe main results are clear and the qualitative examples (Table 2) are a plus, especially the failure cases where the reasoning is flawed but the prediction is correct. That undercuts any strong interpretability claim, and the authors do not overclaim there.\n\nThe load-bearing problem is the control condition. Q->RA and Q->A differ in output length, format, and alignment with the base model's instruction-following objective. Llama-3.2-1B-Instruct is trained to generate text, so forcing it to output a single label may be an unnatural task that depresses the baseline. Without a control that emits fixed-length filler text, or shuffled reasonings reassigned to different inputs, the 8.7 points cannot be attributed to the semantic content of the reasoning. The stress-test note lands: this confound directly undermines the central interpretation, not just its statistical stability. The statistical test is also wrong—two-proportion z-tests assume independent samples, but the predictions are paired; a McNemar test is appropriate. And there is a single run per condition, so no variance estimate. These are all fixable with modest effort.\n\nThe surprise-class collapse (13.8% to 1.5%) is honestly discussed. I take the authors' explanation that reasoning quality for minority classes is a factor, but the format change could also be responsible.\n\nThis paper deserves a serious referee because the question—does generated reasoning help, or is it sequence length and format—is worth pinning down, and the released datasets make the follow-up cheap. With a shuffled-reasoning control, multiple seeds, and a paired test, it could be a solid paper. As is, it is a borderline workshop contribution with a real, reproducible empirical finding and an over-broad interpretation.","headline":"A useful but confounded empirical study: the 8.7-point gain is real, but the attribution to reasoning content does not survive the missing control for output length and format.","tokens_in":8016,"tokens_out":2700,"would_cite":false,"duration_ms":27165,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reasoning before the label lifts emotion accuracy by 8.7 points.","keywords":["reasoning-infused learning","generative classification","emotion classification","LLM-generated reasoning","data augmentation","chain-of-thought","explanatory supervision","interpretable classifiers"],"falsifier":"Fine-tune the same baseline model to output a fixed, non-reasoning prefix (for example, \"The emotion expressed is\") followed by the label, matching the proposed model's output length and token count; if accuracy rises to roughly 58.4%, the reported gain is not caused by reasoning content. Alternatively, train with randomly shuffled reasonings attached to labels and check whether the accuracy gain disappears.","tokens_in":7008,"feed_emoji":"🧠","tokens_out":5970,"duration_ms":62467,"temperature":0.7,"pith_summary":"This paper claims that a classifier trained to write a short explanation before its label learns to classify better than the same model trained to output only the label. The authors build the explanations offline with a small reasoning generator fine-tuned on a general-purpose corpus, then fine-tune a downstream generative classifier on pairs of input text and reasoning-plus-label output. On an emotion-classification test set, this raises accuracy from 49.7% to 58.4%, an 8.7-percentage-point gain. The authors interpret the gain as evidence that the semantic content of reasoning, not just the label, is a useful supervisory signal.","feed_headline":"Reasoning before the label lifts emotion accuracy by 8.7 points","feed_subtitle":"A label-only baseline scores 49.7%; adding a generated explanation before the answer reaches 58.4%.","key_machinery":"The load-bearing object is the augmented training target: for each input text Q, the training sequence is the concatenation of a reasoning R and the gold label A, where R was generated offline by a model fine-tuned to produce reasoning from question-answer pairs. The downstream model is trained with a standard language-modeling objective to generate this whole sequence from Q alone, so reasoning becomes part of the learned mapping rather than a prompt-time add-on. This lets a 1-billion-parameter model carry reasoning ability from a general corpus into a specific classification task and emit an explanation at inference time.","core_discovery":"The paper's central discovery is that prepending a generated reasoning to the target label changes what a small generative classifier learns: the model trained to emit reasoning followed by the label reaches 58.4% accuracy on the emotion test set, while an identical model trained to emit only the label reaches 49.7%. The reasoning generator was trained on a broad corpus of question-answer-reasoning triples spanning math, code, and science, and it transfers to the emotion domain without task-specific reasoning examples. Per-class gains are concentrated in the frequent classes, with sadness rising 19.6 points and fear 18.2 points, while the rare surprise class collapses from 13.8% to 1.5% accuracy, which the paper attributes to low-quality generated reasonings for that class.","pith_inferences":["The paper does not compare against a control that outputs a fixed non-reasoning prefix before the label, so part of the 8.7-point gain could come from the longer target sequence rather than from the reasoning's semantic content; a filler-text control would isolate the reasoning effect.","If the gain is genuinely semantic, then replacing generated reasonings with random or template explanations should degrade accuracy, which is a cheap test the paper does not run.","The method may invite reasoning shortcuts where the model learns to produce plausible-sounding text that rationalizes a guess rather than reflecting its decision process; the paper's own example of a correct label paired with a wrong justification already shows this failure mode.","The reasoning generator could be iteratively improved by filtering or regenerating low-quality reasonings for minority classes, which would test whether the surprise-class collapse is fixable."],"forward_implications":["If the central claim holds, the same two-stage recipe can be applied to other classification tasks without human-written explanations, using a general reasoning generator to synthesize training data.","The gain implies that reasoning content can act as a richer supervisory signal than labels alone, so datasets augmented this way may improve downstream performance beyond what label-only fine-tuning achieves.","The framework yields explanations at inference time as a byproduct, giving applications a built-in trace of why a text was assigned a label.","Performance will depend on the quality of generated reasoning, especially for rare classes, so reasoning-quality filtering becomes a natural next lever.","The surprise-class collapse indicates that imbalanced targets need special handling before this method is applied broadly."],"supporting_citations":[{"why":"Supplies the base instruction-tuned model that is fine-tuned both as the reasoning generator and as the downstream classifier.","marker":"[1]"},{"why":"Establishes that intermediate reasoning steps improve LLM performance, the premise that this paper distills into a small fine-tuned model.","marker":"[11]"},{"why":"Shows that training on human-written explanations can improve performance and generalization, the empirical precedent for the method.","marker":"[7]"},{"why":"Frames explanations as a valuable supervisory signal, which the two-stage pipeline operationalizes at scale.","marker":"[12]"},{"why":"Shows that zero-shot reasoning can be elicited without task-specific examples, motivating the general-purpose reasoning generator.","marker":"[5]"}],"fun_headline_variants":["Reasoning before the label boosts emotion accuracy 8.7 points","Adding a reasoning step improves emotion classification by 8.7 points","LLM-generated reasoning lifts small model's emotion accuracy","Explain-then-classify beats label-only for emotion detection","Two-stage reasoning training yields 8.7-point emotion accuracy gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that the accuracy gain comes from the content of the generated reasoning, because the proposed model's training target is a long explanation followed by the label while the baseline's target is just the label; no control separates reasoning from extra text before the label.","fun_headline_variants_meta":{"raw":{"variants":["Reasoning before the label boosts emotion accuracy 8.7 points","Adding a reasoning step improves emotion classification by 8.7 points","LLM-generated reasoning lifts small model's emotion accuracy","Explain-then-classify beats label-only for emotion detection","Two-stage reasoning training yields 8.7-point emotion accuracy gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000659,"raw_usage":{"total_tokens":3034,"prompt_tokens":986,"completion_tokens":2048,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":1962}},"tokens_in":602,"tokens_out":2048,"duration_ms":14330,"temperature":1.0,"reasoning_tokens":1962,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:20:33.233639+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fine-tune the same baseline model to output a fixed, non-reasoning prefix (for example, \"The emotion expressed is\") followed by the label, matching the proposed model's output length and token count; if accuracy rises to roughly 58.4%, the reported gain is not caused by reasoning content. Alternatively, train with randomly shuffled reasonings attached to labels and check whether the accuracy gain disappears.","supporting_citations":[{"cited_title":"Meta llama 3.2: A 1.4t parameter class of language models","cited_arxiv_id":null,"evidence_quote":"Supplies the base instruction-tuned model that is fine-tuned both as the reasoning generator and as the downstream classifier."},{"cited_title":"Chain-of-thought prompting elicits rea- soning in large language models","cited_arxiv_id":null,"evidence_quote":"Establishes that intermediate reasoning steps improve LLM performance, the premise that this paper distills into a small fine-tuned model."},{"cited_title":"Explain yourself! : Leveraging language models for faithful rationalization","cited_arxiv_id":null,"evidence_quote":"Shows that training on human-written explanations can improve performance and generalization, the empirical precedent for the method."},{"cited_title":"Spiraling of sub-Riemannian geodesics around the Reeb flow in the 3D contact case","cited_arxiv_id":"2102.12741","evidence_quote":"Frames explanations as a valuable supervisory signal, which the two-stage pipeline operationalizes at scale."}],"review_version":1}