{"id":"f3703969-5d42-404f-9f54-d13a8a21034c","arxiv_id":"1909.00734","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A two-step neural text generator with sentence-level keyphrase planning and style control improves coherence and style fidelity over standard sequence-to-sequence baselines.","lead":"This paper builds a neural text generator that first plans which key ideas to mention and in what style, then writes the sentences. It shows this two-step design beats simpler one-step generators on argument writing, Wikipedia paragraphs, and scientific abstracts.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Wikipedia 'style' labels are sentence-length bins; the style-vs-no-style ablation may only demonstrate length control, not linguistic style control.","rationale":"The paper proposes a two-step content-planning and style-controlled generation model and provides experiments on three datasets. The strongest evidence for the style component is the ablation w/ Style vs w/o Style on argument and Wikipedia tasks. I agree with the reader that the automatically constructed style labels are the weakest load-bearing assumption. I narrow the concern to Wikipedia, where style is operationalized as sentence-length bins. Since the w/o Style ablation removes both style semantics and length information, the comparison is confounded: the decoder may simply benefit from a length prior, not from stylistic control. This is a correctable concern, not a fatal one; the argument-task labels are more semantically grounded, and the content-planning contribution is supported independently by the oracle-planning results and the F1/BLEU correlation in Figure 3. The paper's own limitation note (Section 6.3) that system arguments 'overfit on stylistic languages' further indicates the style component may capture surface patterns rather than deep functional control. I also note the abstract's blanket claim of significantly outperforming competitive comparisons is contradicted by the AGENDA results in Table 5, where GraphWriter scores higher on all metrics; this strengthens the need for a conditional verdict. The proposed concrete test (a length-conditioned control and human validation of the length-bin labels) would settle whether the Wikipedia style effect is genuinely stylistic. Because the reader already arrived at a CONDITIONAL verdict, my concern does not change the recommended verdict, hence UNCHANGED. The concern is raised in good faith and does not question the integrity of the authors; it targets the interpretability of an ablation and the validity of a proxy label.","tokens_in":19268,"tokens_out":8058,"duration_ms":88337,"concrete_test":"Add a control variant for Wikipedia generation that conditions the decoder on a continuous sentence-length embedding (or the predicted length bin) but carries no 'simple/normal' semantics, and compare against w/ Style and w/o Style on BLEU, ROUGE, METEOR, and the human judgments. If the length-conditioned baseline matches or exceeds w/ Style, the style-specific effect is not demonstrated. As a second check, annotate a random sample of 100 Wikipedia sentences with human simplicity ratings, compute agreement with the automatic length bins, and report style-prediction accuracy on the dev set; low agreement or low prediction accuracy would indicate the style labels are not a valid proxy for linguistic style.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that explicit style specification improves generation rests heavily on ablations that remove the style input. For Wikipedia, style labels are assigned purely by sentence-length bins (Section 4.2, Appendix A.2). The one-hot style vector fed to the decoder (Eq. 9) is thus a length category, and the w/o Style ablation removes not only stylistic information but also any length prior. The observed BLEU/ROUGE gains and the human 'fluent/correct' improvements could therefore be due to the decoder being told the target sentence-length range, not due to controlling linguistic simplicity. This is a real confound: the length distribution differs strongly between normal and simple Wikipedia (Figure 5), so a length bit alone can improve metric scores by aligning output length with the reference. Moreover, the paper does not report style-prediction accuracy or any validation that the automatically assigned length bins correspond to perceived simple vs normal language. In the argument task the CLAIM/PREMISE/FUNCTIONAL rules are more semantically motivated, but the Wikipedia experiment is a large part of the empirical support, and the style claim is not cleanly established there.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an end-to-end trained two-step neural text generation model with sentence-level content planning and explicit style specification. A planning decoder first selects keyphrases from a memory bank for each sentence and predicts a sentence-level style label; a realization decoder then generates the output conditioned on the planned content and style. The model is evaluated on three tasks: persuasive counter-argument generation from Reddit ChangeMyView, paragraph generation for normal and simple Wikipedia, and paper abstract generation on AGENDA. The authors report automatic BLEU/ROUGE/METEOR scores, ablations that remove style or content planning, and a human study on two of the tasks, concluding that content planning and style specification improve fluency, correctness, and coherence relative to one-step baselines.","tokens_in":19490,"tokens_out":4295,"duration_ms":51530,"significance":"If the reported gains are robust, the paper makes a useful contribution to neural text generation by demonstrating that separating sentence-level content selection from style-conditioned realization can improve faithfulness and coherence on longer outputs. The work is strengthened by evaluating on three diverse datasets, by providing ablations with oracle plans, and by making data and code available. The central idea is timely, and the planning module's use of selection history is a plausible improvement over static content selection. However, the significance is currently limited by two empirical weaknesses: the Wikipedia style manipulation is confounded with sentence length, and the system underperforms the leading graph-based baseline on AGENDA despite the abstract's general claim of outperforming competitive comparisons. The human evaluation also does not uniformly support the fluency claim for Wikipedia. These issues are fixable and do not invalidate the overall architecture, but they need to be addressed before the paper's conclusions can be accepted as stated.","major_comments":[{"comment":"The Wikipedia sentence-style labels are defined purely by sentence-length bins, so the one-hot style vector fed to the decoder in Eq. (9) is a length category. The 'w/o Style' ablation therefore removes length information together with style information, and because normal and simple Wikipedia differ strongly in sentence-length distribution (Figure 5), the reported BLEU/ROUGE and human correctness gains may reflect matching the reference length distribution rather than controlling linguistic style. Please report style-prediction accuracy, add an ablation that conditions only on length, and ideally include human judgments of perceived 'simple' versus 'normal' style to establish that the style variable captures something beyond length.","section":"§4.2, Appendix A.2, Eq. (9)"},{"comment":"The abstract states that the system 'can significantly outperform competitive comparisons,' but Table 5 shows that on AGENDA the model is below GraphWriter on BLEU (20.32 vs. 29.95), ROUGE-L (23.30 vs. 28.56), and METEOR (15.95 vs. 19.90), and even with oracle plans it reaches only 25.03/26.18/19.21. The current wording overgeneralizes the result. Please qualify the claim by dataset or by the settings in which the advantage actually holds.","section":"Abstract, §6.1, Table 5"},{"comment":"Argument style labels are constructed by hand-written surface rules applied to the target sentences (e.g., 'CLAIM' requires patterns such as 'i (don't)? (believe|agree|...)' and length under 20 tokens). No validation of these labels is reported, and style-prediction accuracy is not given. Because the style classifier is trained on labels generated by the same patterns, the style-specification gains in Table 3 and Table 6 may reflect learning to predict surface features of the gold text rather than meaningful discourse functions. Please add label-quality validation (e.g., comparison with human annotation) and report the style-prediction accuracy on held-out data.","section":"§4.1, Appendix A.1, §3.2"},{"comment":"The text says that style-aware output is rated 'more fluent and correct' compared with the variant without style, but the significance markers in Table 6 do not support this uniformly. On Wikipedia, grammaticality (3.38 vs. 2.99) and content richness (3.43 vs. 3.50) are not marked as statistically significant, and content richness is numerically lower for the style-aware model. On the argument task, content richness is not improved at a significant level either. Please align the human-evaluation claims with the reported significance tests and discuss the null or negative results.","section":"§6.2, Table 6"}],"minor_comments":[{"comment":"There is a typo in the first sentence of Section 6.1: 'precesion-oriented' should be 'precision-oriented.'","section":"§6.1"},{"comment":"Section 4.2 says the global Wikipedia style is encoded as one extra bit appended to mj in Eq. (2), but the equation does not show this bit. Please clarify whether the global style bit also influences the keyphrase selection probability in Eq. (4) or only the planner hidden state.","section":"§3.2, §4.2"},{"comment":"The caption uses 'Notice'; use 'Note' for a formal style.","section":"Table 5 caption"},{"comment":"Several sample outputs contain clear repetitive n-grams (e.g., 'the the moon the', 'in 2013 and 2013 respectively', 'cable cable channel 2'). Although these are examples, they suggest that the trigram repetition disallowal is not fully effective; a quantitative repetition analysis across the test set would be informative.","section":"Figures 4, 8, and 9"},{"comment":"The hyperparameters gamma and eta are set to 1.0 'for simplicity'; no sensitivity analysis is reported. A sentence noting that results were robust to nearby values (or a small grid search) would strengthen the reproducibility claims.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the planner with selection history and the style-conditioned decoder is a sensible incremental contribution, and the new aligned Wikipedia corpus is a real asset. The weak joint is the Wikipedia style signal: it is just sentence length, so the style ablation shows length control more than linguistic style control.\n\nWhat is actually new: the qj query for selection history (Eq. 3-4) is a small but real improvement over Hua et al. 2019. Feeding the predicted one-hot style into the decoder (Eq. 9) is a clean way to condition on style. The aligned normal/simple Wikipedia dataset is not in prior work. The paper runs three tasks, reports BLEU/ROUGE/METEOR plus ablations and a human study, and ships code and data. That is solid workmanship.\n\nWhere it gets soft: on Wikipedia, sentence-level \"style\" is defined as four length bins (Section 4.2, Appendix A.2). The w/o Style ablation removes the length category bit, so the BLEU/ROUGE and human grammaticality/correctness gains could simply reflect the decoder knowing the target length range, not simplicity of language. The paper never reports style-prediction accuracy or validates that the bins correspond to perceived simple vs. normal style. The human evaluation only compares against their own w/o Style variant; it does not compare to a length-aware baseline or a competitive system. So the style-control claim on Wikipedia is not cleanly established. The argument-task styles (CLAIM/PREMISE/FUNCTIONAL) are more semantically grounded, so that part is healthier. Also, the abstract's \"significantly outperform competitive comparisons\" is too strong given that on AGENDA the model is below GraphWriter on all metrics; the body text correctly says \"competitive\" once oracle plans are used. That gap between abstract and results should be fixed.\n\nBottom line: the planning/realization architecture and the Wikipedia dataset are worth engaging with, but the style-control evidence needs a length-conditioned control and a calibration of the claims. This deserves peer review, not a desk reject, and a good reviewer will ask for those controls.","headline":"A solid planner-realizer with a useful new Wikipedia corpus, but the Wikipedia style signal is just sentence length, so the style-control evidence is weaker than the abstract claims.","tokens_in":19993,"tokens_out":2490,"would_cite":false,"duration_ms":32583,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-step neural model that first plans each sentence's content and style, then writes it, produces more coherent and faithful text than one-step baselines on arguments, Wikipedia paragraphs, and scientific abstracts.","keywords":["text generation","content planning","style control","keyphrase selection","sentence-level planning","neural text generation","argument generation","text simplification"],"falsifier":"A controlled experiment that swaps the style labels for random but equally distributed categorical labels and retrains the model would test the mechanism: if the random-label model matches the reported improvements over the no-style variant, then the specific style specification is not doing the work. Alternatively, if human raters presented with generated sentences cannot distinguish CLAIM from PREMISE sentences, or short from long style classes, above chance, the styles are not being realized in the output.","tokens_in":19061,"feed_emoji":"✍️","tokens_out":4354,"duration_ms":39346,"temperature":0.7,"pith_summary":"The paper argues that the classic decomposition of text generation into content selection, planning, and surface realization can be restored inside an end-to-end neural system without giving up end-to-end training. Its proposed model uses two decoders: a sentence-level content planner that picks keyphrases from a memory bank while tracking what has been selected, and also assigns each sentence a categorical style; then a realization decoder writes the sentence with that style encoding fed into it. The paper claims this separation produces more coherent, faithful, and fluent text than one-step sequence-to-sequence models, and supports the claim with automatic metrics and human ratings on Reddit counter-arguments, normal and simple Wikipedia introductions, and scientific abstracts. The broader point is that interpretable intermediate decisions around what to say and how to say it can be learned and explicitly used, rather than left as uncontrolled side effects of a single decoder.","feed_headline":"Plan sentences first, then write: two-step model beats one-step text generation","feed_subtitle":"A sentence-level planner chooses keyphrases and style before a second decoder writes, lifting BLEU, ROUGE, and METEOR across three domains.","key_machinery":"The central mechanism is the two-decoder decomposition: a sentence-level content-planning LSTM whose attention query is the running sum of previously selected keyphrase encodings, so each selection is conditioned on the history and tends to avoid repetition, and which also emits a per-sentence style label; and a surface-realization LSTM that appends that one-hot style vector to its hidden state and context vectors, so word choice is explicitly conditioned on the intended style. The whole system is trained jointly by summing word-generation loss, keyphrase-selection cross-entropy loss, and style-prediction cross-entropy loss.","core_discovery":"On the paper's own terms, an end-to-end trained two-step model with a content-planning LSTM that selects keyphrases sentence by sentence while conditioning on the selection history, and a style-specification head that predicts a per-sentence category such as CLAIM, PREMISE, or FUNCTIONAL for arguments or a length-based complexity bin for Wikipedia, followed by a realization LSTM that receives the predicted style as an extra input, generates text that automatic metrics and human judges rate above one-step baselines and above the same model without style specification. Experiments across three tasks show significant improvements in BLEU, ROUGE, and METEOR over sequence-to-sequence, retrieval, and prior planning-based systems, and on AGENDA the model is competitive with a graph-transformer state of the art while using only titles and entities as input. Ablations show that removing style specification hurts scores and that providing oracle keyphrase plans helps further, indicating that both planning and style conditioning carry the gain.","pith_inferences":["A natural testable extension is replacing the hand-written argument style rules and length-based Wikipedia bins with learned or latent style categories; if the gains persist, the mechanism is about explicit style conditioning rather than this particular taxonomy.","The history-aware selection mechanism resembles neural checklist methods and could transfer to summarization or dialogue, where sentence-level what-to-say decisions are central.","The style-conditioned realizer could be applied to controllable readability or formality rewriting by inferring style labels from parallel data rather than from hand-crafted surface patterns.","Because the AGENDA experiments omit style specification, the paper's claim about style control is only directly tested where style labels exist; the abstract-generation gains must be attributed to planning alone."],"forward_implications":["If the result holds, end-to-end neural systems can recover the classical NLG structure of content selection, ordering, and realization without sacrificing end-to-end differentiability.","Style specification is portable across domains: the same mechanism improves persuasive argument text, encyclopedic paragraphs, and simplified-language text, so it likely transfers to other controlled-generation settings.","Content-selection quality is a strong predictor of generation quality, with Pearson correlations of 0.95 and above between keyphrase-selection F1 and BLEU on Wikipedia, so better selection models should directly improve output.","Oracle keyphrase plans consistently beat predicted plans, giving a clear upper-bound signal and pointing to content selection as the next bottleneck.","The no-style ablation's drop shows that explicit style conditioning, not just planning, drives the reported gains."],"supporting_citations":[{"why":"Provides the prior argument-generation framework with retrieval, planning, and realization that this paper extends with history-aware selection and style-conditioned realization.","marker":"Hua et al. (2019)"},{"why":"Establishes the planning-then-realization separation that the paper improves on by conditioning selection on history and feeding style into the realizer.","marker":"Moryossef et al. (2019)"},{"why":"Supplies the AGENDA dataset and the GraphWriter state-of-the-art baseline that the paper compares against on abstract generation.","marker":"Koncel-Kedziorski et al. (2019)"},{"why":"Provides the pointer-generator copying mechanism that the paper uses to allow out-of-vocabulary words from the input and keyphrase bank.","marker":"See et al. (2017)"},{"why":"Supplies the stance classifier the paper uses at test time to filter retrieved passages for persuasive-argument generation.","marker":"Bar-Haim et al. (2017)"},{"why":"Provides the claim-detection patterns that the paper extends into the rule-based CLAIM, PREMISE, and FUNCTIONAL style labels for argument data.","marker":"Levy et al. (2018)"},{"why":"Defines the attention-based sequence-to-sequence baseline that the paper improves upon on all three tasks.","marker":"Bahdanau et al. (2015)"}],"fun_headline_variants":["Two-step generation: plan content and style, then write","Sentence-level planning boosts neural text generation","Style-aware sentence planning improves text quality","Plan keyphrases and style first, then generate text","Neural text generation with two-step planning beats one-step"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The automatically constructed sentence-level style labels (hand-written surface patterns for argumentative function, and sentence-length bins for Wikipedia) must correspond to the linguistic styles the model is meant to control; if they only reflect surface statistics, the reported style-control improvements could be artifacts of matching those statistics rather than true stylistic control.","fun_headline_variants_meta":{"raw":{"variants":["Two-step generation: plan content and style, then write","Sentence-level planning boosts neural text generation","Style-aware sentence planning improves text quality","Plan keyphrases and style first, then generate text","Neural text generation with two-step planning beats one-step"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2578,"prompt_tokens":920,"completion_tokens":1658,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":536,"completion_tokens_details":{"reasoning_tokens":1585}},"tokens_in":536,"tokens_out":1658,"duration_ms":123214,"temperature":1.0,"reasoning_tokens":1585,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:37:07.068560+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled experiment that swaps the style labels for random but equally distributed categorical labels and retrains the model would test the mechanism: if the random-label model matches the reported improvements over the no-style variant, then the specific style specification is not doing the work. Alternatively, if human raters presented with generated sentences cannot distinguish CLAIM from PREMISE sentences, or short from long style classes, above chance, the styles are not being realized in the output.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the prior argument-generation framework with retrieval, planning, and realization that this paper extends with history-aware selection and style-conditioned realization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the stance classifier the paper uses at test time to filter retrieved passages for persuasive-argument generation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the claim-detection patterns that the paper extends into the rule-based CLAIM, PREMISE, and FUNCTIONAL style labels for argument data."}],"review_version":1}