Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Long and Diverse Text Generation with Planning-based Hierarchical Variational Model

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Planning first, then sentence-by-sentence writing, makes long text generation both more complete and far more varied.

desk verdict A real architectural contribution to data-to-text with credible diversity gains, but the coverage headline is entangled with the same string-match heuristic used for training. read the letter →

arxiv 1908.06605 v2 pith:LQ4FRG7Y submitted 2019-08-19 cs.CL cs.LG

Zhihong Shao , Minlie Huang , Jiangtao Wen , Wenfei Xu , Xiaoyan Zhu This is my paper · ORCID
classification cs.CLcs.LG
keywords data-to-textgenerationcontentplanninghierarchicalvariationalautoencoderlongtextdiversitycoveragerepetitionconditionalVAE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Long data-to-text generation usually asks one decoder to do three jobs at once: choose which input facts to mention, keep successive sentences coherent, and vary the wording. The paper's claim is that these failures are best attacked by separating a high-level plan from low-level realization. The proposed PHVM first partitions the input into a sequence of groups, each group being the subset of items one sentence should cover, and then generates each sentence conditioned on its group, the previous context, and two tiers of latent variables: a global planning latent that captures the variety of reasonable orderings and per-sentence local latents that control expression and carry dependencies across sentences. On a new advertising-text dataset and on recipe generation, this design raises coverage and distinct-n diversity while cutting 4-gram repetition from 36.58% to 3.90% on advertising text. A sympathetic reader should take away that planning plus hierarchical variability, rather than a more powerful single decoder, is the load-bearing idea.

What carries the argument

The central object is the group plan combined with a hierarchical variational latent structure. A plan is a sequence $g = g_1 \cdots g_T$ in which each $g_t$ is a subset of input items meant to be covered by sentence $s_t$; a GRU plan decoder selects items with a per-item sigmoid conditioned on the global latent $z_p$ and the bag-of-words of previous groups, and stops when its stop probability exceeds 0.5. Generation then proceeds sentence by sentence: a sentence decoder produces a representation $h_t^s$ and samples a local latent $z_t^s$, with $z_t^s$ depending on previous local latents through the recurrent state, and a word decoder realizes $s_t$ from $\{h_t^s, z_t^s, g_t\}$. The training objective is an evidence lower bound over planning and sentence generation, plus a stop-signal loss and per-sentence bag-of-word losses, with KL annealing to counter collapse. This machinery is what lets the model first decide what to say and only then decide how to say it.

What would settle it

Have annotators mark the true input items expressed by each sentence on a random sample of the advertising and recipe training data, then compare those human groups with the string-match groups used for training; if agreement is low, or if retraining PHVM with human-annotated plans changes coverage and repetition outcomes on a fixed test set, the planning-supervision premise is decided.

Watch

Extended reading notes

Core claim

The paper establishes that a planning-based hierarchical variational model outperforms state-of-the-art data-to-text baselines on long, diverse generation. On advertising text, PHVM covers 87.05% of input attribute-value pairs (best baseline 85.97%), reaches distinct-4 of 72.87% against 41.69% for the best baseline, and reduces repetition-4 to 3.90% from 36.58%; manual pairwise comparisons favor it on both grammaticality and coherence. On recipes, it attains 73.2% coverage and 67.86% distinct-4, again best among the compared models, with repetition-4 at 17.3%. Ablations attribute part of the gain to the global planning latent, whose removal lowers coverage and distinct-4, and part to the local sentence latents, whose removal raises repetition-4 to 20.88% and shortens sentences. The intended message is that decomposing generation into planned, dependent sentence sub-tasks with variational control at both levels is what produces complete, coherent, and varied long texts.

Load-bearing premise

The entire planning supervision rests on string-match heuristics that approximate which input items each sentence covers, so if paraphrases or split mentions make those extracted groups unreliable, the plan decoder learns from corrupted targets and the reported coverage and coherence gains weaken.

Editorial extensions

If this is right

  • If the central claim holds, long structured-to-text tasks beyond advertising copy and recipes—such as game reports, biographies, or event summaries—can be improved by inserting a similar group-planning stage before sentence realization.
  • Repeatedly sampling the latent variables yields multiple high-quality texts for the same input: with ten samples per input, 79.0% of the 1,000 generated advertising texts scored above 4 on a 1–5 scale, and the average self-BLEU across texts was only 16.87%.
  • The ablation results imply that the global planning latent is specifically responsible for planning completeness and diversity, while local latents are responsible for expression variation and cross-sentence dependency; removing local latents raises repetition-4 to 20.88%.
  • The recipe results suggest the approach generalizes to other domains: PHVM uses more given ingredients, generates more varied cooking steps, and produces the generic phrase 'all ingredients' less often than the baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a testable extension is replacing the string-match reference plans with human-annotated or learned alignments, which would show how much of the coverage gain depends on the quality of the planning supervision, since paraphrase-heavy sentences are exactly where string matching misassigns groups.
  • Beyond the paper: the same plan-and-realize decomposition could apply to long-form machine translation or story generation, where no explicit input table exists; in those settings one could treat the latent plan as an unsupervised intermediate and evaluate whether coherence and diversity improve over flat hierarchical decoders.
  • Beyond the paper: the large reduction in repetition-4 suggests the planning mechanism may be acting as a coverage regularizer; one could test this by attaching a group-planning head to an existing attention-based generator and comparing repetition and coverage without variational latents.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a Planning-based Hierarchical Variational Model (PHVM) for long and diverse data-to-text generation. The model first generates a sequence of groups of input items via a planning decoder, then generates each sentence conditioned on its group, previous context, a global planning latent variable, and local sentence-level latent variables. The method is evaluated on a new Chinese advertising dataset and on a recipe generation dataset. The reported results claim that PHVM outperforms four baselines in coverage, diversity (distinct-4, repetition-4), and human-judged grammaticality and coherence. The paper also includes ablations, a planning-diversity analysis, and a case study.

Significance. If the claims hold, PHVM addresses two recognised problems in neural data-to-text generation: long-text coherence and expression diversity. The model is clearly specified, the code and corpus are released, and the experimental section is extensive, including automatic metrics, manual pairwise evaluation with inter-annotator agreement, ablation studies, and a new dataset for long diverse text generation. The diversity gains (distinct-4 and repetition-4) are not tied to the string-match heuristic and are large and internally consistent across both tasks. However, the central coverage claim is weakened by the fact that the coverage metric uses the same string-match heuristic that provides the supervision signal for the plan decoder, and by the dataset construction which filters out uncovered attribute-value pairs. These issues undermine the evidence that planning is responsible for the coverage improvements, although they do not invalidate the diversity or coherence findings. The paper merits revision to provide an independent evaluation of coverage and to justify or correct the planning-supervision loop.

major comments (3)
  1. [5.3, 5.4, Eq. (6), Table 3] The reference plans used to train the plan decoder are extracted by 'string match heuristics' (Sec. 5.3), and the coverage metric is computed by the same 'string match heuristics' (Sec. 5.4). Since Eq. (6) directly optimizes the plan decoder to reproduce these extracted groups, the coverage metric is not independent of the training objective. The reported 87.05% coverage in Table 3 may partly reflect PHVM's ability to match surface tokens rather than to produce semantically complete content. Please add a human-annotated coverage evaluation on a sample of generated texts, or an alternative metric that does not rely on exact string matching (e.g., semantic similarity or textual entailment), to validate the coverage claim.
  2. [5.1, Table 1] The advertising dataset is constructed by removing any attribute-value pair that does not occur in the corresponding text and adding pairs that occur in the text but not in the table. This guarantees that every reference text covers all input pairs by construction, which simplifies the coverage task and makes the coverage score trivially bounded at 100% for the training data. This construction may favor a model explicitly supervised to place input tokens in specific groups (as PHVM is) over models that generate content without token-level plan supervision. Please report coverage on a held-out set constructed without such filtering, or otherwise justify that the filtering does not bias the comparison in favor of PHVM.
  3. [4.3] The paper states that 'the extracted plans are sufficient to train a good plan decoder' but provides no supporting evidence. Since the plan decoder is a core component of the model, the assertion is load-bearing for the claimed causal link between planning and the reported gains. Please provide either (a) an estimate of plan-extraction accuracy on a sample, (b) a comparison of PHVM trained with heuristic plans versus plans corrected by human annotators, or (c) an analysis of how extraction errors propagate to final generation quality. Without one of these, the planning supervision remains an uncontrolled source of noise that limits the interpretability of the experimental results.
minor comments (5)
  1. [Figure 2] The caption of Figure 2 says 'group' but the figure's internal labels (e.g., the small subscripts and the 'Probability of discarding/selecting' arrows) are not defined in the caption. Please clarify the notation so that the planning and generation flow is understandable without referring back to the text.
  2. [5.5.3, Figure 3] The y-axis labels 'Avg. Score' and 'Avg. # Distinct Planning' are not defined precisely. Please specify how 'distinct planning' is counted (e.g., exact group-sequence equality, or some distance measure) and how the average score is computed.
  3. [Table 4] The Fleiss' kappa values appear as column headings ('Grammaticality κ' and 'Coherence κ'). It would be clearer to present the kappa values in a separate row or explicitly state in the caption that they denote inter-annotator agreement coefficients.
  4. [5.3] The paper states that the best model is selected by 'L1 + L2' on the validation set, but L3 (bag-of-word loss) is part of the training objective. Please clarify whether L3 was intentionally excluded from model selection and explain the rationale.
  5. [5.4] The BLEU scores in Tables 3 and 5 are all very low (around 2-5). A brief comment on why BLEU is a weak metric for this task (e.g., due to the high diversity of the reference texts) would help readers interpret those numbers.

Circularity Check

1 steps flagged · score 6.0 of 10

Coverage is scored by the same string-match heuristic used to build the plan decoder's supervised target.

  1. fitted input called prediction [Section 5.3 (Implementation Details; Eq. 6) and Section 5.4 (Automatic Evaluation Metrics)]
    "As we need to train the plan decoder with full supervision, we extracted plans from the texts by recognizing attribute values (or ingredients) in each sentence with string match heuristics. ... Although our extraction method can introduce errors, the extracted plans are sufficient to train a good plan decoder. ... (2) Coverage: This metric measures the average proportion of input items that are covered by a generated text. We recognized attribute values (ingredients) with string match heuristics."

    Eq. (6) optimizes the plan decoder against reference groups ~g_t produced by the same string-match heuristic that Section 5.4 uses to define Coverage. Thus the model is supervised to predict exactly which value/ingredient strings appear in each sentence, and Coverage then rewards a generated sentence whenever those same strings occur. The automatic coverage gains in Tables 3 and 5 therefore partly reflect training to the evaluation protocol, not an independent measure of planning quality. The manual coherence and distinct-4 results are independent, so the circularity is partial, but the coverage claim is not backed by a metric separate from the training signal.

full rationale

PHVM's architecture and hierarchical latent-variable derivation are not circular: the ELBO, plan decoder, and sentence decoder are all specified independently of the conclusions, and the recipe experiments plus manual pairwise evaluation provide external evidence. The significant circular step is confined to the automatic Coverage metric. The reference plans used as supervised targets in Section 5.3 are extracted by recognizing attribute values/ingredients with string-match heuristics, and Section 5.4 defines Coverage by recognizing the same values/ingredients with the same string-match heuristics. Eq. (6) optimizes the plan decoder against those extracted groups, so the plan decoder is trained to reproduce the exact token-occurrence pattern that Coverage rewards. This does not make the numerical results logically forced (baselines are not trained on extracted plans, and generation can still fail), but it means the advertised coverage advantage cannot distinguish genuine content planning from optimizing the evaluation protocol. The paper's assertion that the heuristically extracted plans are sufficient to train a good plan decoder is load-bearing and unsupported; the manual pairwise evaluation supports grammaticality and coherence but does not independently validate coverage. Diversity metrics (distinct-4, repetition-4) and the self-BLEU planning-diversity analysis are independent of the string-match overlap.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The model rests on standard deep-learning and VAE assumptions plus a domain-specific planning decomposition. The most fragile premises are the string-match supervision of plans and the 0.5 thresholds; these are choices, not theorems.

free parameters (3)
  • group/stop threshold = 0.5
    Group membership (Eq. 4) and stop decision (Eq. 5) use a fixed 0.5 threshold. No sensitivity analysis is reported; this binarization directly determines plan length and content.
  • latent dimension = 200
    Global and local latent variables have 200 dimensions (Section 5.3). Chosen without reported grid search.
  • special tag = special input item
    A sentinel input item is added to represent sentences that mention no input items (Section 5.3). This is a manual treatment that affects planning supervision.
assumptions (4)
  • domain assumption Long texts can be decomposed into a sequence of sentence-level content groups, each a subset of input items, and this decomposition captures inter-sentence structure (Section 4.1).
    The entire method assumes the planning-then-realization decomposition is a faithful model of how the target texts are organized.
  • domain assumption Reference plans can be approximated by string-matching attribute values/ingredients in reference sentences (Section 5.3).
    The plan decoder is supervised with these approximations; if the heuristic is inaccurate, the training signal is corrupted.
  • domain assumption Latent variables follow isotropic Gaussian distributions and are sampled via reparameterization (Section 4.3, 4.4).
    A standard VAE assumption that is not justified for this domain but is common practice.
  • domain assumption The generative model factorizes as P(y|x) = Σ_g P(g|x) ∏_t P(g_t | g<t, x, z_p) P(s_t | s<t, g, z_p, z_s_t), with dependencies as in Eq. 16-17.
    This factorization is the model definition; it assumes conditional independence of group membership decisions given the decoder state and that sentence generation is Markov in the previous sentence, group, and latents.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Long and Diverse Text Generation with Planning-based Hierarchical Variational Model." pith.science (2026). https://pith.science/paper/LQ4FRG7Y

@misc{pith2026190806605,
  author       = {Pith},
  title        = {Pith review of: Long and Diverse Text Generation with Planning-based Hierarchical Variational Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LQ4FRG7Y}},
  note         = {Machine review of arXiv:1908.06605}
}
read the original abstract

Existing neural methods for data-to-text generation are still struggling to produce long and diverse texts: they are insufficient to model input data dynamically during generation, to capture inter-sentence coherence, or to generate diversified expressions. To address these issues, we propose a Planning-based Hierarchical Variational Model (PHVM). Our model first plans a sequence of groups (each group is a subset of input items to be covered by a sentence) and then realizes each sentence conditioned on the planning result and the previously generated context, thereby decomposing long text generation into dependent sentence generation sub-tasks. To capture expression diversity, we devise a hierarchical latent structure where a global planning latent variable models the diversity of reasonable planning and a sequence of local latent variables controls sentence realization. Experiments show that our model outperforms state-of-the-art baselines in long and diverse text generation.

Figures

Figures reproduced from arXiv: 1908.06605 by the authors.

Figure 1
Figure 1. Generation process of PHVM. After encod [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Architecture of PHVM. The model controls planning with a global latent variable [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Average number of distinct planning results [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Samples of attributes and attribute values for [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Generated advertising texts from different models. Attribute values are colored in red. Repeated expres [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Generated recipes from different models. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TransSent: Towards Generation of Structured Sentences with Discourse Marker

    cs.CL 2019-09 conditional novelty 6.0 of 10

    TransSent generates a tail discourse from a head discourse and a discourse marker by treating the marker as a translation in embedding space, with new datasets and improved scores over baselines.

  2. Improving Linguistic Diversity of Large Language Models with Possibility Exploration Fine-Tuning

    cs.CL 2024-12 conditional novelty 5.0 of 10

    Possibility Exploration Fine-Tuning (PEFT) conditions LLMs on a random possibility number and trains with unlikelihood to generate diverse, controllable responses without added latency, as shown on dialogue and story tasks.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages · cited by 2 Pith papers

  1. [10]

    <裙领型, 圆领> <Collar, Round>

    work page

  2. [11]

    combine all of the ingredients

    Learning discourse-level diversity for neural dialog models using conditional variational autoen- coders. In Proceedings of the 55th Annual Meet- ing of the Association for Computational Linguis- tics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 654–664. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang...

    work page 2017

  3. [13]

    七分袖设计，修饰手臂曲线，更显纤细修长。 The three-quarter sleeves flatter the curves of your arms, making your arms look slender

    <裙款式, 拼接> <Element, Stitching> Methods Missing Pairs texts Checklist 11 这款黑色蕾丝长裙，简约的圆领设计，修饰颈部曲线，性感迷人。 This dress with black lace, the minimalist round collar flatters the curve of your neck which is sexy and attractive. 七分袖设计，修饰手臂曲线，更显纤细修长。 The three-quarter sleeves flatter the curves of your arms, making your arms look slender. A字版型，遮肉显瘦，不挑身材，适合各种身材。 The...

    work page

  4. [318]

    Yookoon Park, Jaemin Cho, and Gunhee Kim

    ACL. Yookoon Park, Jaemin Cho, and Gunhee Kim. 2018. A hierarchical latent structure for variational conversa- tion modeling. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies, NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long ...

    work page 2018

  5. [1514]

    Karen Kukich

    ACL. Karen Kukich. 1983. Design of a knowledge-based report generator. In 21st Annual Meeting of the Association for Computational Linguistics, Mas- sachusetts Institute of Technology, Cambridge, Mas- sachusetts, USA, June 15-17, 1983., pages 145–150. R´emi Lebret, David Grangier, and Michael Auli. 2016. Neural text generation from structured data with ap...

    work page 1983

  6. [2004]

    In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, 21-26 July, 2004, Barcelona, Spain., pages 79–86

    Trainable sentence planning for complex in- formation presentations in spoken dialog systems. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, 21-26 July, 2004, Barcelona, Spain., pages 79–86. Jian Su, Xavier Carreras, and Kevin Duh, editors. 2016. Proceedings of the 2016 Conference on Empirical Methods in Natura...

    work page 2004

  7. [2005]

    Pablo A Duboue and Kathleen R McKeown

    Real versus template-based natural language generation: A false opposition? Computational Lin- guistics, 31(1):15–24. Pablo A Duboue and Kathleen R McKeown. 2003. Sta- tistical acquisition of content selection rules for nat- ural language generation. In Proceedings of the 2003 conference on Empirical methods in natural language processing , pages 121–128....

    work page 2003

  8. [2008]

    In ACL 2008, Proceedings of the 46th Annual Meeting of the Association for Compu- tational Linguistics, June 15-20, 2008, Columbus, Ohio, USA, pages 183–191

    Hypertagging: Supertagging for surface real- ization with CCG. In ACL 2008, Proceedings of the 46th Annual Meeting of the Association for Compu- tational Linguistics, June 15-20, 2008, Columbus, Ohio, USA, pages 183–191. Xiaocheng Feng, Ming Liu, Jiahao Liu, Bing Qin, Yibo Sun, and Ting Liu. 2018. Topic-to-essay generation with neural networks. In Proceed...

    work page 2008

Show all 13 references
  1. [2011]

    In ACL, pages 142–150

    Learning word vectors for sentiment analysis. In ACL, pages 142–150. The Association for Com- puter Linguistics. Julian J. McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-based rec- ommendations on styles and substitutes. In SIGIR, pages 43–52....

    2015

  2. [2015]

    Pointer networks. In Advances in Neural Information Processing Systems 28: Annual Con- ference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 2692–2700. Sam Wiseman, Stuart M. Shieber, and Alexander M. Rush. 2017. Challenges...

    2015

  3. [2016]

    In (Su et al., 2016), pages 329– 339

    Globally coherent text generation with neural checklist models. In (Su et al., 2016), pages 329– 339. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations. Philipp Koehn. 2004. Statistical si...

    2016

  4. [2017]

    In Proceedings of the 18th An- nual SIGdial Meeting on Discourse and Dialogue, Saarbr¨ucken, Germany, August 15-17, 2017 , pages 201–206

    The E2E dataset: New challenges for end- to-end generation. In Proceedings of the 18th An- nual SIGdial Meeting on Discourse and Dialogue, Saarbr¨ucken, Germany, August 15-17, 2017 , pages 201–206. Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors. 2017. Proceedings of...

    2017

  5. [3301]

    Lei Sha, Lili Mou, Tianyu Liu, Pascal Poupart, Sujian Li, Baobao Chang, and Zhifang Sui

    AAAI Press. Lei Sha, Lili Mou, Tianyu Liu, Pascal Poupart, Sujian Li, Baobao Chang, and Zhifang Sui. 2018. Order- planning neural text generation from structured data. In (McIlraith and Weinberger, 2018), pages 5414– 5421. Yuanlong Shao, Stephan Gouws, Denny Britz, Anna Goldie...

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.