Pith. sign in

REVIEW 4 major objections 5 minor 44 references

Pipelined Decoder for Efficient Context-Aware Text Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes a pipelined decoder that generates multiple text subsequences in parallel, yielding up to 7x faster inference on question answering, summarization, and keyphrase generation while preserving generation quality.

desk verdict The core idea is a reasonable extension of the authors' branching decoder, but the abstract overclaims quality preservation: the paper's own Table 3 shows a 5.9-point drop in present keyphrase F1@5 on KP20K, and the lack of significance testing and a missing branching-decoder comparison are real gaps. read the letter →

arxiv 2506.23431 v2 pith:WZAFO5WV submitted 2025-06-29 cs.CL cs.AI

Zixian Huang , Chenxu Niu , Yu Gu , Gengyang Xiao , Xinwei Huang , Gong Cheng This is my paper · ORCID
classification cs.CLcs.AI
keywords pipelineddecoderparalleldecodingcontext-awaretextgenerationinferenceaccelerationautoregressivesummarizationquestionansweringkeyphrase
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Autoregressive text generation is inherently serial: each new token depends on all previous tokens, which creates a speed bottleneck. This paper argues that such strict dependence is unnecessary for context-aware generation, and proposes a pipelined decoder that opens multiple subsequences one after another and then advances them together, generating one new token per open subsequence at each time step. The result is that a T5 model with the pipelined decoder produces answers, summaries, and keyphrases 1.7 to 7.0 times faster than the sequential decoder while staying within a few points in F1/ROUGE and using no more GPU memory. The authors claim this shows that context-aware text generation can be parallelized at the subsequence level without sacrificing quality.

What carries the argument

The central object is the pipelined decoder, a transformer decoder identical in parameters and architecture to a sequential decoder but with a different dependency pattern: a token in subsequence $i$ at time step $t$ attends to the encoder output and to the tokens of the first $i$ subsequences generated up to time $t$ ($G^{\le t}_{\le i}$), rather than to the entire preceding sequence. This partial, subsequence-level conditioning lets $k = \lceil t/\Delta t \rceil$ tokens be generated in parallel at each step. The delay time $\Delta t$ controls how often a new subsequence is opened, trading parallelism against how much previous context each new subsequence receives; the paper sets $\Delta t = 1$ for maximum acceleration.

What would settle it

Run the pipelined decoder versus the sequential decoder on a task with strong long-range dependencies that the paper itself flags as unsuitable, such as math word problems, and compare answer accuracy; a large accuracy drop would refute the claim that initial-token dependencies suffice. A more targeted test is to fix the delay time and measure output quality as the average subsequence length grows: if ROUGE or F1 degrades sharply with subsequence length, the truncated-dependency assumption is the bottleneck.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that the hidden states of a subsequence's early tokens encode enough about the subsequence to support generating later subsequences without full autoregressive context. To exploit this, the pipelined decoder trains a standard transformer decoder with a modified attention pattern: at time step $t$ there are $k = \lceil t/\Delta t \rceil$ open subsequences, and the model produces $k$ tokens in parallel, each conditioned on the encoded input and on the tokens of the first $i$ subsequences produced so far, which may be incomplete. Subsequences are delimited by $\langle bos \rangle$ and $\langle eos \rangle$ tokens, and an extra empty subsequence teaches the model to stop opening new subsequences. On six datasets (MSQA, KP20K, KPTimes, WikiHowQA, CNN/DMail, PubMed) and two T5 sizes, the pipelined decoder improves throughput by factors of 1.7x to 7.0x compared to the sequential decoder, with the largest ROUGE/F1 gap about 2.1 points and no increase in GPU memory.

Load-bearing premise

The whole approach rests on the assumption that the hidden states of the first few tokens of a subsequence encode enough about the rest of that subsequence to let later subsequences be generated correctly without seeing the full text; if this truncated-dependency assumption fails for longer or more interdependent text, generation quality will degrade.

Editorial extensions

If this is right

  • If the pipelined decoder is right, context-aware generation with multi-phrase or multi-sentence outputs can be made 2-7x faster on standard T5 backbones with only minor quality changes and no extra memory.
  • Speedup scales with the number of target subsequences, so tasks that produce many short phrases, such as keyphrase generation and multi-span question answering, benefit most from parallel decoding.
  • Because it uses partial token dependencies, the pipelined decoder is orthogonal to speculative decoding and other inference accelerators, so stacking them is a plausible route to further gains.
  • The technique transfers to any text generation task where output can be split into subsequences by punctuation or explicit separators, which covers most structured context-aware outputs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We suspect the truncated-dependency principle will extend to decoder-only LLMs and open-ended generation, where subsequences could be split by punctuation; the paper only tests T5 encoder-decoder, so this is an extrapolation.
  • The delay time sweep suggests a quality–parallelism continuum, and an adaptive scheduler that opens a new subsequence only when the current one is sufficiently predictable might recover the missing quality at high speed, which the authors list as future work.
  • The consistent improvement in absent keyphrase F1 on KP20K hints that parallel subsequence generation may act as a light regularizer or diversity booster; if true, it would make pipelined decoding attractive beyond speed alone.
  • In practice, the need for known subsequence boundaries in training data is a real constraint; applying the decoder to plain text would require learning a boundary predictor, which the paper does not address.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a pipelined decoder for context-aware text generation. The target text is split into subsequences (phrases or sentences), and the decoder starts a new subsequence every Δt time-steps, generating one token per active subsequence at each step. This relaxes full autoregressive dependence: later subsequences condition only on the initial tokens of earlier subsequences. The authors modify the T5 decoder with this dependency and train it on six datasets covering question answering, keyphrase generation, and summarization, with two model sizes. They report 1.7x–7x throughput improvement, lower GPU memory usage, and generation quality described as comparable to that of a sequential decoder. They also analyze speedup as a function of the number of target subsequences and study the effect of the delay time.

Significance. The idea is timely and the experimental scope is broad: six datasets, two model sizes, a clean training objective, and publicly released code. If the quality claim held, this would be a practical parallel-decoding method complementary to speculative decoding, and the paper would make a useful contribution to efficient generation. The main strengths are the multi-dataset evaluation and the simple, reproducible training setup. However, the headline claim of 'no significant quality loss' is not supported by the paper's own Table 3 on present-keyphrase F1@5, and a likely data error in Table 4 casts doubt on the sentence-level results. The core assumption—that initial tokens encode enough information for later subsequences—must be validated more carefully, and the current Limitations section does not acknowledge the observed quality degradation on a context-aware task. The central claim therefore needs revision and additional statistical support.

major comments (4)
  1. [Abstract; §5.1; Table 3] The central claim that the pipelined decoder achieves 'comparable' quality or 'without a significant loss' is contradicted by the present-keyphrase results on KP20K: Table 3 shows F1@5 falling from 33.7 to 27.8 with T5-Base (5.9 absolute points, about 18% relative) and from 33.0 to 28.6 with T5-Large (4.4 absolute points). Since the paper reports no confidence intervals or significance tests, this drop cannot be dismissed as noise, and the sentence in §5.1 that the largest decline is only 0.8% refers only to MSQA EM and ignores this headline metric. The abstract and §5.1 should be revised to state explicitly that quality is maintained on most metrics but not on present-keyphrase prediction, and the experimental section should report variability (seeds, standard deviations, or paired tests) for all headline quality metrics.
  2. [Table 4 (CNN/DMail)] In Table 4, the Sequential T5-Large row for CNN/DMail reports exactly the same Rouge scores as the Sequential T5-Base row (39.6/17.8/37.5 on dev, 39.2/17.4/37.1 on test) while only the throughput and memory differ. This is almost certainly a copy error, and it makes the sentence-level comparison unreliable as printed. The authors should correct the table and, if the T5-Large sequential numbers were not actually obtained, rerun that configuration or remove it from the comparison.
  3. [§3.2 and Eq. (4)] The core truncated-dependency assumption—that a later subsequence can be generated from only the first few tokens of earlier subsequences—is not directly validated. With Δt=1, the first token of the second subsequence conditions on a single token from the first subsequence plus the bos tokens, as shown in Figure 2. The KP20K present-keyphrase degradation in Table 3 is consistent with this assumption failing for short, document-dependent phrases. The paper should include an analysis of quality by subsequence position or a study with larger Δt on the phrase-level datasets to delineate when the assumption holds; the current Limitations paragraph does not mention this observed failure mode.
  4. [§4.3; §5.1; §5.2] All quality numbers are reported from a single training run, and no standard deviations, confidence intervals, or significance tests are provided. The phrase 'without a significant loss' in the abstract is a statistical claim, and the reported differences across Tables 2–4 are small enough that seed variance could change the conclusions; for example, the 0.1–0.3 point differences in Figure 5 are likely within noise. At minimum, the authors should report the number of seeds and the variance for the main quality metrics, or use a paired test, to support the 'no significant loss' wording.
minor comments (5)
  1. [Section 6] The conclusion refers to 'keyword generation'; this should be 'keyphrase generation' to match the task and the rest of the paper.
  2. [Section 4.1] The dataset is called CNN/DMail in Table 1 and Table 4 but 'CNN-DM' in the text of §4.1; please use one consistent name.
  3. [Figure 5] The figure caption should identify which curve is the sequential decoder and which is the pipelined decoder, since the text refers to a comparison but the figure has no visible legend.
  4. [Algorithm 1] The pseudocode would benefit from a note that a subsequence whose first generated token is eos represents an empty subsequence used to signal termination, and that line 12 counts eos across all subsequences including the newly added one.
  5. [§4.3] In the relative position formula |t1−t2|+|i1−i2|, please state explicitly that i indexes the subsequence and t indexes the time-step, to avoid confusion with the subsequence index in Eq. (4).

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reduction: the speed and quality claims are measured experimental results, not consequences derived from self-citations or fitted inputs.

full rationale

The paper's central claims are empirical. The pipelined decoder is a proposed architecture, and the reported speedups and quality scores come from training and evaluation on standard datasets, not from a derivation that already contains the target metrics as assumptions. The only hand-set quantities are decoding hyperparameters (delay time, timemax, submax), which are transparent experimental choices rather than fitted parameters renamed as predictions. The truncated-dependency assumption is motivated by external prior work (Cai et al. 2024; Gloeckle et al. 2024), not by a self-citation chain. The self-citations in the paper, such as the branching decoder (Huang et al. 2024a), appear only in related work and are not load-bearing for the speed or quality conclusions. The observed drop in keyphrase F1 on KP20K in Table 3 is a substantive empirical concern about the abstract's 'without a significant loss of generation quality' claim, but that is a correctness or robustness issue, not a circularity issue. No equation or fitted value reduces to the claimed result by construction.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the empirical success of a training objective that approximates full autoregressive dependency with a truncated dependency across subsequences. No external constants are derived; the only hand-set numbers are decoding hyperparameters.

free parameters (3)
  • delay time Δt = 1 (tested 1, 2, 3)
    Controls how often a new subsequence is started; set to 1 for maximum acceleration, not fitted to quality metrics.
  • timemax = 50
    Upper bound on decoding time-steps to avoid infinite loops.
  • submax = 20
    Upper bound on number of subsequences to avoid excessive generation in extreme cases.
assumptions (3)
  • domain assumption The transformer decoder (T5) with relative position biases can model the truncated inter-subsequence dependencies.
    Section 3.2 assumes the standard T5 decoder stack, only the dependency mask and relative position distance are changed.
  • domain assumption Hidden states of an early token encode enough information about current and future tokens that a later subsequence needs only a prefix of earlier subsequences.
    Section 1 Motivation explicitly cites Cai et al. (2024) and Gloeckle et al. (2024) for this premise.
  • ad hoc to paper Target text can be split into subsequences for which the ordering dependency is predominantly from earlier subsequences to later ones.
    The training target in Eq. (5) assumes each subsequence is preceded by a bos and followed by an eos, and that the empty subsequence YF teaches stopping; the paper's own Limitations section says this may not hold for context-free tasks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pipelined Decoder for Efficient Context-Aware Text Generation." pith.science (2026). https://pith.science/paper/WZAFO5WV

@misc{pith2026250623431,
  author       = {Pith},
  title        = {Pith review of: Pipelined Decoder for Efficient Context-Aware Text Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WZAFO5WV}},
  note         = {Machine review of arXiv:2506.23431}
}
read the original abstract

As the basis of generative AI, an autoregressive model requires the generation of a new token depending on all the previously generated tokens, which brings high quality but also restricts the model to generate tokens one by one, forming a bottleneck limiting the generation speed. In this paper, we propose a new decoder architecture that efficiently generates text in parallel for context-aware generation tasks. Our proposed pipelined decoder initiates the generation of multiple subsequences simultaneously, and, at each time-step, it generates a new token for each subsequence to realize parallelism. Experiments on multiple text generation tasks, including question answering, text summarization, and keyphrase generation, show that our pipelined decoder significantly improves the generation speed without a significant loss of generation quality or additional memory consumption.

Figures

Figures reproduced from arXiv: 2506.23431 by the authors.

Figure 1
Figure 1. Comparison between the traditional sequential decoder and our pipelined decoder on an example of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An example of the pipelined decoder for gen [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison between the sequential decoder and the pipelined decoder in attention distribution for [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Speedup ratio of the pipelined decoder to the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparison between the sequential decoder [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 20 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, and Yuxiong He. 2022. https://doi.org/10.1109/SC41404.2022.00051 Deepspeed- inference: Enabling efficient inference of transformer models at unprecedented scale . In SC22: International Conference for High P...

    arXiv 2022

  4. [4]

    Lee, Deming Chen, and Tri Dao

    Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. 2024. https://doi.org/10.48550/ARXIV.2401.10774 Medusa: Simple LLM inference acceleration framework with multiple decoding heads . CoRR, abs/2401.10774

    arXiv doi:10.48550/arxiv.2401.10774 2024

  5. [5]

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean - Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023. https://doi.org/10.48550/ARXIV.2302.01318 Accelerating large language model decoding with speculative sampling . CoRR, abs/2302.01318

    arXiv doi:10.48550/arxiv.2302.01318 2023

  6. [6]

    Wang Chen, Hou Pong Chan, Piji Li, and Irwin King. 2020. https://doi.org/10.18653/v1/2020.acl-main.103 Exclusive hierarchical decoding for deep keyphrase generation . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020 , pages 1095--1105

    work page doi:10.18653/v1/2020.acl-main.103 2020

  7. [7]

    Fu, Stefano Ermon, Atri Rudra, and Christopher R \' e

    Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher R \' e . 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/67d57c32e20fd0a7a302cb81d36e40d5-Abstract-Conference.html Flashattention: Fast and memory-efficient exact attention with io-awareness . In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Info...

    work page 2022

  8. [8]

    Yang Deng, Wai Lam, Yuexiang Xie, Daoyuan Chen, Yaliang Li, Min Yang, and Ying Shen. 2020. https://doi.org/10.1609/AAAI.V34I05.6266 Joint learning of answer selection and answer summary generation in community question answering . In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artif...

    work page doi:10.1609/aaai.v34i05.6266 2020

Show all 44 references
  1. [9]

    Yuntian Deng and Alexander M. Rush. 2020. https://proceedings.neurips.cc/paper/2020/hash/01a0683665f38d8e5e567b3b15ca98bf-Abstract.html Cascaded text generation with markov transformers . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Info...

    2020

  2. [10]

    Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2021. http://arxiv.org/abs/2103.10360 All NLP tasks are generation tasks: A general pretraining framework . CoRR, abs/2103.10360

    2021 arXiv

  3. [11]

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. https://doi.org/10.48550/ARXIV.2210.17323 GPTQ: accurate post-training quantization for generative pre-trained transformers . CoRR, abs/2210.17323

    2022 doi arXiv

  4. [12]

    Ygor Gallina, Florian Boudin, and B \' e atrice Daille. 2019. https://doi.org/10.18653/v1/W19-8617 Kptimes: A large-scale dataset for keyphrase generation on news documents . In Proceedings of the 12th International Conference on Natural Language Generation, INLG 2019, Tokyo, ...

    2019 doi

  5. [13]

    Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozi \` e re, David Lopez - Paz, and Gabriel Synnaeve. 2024. https://openreview.net/forum?id=pEWAcejiU2 Better & faster large language models via multi-token prediction . In Forty-first International Conference on Machine Learning,...

    2024

  6. [14]

    Jiatao Gu, James Bradbury, Caiming Xiong, Victor O. K. Li, and Richard Socher. 2017. http://arxiv.org/abs/1711.02281 Non-autoregressive neural machine translation . CoRR, abs/1711.02281

    2017 arXiv

  7. [15]

    Yu Gu, Kai Zhang, Yuting Ning, Boyuan Zheng, Boyu Gou, Cheng Chang, Sanjari Srivastava, Yanan Xie, Peng Qi, Huan Sun, and Yu Su. 2024. https://doi.org/10.48550/ARXIV.2411.06559 Is your LLM secretly a world model of the internet? model-based planning for web agents . CoRR, abs/...

    2024 doi arXiv

  8. [16]

    Junliang Guo, Xu Tan, Linli Xu, Tao Qin, Enhong Chen, and Tie - Yan Liu. 2020. https://doi.org/10.1609/AAAI.V34I05.6289 Fine-tuning by curriculum learning for non-autoregressive neural machine translation . In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI ...

    2020 doi

  9. [17]

    Vivek Gupta, Prerna Bharti, Pegah Nokhiz, and Harish Karnick. 2021. https://doi.org/10.18653/V1/2021.ACL-SRW.30 Sumpubmed: Summarization dataset of pubmed scientific articles . In Proceedings of the ACL-IJCNLP 2021 Student Research Workshop, ACL 2021, Online, JUli 5-10, 2021 ,...

    2021 doi

  10. [18]

    Karl Moritz Hermann, Tom \' a s Kocisk \' y , Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. https://proceedings.neurips.cc/paper/2015/hash/afdec7005cc9f14302cd0474fd0f3c96-Abstract.html Teaching machines to read and comprehend . In Ad...

    2015

  11. [19]

    Zixian Huang, Xinwei Huang, Ao Wu, Xiaxia Wang, and Gong Cheng. 2025. https://doi.org/10.1016/J.KNOSYS.2024.112907 Transforming decoder-only models into encoder-only models with improved understanding capabilities . Knowl. Based Syst., 309:112907

    2025

  12. [20]

    Zixian Huang, Ao Wu, Jiaying Zhou, Yu Gu, Yue Zhao, and Gong Cheng. 2022. https://doi.org/10.18653/V1/2022.NAACL-MAIN.239 Clues before answers: Generation-enhanced multiple-choice QA . In Proceedings of the 2022 Conference of the North American Chapter of the Association for C...

    2022 doi

  13. [21]

    Zixian Huang, Gengyang Xiao, Yu Gu, and Gong Cheng. 2024 a . https://openreview.net/forum?id=riNuqYiD66 A branching decoder for set generation . In The Twelfth International Conference on Learning Representations, ICLR

    2024

  14. [22]

    Zixian Huang, Jiaying Zhou, Chenxu Niu, and Gong Cheng. 2023 a . https://doi.org/10.1145/3583780.3615064 Spans, not tokens: A span-centric model for multi-span reading comprehension . In Proceedings of the 32nd ACM International Conference on Information and Knowledge Manageme...

    2023

  15. [23]

    Zixian Huang, Jiaying Zhou, Gengyang Xiao, and Gong Cheng. 2023 b . https://doi.org/10.1007/978-3-031-44696-2\_58 Enhancing in-context learning with answer feedback for multi-span question answering . In Natural Language Processing and Chinese Computing - 12th National CCF Con...

    2023 doi

  16. [24]

    Zixian Huang, Wenhao Zhu, Gong Cheng, Lei Li, and Fei Yuan. 2024 b . http://papers.nips.cc/paper\_files/paper/2024/hash/3bf80b34f731313b8292f4578e820c90-Abstract-Conference.html Mindmerger: Efficiently boosting LLM reasoning in non-english languages . In Advances in Neural Inf...

    2024

  17. [25]

    Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020. https://openreview.net/forum?id=rkgNKkHtvB Reformer: The efficient transformer . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020

    2020

  18. [26]

    Jason Lee, Elman Mansimov, and Kyunghyun Cho. 2018. https://doi.org/10.18653/V1/D18-1149 Deterministic non-autoregressive neural sequence modeling by iterative refinement . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Bel...

    2018 doi

  19. [27]

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. https://proceedings.mlr.press/v202/leviathan23a.html Fast inference from transformers via speculative decoding . In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , volume 2...

    2023

  20. [28]

    Haonan Li, Martin Tomko, Maria Vasardani, and Timothy Baldwin. 2022 a . https://doi.org/10.18653/V1/2022.NAACL-MAIN.90 Multispanqa: A dataset for multi-span question answering . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computat...

    2022 doi

  21. [29]

    Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian - Yun Nie, and Ji - Rong Wen. 2022 b . https://doi.org/10.18653/V1/2022.EMNLP-MAIN.68 ELMER: A non-autoregressive pre-trained language model for efficient and effective text generation . In Proceedings of the 2022 Conference on Empir...

    2022 doi

  22. [30]

    Guangyi Liu, Zichao Yang, Tianhua Tao, Xiaodan Liang, Junwei Bao, Zhen Li, Xiaodong He, Shuguang Cui, and Zhiting Hu. 2022. https://doi.org/10.18653/V1/2022.NAACL-MAIN.150 Don't take it literally: An edit-invariant sequence loss for text generation . In Proceedings of the 2022...

    2022 doi

  23. [31]

    Mingdao Liu, Aohan Zeng, Bowen Wang, Peng Zhang, Jie Tang, and Yuxiao Dong. 2024. https://doi.org/10.48550/ARXIV.2401.06761 APAR: llms can do auto-parallel auto-regressive decoding . CoRR, abs/2401.06761

    2024 doi arXiv

  24. [32]

    Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017. https://doi.org/10.18653/v1/P17-1054 Deep keyphrase generation . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July...

    2017 doi

  25. [33]

    Xuefei Ning, Zinan Lin, Zixuan Zhou, Huazhong Yang, and Yu Wang. 2023. https://doi.org/10.48550/ARXIV.2307.15337 Skeleton-of-thought: Large language models can do parallel decoding . CoRR, abs/2307.15337

    2023 doi arXiv

  26. [34]

    OpenAI. 2023. https://doi.org/10.48550/arXiv.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  27. [35]

    Weizhen Qi, Yeyun Gong, Jian Jiao, Yu Yan, Weizhu Chen, Dayiheng Liu, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, and Nan Duan. 2021. http://proceedings.mlr.press/v139/qi21a.html BANG: bridging autoregressive and non-autoregressive generation with large sc...

    2021

  28. [36]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. http://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . J. Mach. Learn. Res., ...

    2020

  29. [37]

    Chitwan Saharia, William Chan, Saurabh Saxena, and Mohammad Norouzi. 2020. https://doi.org/10.18653/V1/2020.EMNLP-MAIN.83 Non-autoregressive machine translation with latent alignments . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, ...

    2020 doi

  30. [38]

    Noam Shazeer. 2019. http://arxiv.org/abs/1911.02150 Fast transformer decoding: One write-head is all you need . CoRR, abs/1911.02150

    2019 arXiv

  31. [39]

    Xing, and Zhiting Hu

    Bowen Tan, Zichao Yang, Maruan Al - Shedivat, Eric P. Xing, and Zhiting Hu. 2021. https://doi.org/10.18653/V1/2021.NAACL-MAIN.341 Progressive generation of long text with pretrained language models . In Proceedings of the 2021 Conference of the North American Chapter of the As...

    2021 doi

  32. [40]

    Guangxuan Xiao, Ji Lin, Micka \" e l Seznec, Hao Wu, Julien Demouth, and Song Han. 2023 a . https://proceedings.mlr.press/v202/xiao23c.html Smoothquant: Accurate and efficient post-training quantization for large language models . In International Conference on Machine Learnin...

    2023

  33. [41]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2023 b . https://doi.org/10.48550/ARXIV.2309.17453 Efficient streaming language models with attention sinks . CoRR, abs/2309.17453

    2023 doi arXiv

  34. [42]

    Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan

    Lili Yao, Nanyun Peng, Ralph M. Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. https://doi.org/10.1609/AAAI.V33I01.33017378 Plan-and-write: Towards better automatic storytelling . In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty...

    2019 doi

  35. [43]

    Barrett, Zhangyang Wang, and Beidi Chen

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher R \' e , Clark W. Barrett, Zhangyang Wang, and Beidi Chen. 2023. https://doi.org/10.48550/ARXIV.2306.14048 H\( _ 2 \)o: Heavy-hitter oracle for efficient gener...

    2023 doi arXiv

  36. [44]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.