Pith. sign in

REVIEW 5 major objections 4 minor 2 cited by

M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models

T0 review · 5 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that averaging the token-level probability distributions of several paraphrased prompts in a single batched decode improves generation quality over any one prompt, with gains of roughly 0.5–2 points on translation, code…

desk verdict M-Ped renames standard ensemble decoding and the headline gains don't survive the paper's own per-seed data; the only robust effect is on text simplification. read the letter →

arxiv 2412.18299 v1 pith:WS65ZRVY submitted 2024-12-24 cs.CL cs.AI

Jiaxin Guo , Daimeng Wei , Yuanchang Luo , Shimin Tao , Hengchao Shang , Zongyao Li , Shaojun Li , Jinlong Yang
show 3 more authors
Zhanglin Wu Zhiqiang Rao Hao Yang
This is my paper · ORCID
classification cs.CLcs.AI
keywords multi-promptensembledecodinginner-batchleft-paddinglargelanguagemodelsdocument-levelmachinetranslationcodegenerationtextsimplificationpass@k
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-prompt ensemble decoding (M-Ped) proposes that instead of running one prompt through an LLM, you can batch several paraphrases of the same request, average the per-token probability distributions across the batch, and decode from that averaged distribution. The paper reports that this Inner-Batch Ensemble consistently beats the best individual prompt on document-level machine translation (d-BLEU), code generation (pass@k), and text simplification (LENS), with improvements of roughly 0.5 to 2 points depending on task and direction. Because all n prompts share a single batched forward pass, the method costs essentially the same inference time as a single prompt, with only extra memory for the additional batch slots and padding tokens. If the pattern holds, it offers a nearly free way to raise LLM output quality without retraining or deploying multiple models.

What carries the argument

The load-bearing object is the Inner-Batch Ensemble formula (Equation 3), together with the batched decoding procedure in Algorithm 1 that implements it. At each decoding step, the n prompts' logit vectors are partitioned by batch position, each partition is repeated n times, and the repeated copies are averaged, producing a single combined distribution from which the next token is drawn. A Left-Padding strategy makes the batch feasible by aligning all prompt lengths to the longest input on the left side, so that the source tokens occupy the same relative positions and right-padding does not sever the input-output connection. The uniformity of the averaging means the method introduces no learned weights; the only design choice is the list of paraphrases.

What would settle it

Re-analyze the per-seed results in Tables 15–17 with a paired test comparing the ensemble score against the better-performing single prompt for each language pair and task; if the ensemble does not win on a significant majority of the ten seeds, or the paired difference is not significant, the claimed advantage over the best single prompt is not established.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that the diversity needed for effective ensemble decoding can come from prompts rather than from model parameters. Classical ensemble decoding averages the predictive distributions of n different models on one prompt; M-Ped keeps a single model and averages the distributions produced by n semantically equivalent prompts, defining the ensemble probability of token $y_j$ as $P(y_j | y_{0:j-1}, X) = (1/n) \sum_{i=1}^{n} P(y_j | y_{0:j-1}, X, P_i; \theta)$. The paper presents this as a drop-in replacement: the same batched logits are sliced, repeated, and averaged inside the decoder. Its evidence is empirical and cross-task: d-BLEU gains on IWSLT 2017 in eight language directions, pass@k gains on HumanEval with two CodeLlama sizes, and LENS gains on SimpEval_2022, with ablation studies suggesting that two to three prompts capture most of the benefit.

Load-bearing premise

The n prompts must be semantically equivalent paraphrases of the same instruction; if any prompt diverges in meaning, the averaged token distribution is an average over different tasks and the ensemble has no coherent interpretation.

Editorial extensions

If this is right

  • On IWSLT 2017 document translation, M-Ped raises d-BLEU by about 1.5 points for en→zh and en→ja and by roughly 0.5 points in the other six tested directions relative to the best single prompt.
  • On HumanEval, M-Ped improves pass@1, pass@5, and pass@10 by about 1 percentage point with CodeLlama-7B-Python, and pass@10 by nearly 2 points with CodeLlama-13B-Python.
  • On SimpEval_2022, M-Ped improves LENS by about 1.5 points with reference and 0.5 points without reference over the best single prompt.
  • The gains appear with Top-p and Top-k decoding, shrink under beam search, and plateau once the prompt count reaches three, so two to three paraphrases are the recommended operating point.
  • Because all prompts are decoded in one batched forward pass, the wall-clock inference cost is comparable to a single-prompt decode, making the ensemble nearly compute-neutral.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Since the method's gain presumably comes from reducing the variance of the next-token distribution, the same batched-averaging trick should transfer to other cheap sources of input diversity, such as mixing system-prompt languages, varying temperature, or shuffling few-shot example order; the paper only varies instruction phrasing.
  • The reported per-seed tables show wide overlapping ranges between single-prompt and ensemble scores, so part of the headline improvement may reflect seed noise rather than a systematic ensemble effect; a paired significance test across the ten seeds would settle this.
  • The code-generation p2 prompt appends a triple-quoted string 'This is a good code.' inside the function body, which is syntactically part of the function definition and may change the problem's meaning; those results are therefore not a clean test of averaging equivalent paraphrases.
  • Left-padding as a mechanism is independent of the ensemble idea and could be reused in other batched decoding setups; the paper leaves open how it interacts with position-dependent attention biases such as RoPE or ALiBi.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes M-Ped, a multi-prompt ensemble decoding method for LLMs. For a single input X, the method constructs n semantically equivalent prompt variants, runs them as one batched inference pass through the same model, and at each decoding step averages the n token-level probability distributions before sampling or selecting the next token. To make the batch feasible, the paper recommends left padding so that all prompts in the batch share the same input length. The method is evaluated on document-level machine translation with Llama-3.1-8B-Instruct (eight IWSLT 2017 directions), code generation with CodeLlama-7B-Python (HumanEval pass@k), and text simplification with Llama-3.1-8B-Instruct (SimpEval_2022 LENS). The paper reports consistent improvements over either single prompt in all three tasks, plus additional experiments on decoding strategies, model sizes, prompt count n, multilingual prompts, and MBR combination.

Significance. If the claimed gains are real, this is an attractively simple and low-cost way to improve generation quality: it requires no extra models or training, only a set of paraphrase prompts and a batched forward pass. The paper is commendable for reporting per-seed results in Appendix B and for addressing the practical padding issue. However, the central claim is empirical, and the current evidence does not establish it: the per-seed variability is large, no significance testing or multiple-comparison correction is reported, the prompt count n=2 is selected on the same test sets used for the headline numbers, and the code-generation prompt p2 shown in Appendix A.2 is syntactically malformed. The core contribution is plausible and potentially useful, but the manuscript needs substantial revision before the claimed improvements can be accepted.

major comments (5)
  1. [Appendix B, Table 15; Section 3.1] The seed-level results do not support the claim that M-Ped improves over every single prompt. For example, en→ja p1 values range from 4.69 to 14.02 d-BLEU, and the per-seed differences between Ours and the better of p1/p2 are negative on several seeds; a paired test on the reported 10 seeds gives an approximately non-significant result. Similar patterns appear in the code-generation data in Table 16. Since the paper makes point comparisons ('approximately increase by 1.5 points' in Section 3.1) without confidence intervals or significance tests, the headline improvements are not established.
  2. [2.1, Algorithm 1] Algorithm 1 does not implement the ensemble average described by Eq. (3). As written, the algorithm splits pre_logits into n parts, repeats each part n times along the batch dimension, and divides by n; for n=2 this returns the original rows unchanged rather than averaging the n prompt rows. The correct operation would be to average over the prompt dimension, e.g., pre_logits.view(n, part_size, vocab_size).mean(dim=0). This discrepancy makes the reported implementation non-reproducible and must be fixed.
  3. [4.3, Tables 7 and 8] The choice n=2 is tuned on the same test sets that produce the headline results in Tables 2 and 3. Section 4.3 justifies the standard configuration by comparing n=2, 3, 4 on HumanEval and SimpEval_2022, then uses this configuration for the main claims. Because no held-out validation or selection correction is reported, the reported gains are optimistically biased and the '2-3 prompts are sufficient' conclusion is not a validated finding.
  4. [Appendix A.2, Tables 12 and 13] The paper's second code-generation prompt is syntactically malformed. Inserting the triple-quoted string '""" This is a good code.' before a function that already contains triple-quoted docstrings creates nested triple quotes and a syntax error, and it does not preserve the semantic content of p1 as claimed in Section 3.2. The code-generation results in Tables 2 and 6 therefore rest on a defective prompt construction, and the claim that p2 'does not alter the meaning of the original prompt' is contradicted by the appendix.
  5. [Table 1 vs. Table 15] There is a numerical inconsistency in the main machine-translation result: Table 1 reports Ours en→zh d-BLEU as 18.88, but the corresponding average in Table 15 is 18.38. Since Tables 1 and 15 are supposed to report the same experiment, this discrepancy undermines confidence in the reported values and must be reconciled.
minor comments (4)
  1. [Eqs. (2) and (3)] The sums are written as i=0 to n, which gives n+1 terms; they should be i=1 to n (or the indexing should be otherwise corrected).
  2. [Section 3.2] The model description says 'CodeLlama-7B-Python-hf' but then immediately refers to 'CodeLlama-13B-Python-hf is a member of the Code Llama series'; this appears to be a typo and should be corrected.
  3. [Table 5] The asterisk on Ours under Top-k (67.85%) has no corresponding footnote explaining what it marks.
  4. [Appendix B tables] The rows labeled 'A VG' should be 'AVG' or 'Average' for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: M-Ped is an empirical decoding recipe; the averaged-distribution definition does not entail the reported gains.

full rationale

The paper's central claim is empirical: averaging token-level probability distributions over n prompt variants improves generation quality. Equation (3) defines the multi-prompt ensemble average by replacing the model index in Equation (2) with a prompt index, but this definition does not by itself entail any improvement; the claimed gains come from the benchmark comparisons in Tables 1, 2, and 3. No fitted parameter is renamed as a prediction: the method has no learned or fitted constants, and the choice n=2 is a configuration selected from Section 4.3 on the same test sets, which is a test-set selection and statistical-significance concern, not a circularity reduction. There is no load-bearing self-citation chain: the only author self-citation (Wei et al., 2023) appears as background in the introduction and does not justify the method. The code-generation p2 prompt in Appendix A.2 is inconsistent with the stated semantic-equivalence assumption and is a prompt-validity concern, but it does not make the ensemble result equal to its input by construction. The paper is self-contained against external benchmarks (IWSLT 2017, HumanEval, SimpEval_2022), so no significant circularity is present.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no invented entities or fitted constants. Its load-bearing choices are the number of prompts (n=2, tuned on the test data), the uniform averaging weight, the assumption that paraphrased prompts are semantically equivalent, and the assumption that left-padding is harmless. All are hand-set or assumed rather than derived.

free parameters (2)
  • number of prompts (n) = 2
    Chosen after Section 4.3 showed n=2 scored best on the same test sets used for the headline results.
  • uniform average weight (1/n) = 1/n
    Chosen as the most straightforward option (Section 2.1); no weighted-average variants are tested.
assumptions (4)
  • domain assumption LLMs expose token-level probability distributions to the caller.
    Required for Inner-Batch Ensemble; holds for the open-weight models used (Llama, CodeLlama, Qwen) but not for proprietary APIs without logit access.
  • domain assumption The n prompt variants are semantically equivalent paraphrases of the intended instruction.
    Eq. (3) assumes prompts P1..Pn play the role of models in Eq. (2); the malformed code p2 prompt in Appendix A.2 violates this.
  • ad hoc to paper Averaging token probability distributions reduces prompt-specific bias and improves output quality.
    This is the core hypothesis of the paper; it is not derived and is only tested empirically without significance testing.
  • domain assumption Left-padding with a pad token does not affect next-token predictions for non-padding positions.
    Section 2.2 asserts this, but no attention-mask or embedding-level verification is provided; if the pad token has semantic content, the assumption may fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models." pith.science (2026). https://pith.science/paper/WS65ZRVY

@misc{pith2026241218299,
  author       = {Pith},
  title        = {Pith review of: M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WS65ZRVY}},
  note         = {Machine review of arXiv:2412.18299}
}
abstract

With the widespread application of Large Language Models (LLMs) in the field of Natural Language Processing (NLP), enhancing their performance has become a research hotspot. This paper presents a novel multi-prompt ensemble decoding approach designed to bolster the generation quality of LLMs by leveraging the aggregation of outcomes from multiple prompts. Given a unique input $X$, we submit $n$ variations of prompts with $X$ to LLMs in batch mode to decode and derive probability distributions. For each token prediction, we calculate the ensemble probability by averaging the $n$ probability distributions within the batch, utilizing this aggregated probability to generate the token. This technique is dubbed Inner-Batch Ensemble. To facilitate efficient batch inference, we implement a Left-Padding strategy to maintain uniform input lengths across the n prompts. Through extensive experimentation on diverse NLP tasks, including machine translation, code generation, and text simplification, we demonstrate the efficacy of our method in enhancing LLM performance. The results show substantial improvements in BLEU scores, pass@$k$ rates, and LENS metrics over conventional methods.

Figures

Figures reproduced from arXiv: 2412.18299 by the authors.

Figure 1
Figure 1. (a) Standard Ensemble: Input X with prompt P is fed into models M1, M2, ..., Mn, yielding predictions Y1, Y2, ..., Yn. These are combined to get the final ensemble result Y . (b) Our Multi-Prompt Ensemble: Input X constructs prompts P1, P2, ..., Pn, forming diverse samples batched through one model M to get predictions Y1, Y2, ..., Yn. The ensemble result Y is the average of these predictions. However, in the practi… view at source ↗
Figure 2
Figure 2. The overall process of Our Multi-Prompt Ensemble Decoding. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Test-Time Augmentation for LLMs: When Input Diversity Beats Output Diversity at Matched Compute

    cs.LG 2026-08 conditional novelty 6.0 of 10

    For mid-tier LLMs, generating paraphrased versions of an input and majority voting over answers converts inference budget into accuracy more efficiently than self-consistency sampling.

  2. Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Token-wise minimum and base-relative consensus over per-source fine-tunes suppress hidden and explicit poisoning that union training and weight averaging retain.

Reference graph

Works this paper leans on

33 extracted references · 10 canonical work pages · cited by 2 Pith papers

  1. [1]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv doi:10.48550/arxiv.2309.16609 2023

  2. [2]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bava...

    arXiv 2021

  3. [3]

    Zifeng Cheng, Zhaoling Chen, Zhiwei Jiang, Yafeng Yin, Shiping Ge, Yuliang Liu, and Qing Gu. 2024. https://doi.org/10.48550/ARXIV.2406.06279 Multi-prompting decoder helps better language understanding . CoRR, abs/2406.06279

    work page Pith review arXiv doi:10.48550/arxiv.2406.06279 2024

  4. [4]

    Enrique Dehaerne, Bappaditya Dey, Sandip Halder, Stefan De Gendt, and Wannes Meert. 2022. https://doi.org/10.1109/ACCESS.2022.3196347 Code generation using machine learning: A systematic review . IEEE Access , 10:82434--82455

    work page arXiv 2022

  5. [5]

    Matthew Finlayson, John Hewitt, Alexander Koller, Swabha Swayamdipta, and Ashish Sabharwal. 2024. https://openreview.net/forum?id=dONpC9GL1o Closing the curious case of neural text degeneration . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

    work page 2024

  6. [6]

    M. A. Ganaie, Minghui Hu, Ashwani Kumar Malik, Muhammad Tanveer, and Ponnuthurai N. Suganthan. 2022. https://doi.org/10.1016/J.ENGAPPAI.2022.105151 Ensemble deep learning: A review . Eng. Appl. Artif. Intell., 115:105151

    arXiv 2022

  7. [7]

    Jiatao Gu, James Bradbury, Caiming Xiong, Victor O. K. Li, and Richard Socher. 2018. https://openreview.net/forum?id=B1l8BtlCb Non-autoregressive neural machine translation . In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net

    work page 2018

  8. [8]

    David Heineman, Yao Dou, and Wei Xu. 2024. https://aclanthology.org/2024.emnlp-main.1255 Improving minimum bayes risk decoding with multi-prompt . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024 , pages 22525--22545. Association for Computational Linguistics

    work page 2024

Show all 33 references
  1. [9]

    Chris Hokamp, Demian Gholipour Ghalandari, Nghia The Pham, and John Glover. 2020. https://arxiv.org/abs/2006.08748 Dyne: Dynamic ensemble decoding for multi-document summarization . CoRR, abs/2006.08748

    2020 arXiv

  2. [10]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. https://doi.org/10.48550/ARXIV.2406.00515 A survey on large language models for code generation . CoRR, abs/2406.00515

    2024 doi arXiv

  3. [11]

    Xu, Jun Araki, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. https://doi.org/10.1162/TACL\_A\_00324 How can we know what language models know . Trans. Assoc. Comput. Linguistics, 8:423--438

    2020 doi

  4. [12]

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. https://proceedings.neurips.cc/paper/2017/hash/9ef2ed4b7fd2c810847ffa5fa85bce38-Abstract.html Simple and scalable predictive uncertainty estimation using deep ensembles . In Advances in Neural Information ...

    2017

  5. [13]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. https://doi.org/10.1145/3560815 Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing . ACM Comput. Surv. , 55(9):195:1--195:35

    2023 doi

  6. [14]

    Mounica Maddela, Yao Dou, David Heineman, and Wei Xu. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.905 LENS: A learnable evaluation metric for text simplification . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa...

    2023 doi

  7. [15]

    Akifumi Nakamachi, Tomoyuki Kajiwara, and Yuki Arase. 2020. https://aclanthology.org/2020.aacl-srw.22/ Text simplification with reinforcement learning using supervised rewards on grammaticality, meaning preservation, and simplicity . In Proceedings of the 1st Conference of the...

    2020

  8. [16]

    OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  9. [17]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, ...

    2002

  10. [18]

    Zhang, Andrew Wang, and Jimmy Ba

    Silviu Pitis, Michael R. Zhang, Andrew Wang, and Jimmy Ba. 2023. https://doi.org/10.48550/ARXIV.2304.05970 Boosted prompt ensembles for large language models . CoRR, abs/2304.05970

    2023 doi arXiv

  11. [19]

    Farinha, and Alon Lavie

    Ricardo Rei, Craig Stewart, Ana C. Farinha, and Alon Lavie. 2020. https://doi.org/10.18653/V1/2020.EMNLP-MAIN.213 COMET: A neural framework for MT evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, Novemb...

    2020 doi

  12. [20]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. https://arxiv.org/abs/1511.06709 Improving neural machine translation models with monolingual data . Preprint, arXiv:1511.06709

    2016 arXiv

  13. [21]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie - Anne Lachaux, Timoth \' e e Lacroix, Baptiste Rozi \` e re, Naman Goyal, Eric Hambro, Faisal Azhar, Aur \' e lien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023 a . https://doi.org/10....

    2023 doi arXiv

  14. [22]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

    2023 doi arXiv

  15. [23]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Attention is all you need . In Advances in Neural I...

    2017

  16. [24]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/forum?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Conferen...

    2023

  17. [25]

    Daimeng Wei, Zhanglin Wu, Hengchao Shang, Zongyao Li, Minghan Wang, Jiaxin Guo, Xiaoyu Chen, Zhengzhe Yu, and Hao Yang. 2023. https://arxiv.org/abs/2306.01318 Text style transfer back-translation . Preprint, arXiv:2306.01318

    2023 arXiv

  18. [26]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoning...

    2022

  19. [27]

    Yujia Xie, Hanjun Dai, Minshuo Chen, Bo Dai, Tuo Zhao, Hongyuan Zha, Wei Wei, and Tomas Pfister. 2020. https://proceedings.neurips.cc/paper/2020/hash/ec24a54d62ce57ba93a531b460fa8d18-Abstract.html Differentiable top-k with optimal transport . In Advances in Neural Information ...

    2020

  20. [28]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 doi arXiv

  21. [29]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with BERT . In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April...

    2020

  22. [30]

    Jiangjiang Zhao, Zhuoran Wang, and Fangchun Yang. 2023. https://doi.org/10.24963/IJCAI.2023/588 Genetic prompt search via exploiting language model probabilities . In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-2...

    2023 doi

  23. [31]

    Zhi - Hua Zhou, Jianxin Wu, and Wei Tang. 2002. https://doi.org/10.1016/S0004-3702(02)00190-X Ensembling neural networks: Many could be better than all . Artif. Intell., 137(1-2):239--263

    2002 doi

  24. [32]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  25. [33]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.