Pith. sign in

REVIEW 4 major objections 4 minor 38 references

OptiSeq: Ordering Examples On-The-Fly for In-Context Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read In in-context learning, the order of examples changes what LLMs output, and OptiSeq selects the best order at inference time by scoring each candidate output without the examples.

desk verdict A novel and plausible ordering heuristic whose headline gains outrun its evidence; worth a careful review. read the letter →

arxiv 2501.15030 v2 pith:BWULN2KF submitted 2025-01-25 cs.LG cs.AIcs.CLcs.PF

Rahul Atul Bhope , Praveen Venkateswaran , K. R. Jayaram , Vatche Isahagian , Vinod Muthusamy , Nalini Venkatasubramanian This is my paper · ORCID
classification cs.LGcs.AIcs.CLcs.PF
keywords in-contextlearningexampleorderingpromptsensitivitylogprobabilitiesinference-timeoptimizationAPIsequencegenerationtextclassificationlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the order of in-context examples is not a minor nuisance: the same examples arranged differently can move accuracy by roughly 12 points on API sequencing and 17 points on classification. It proposes OptiSeq, a purely inference-time, dataset-free method that tries all orderings of the examples, generates each candidate output, then re-scores each output with the examples stripped away, using only the task instruction. The ordering whose output scores highest under that zero-shot log-probability score is the one recommended. Across five datasets and five LLMs from 8B to 70B parameters, OptiSeq improves accuracy by 5.5 to 10.5 percentage points over random order, Top-K order, and two recent ordering baselines. The practical point is that an LLM application can get a more reliable prompt without any training data or offline tuning.

What carries the argument

The load-bearing object is the example-free log-probability score $\Phi_k$. Given an output $o_k$ generated under ordering $k$, the paper discards the examples and recomputes the token log probabilities conditioned only on the task instruction $I$; the ordering with the largest $\Phi_k$ is chosen. This score separates correct from incorrect outputs more cleanly than the contextual log probabilities from the full few-shot prompt, and it needs no labels, validation set, or corpus-level assumptions.

What would settle it

For a test instance where the model's zero-shot answer is wrong but has higher log probability than any correct answer, OptiSeq will pick the ordering whose candidate output is that wrong answer. Collect a set of such confidently-wrong instances and count how often OptiSeq beats random selection; if it does not beat random selection there, the selection rule fails exactly where confidence and correctness diverge.

Watch

Extended reading notes

Core claim

The central discovery is that while naive in-context learning gives similar log probabilities to correct and incorrect outputs, the same outputs become separable when the in-context examples are removed and only the task instruction conditions the score. OptiSeq formalizes this with $\Phi_k = \sum_i \log P(x_{ik} \mid I \oplus x_{j<i,k})$, the sum of per-token log probabilities of the generated output $o_k$ under the instruction alone, and selects $k^* = \arg\max_k \Phi_k$. This distinction is enough, the paper claims, to make the ordering that yields the highest zero-shot score align with the ordering that yields the most accurate answer, across API sequence generation and text classification.

Load-bearing premise

The ranking premise: the zero-shot log-probability sum $\Phi_k$, computed without the examples, is a reliable proxy for whether the output is correct, so the ordering with the highest $\Phi_k$ is the best ordering.

Editorial extensions

If this is right

  • A model serving pipeline can pick a prompt order per query at inference time without a held-out set, as long as the model exposes token-level log probabilities.
  • Optimal ordering is instance-specific and model-specific, so fixed 'best' orders found offline are not enough; OptiSeq adapts per query.
  • Ordering a small set of examples well can beat using more examples in a random or Top-K order, which is useful when context length is limited.
  • EOptiSeq cuts the search from $|E|!$ to $(|E|-1)!$ permutations by anchoring the most similar example first, trading some accuracy for latency.
  • OptiSeq's reported gains apply to both classification and structured API sequence generation, not just single-label tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper fixes the example set and only reorders it; combining OptiSeq with example selection could compound the gains, since the score already provides a per-instance ranking.
  • The zero-shot score could double as a confidence signal for abstention or fallback, though the paper does not test that use.
  • If a model is confidently wrong on the instruction alone, OptiSeq will select that wrong output whenever it appears in any ordering; a miscalibrated model is the natural failure mode to probe.
  • Because the method only needs log probabilities, the same recipe could transfer to vision-language or embedding models that expose token scores, which the paper does not evaluate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces OptiSeq, a purely inference-time algorithm for choosing the order of in-context examples in an LLM prompt. For a given test instance and a fixed set of E examples, OptiSeq generates candidate outputs for all E! permutations, removes the in-context examples from each prompt, recomputes the sum of log probabilities of each generated output conditioned only on the task instruction, and selects the ordering whose output has the highest such score. A cheaper variant, EOptiSeq, anchors the most SBERT-similar example in the first position and searches only the remaining (E-1)! permutations. The main empirical claim is that this selection improves accuracy by 5.5-10.5 percentage points over random ordering, Top-K ordering, LocalE, and Influence-score baselines, across two tasks (API sequence generation and text classification), five datasets, and five LLMs ranging from 8B to 70B parameters.

Significance. If the central ranking premise holds, OptiSeq is a simple, dataset-free, and model-agnostic contribution: it requires no training data, no fitted parameters, and only the LLM's own log-probability outputs. The algorithm is fully specified in Algorithm 1, and the evaluation covers a reasonable diversity of models and tasks. These are genuine strengths. However, the paper's central assumption, that zero-shot log-likelihood after removing in-context examples ranks outputs by correctness, is supported only by illustrative examples and global confidence distributions, not by any aggregate selection-quality metric. In addition, all headline numbers are point estimates from single greedy-decoding runs with no variance or significance information, which is insufficient for a journal-level empirical claim. The contribution is potentially publishable, but the current evidence does not yet establish the claimed mechanism or the robustness of the improvements.

major comments (4)
  1. [Section 3.1, selection step; Figures 6 and 9; Appendix Tables 5-13] The entire method depends on the premise that the zero-shot log-probability sum Phi_k, computed after removing in-context examples, is a valid proxy for whether the generated output is correct. The paper supports this only with anecdotal per-instance examples and a global confidence distribution, and it never reports an aggregate selection-quality metric, such as the fraction of instances where argmax_k Phi_k corresponds to a correct output, or the rank correlation between Phi_k and per-order accuracy across all instances and permutations. Without such a metric, the reported accuracy gains can arise even if Phi_k is a weak signal, because OptiSeq always examines multiple orderings and the instances where at least one order is correct may dominate the average. Please report the selection accuracy and a failure-mode analysis, especially for instances where the model is confidently wrong in zero-shot mode.
  2. [Section 5, Table 1] All results are reported as point estimates from greedy decoding, with no standard deviations, confidence intervals, or significance tests. The 'Random' baseline is underspecified: it is unclear whether each test instance is evaluated with a single random order and over how many random seeds the average is taken. Since the 5.5-10.5 percentage point improvements are the headline claim, the authors should report variability across multiple random draws and provide paired significance tests where possible.
  3. [Section 3.1, Eq. (1)] Phi_k is an unnormalized sum of per-token log probabilities, which systematically penalizes longer outputs. In API sequence generation, the number of APIs in a correct sequence is task-dependent, so selection based on the unnormalized sum could be driven by output length rather than by correctness. No length-normalized variant (e.g., average log-probability per token) is reported, and no analysis disentangling length effects from correctness effects is provided. Please add such an analysis or justify the unnormalized choice.
  4. [Section 4.3, 'Comparative Techniques'] The LocalE and Influence-score baselines are described as relying on validation data and label-fairness assumptions, yet they are then compared with OptiSeq in the same inference-time, dataset-free setting. The exact protocol used to instantiate these baselines in this paper's experiments (e.g., what development set was used and how the label-fairness assumptions were satisfied) is not documented. This makes the reported 5.5-6.5 percentage point gains over these baselines difficult to interpret and reproduce. Please specify the baseline configuration in the same detail as OptiSeq.
minor comments (4)
  1. [Figure 2 caption] The caption contains a typo: 'outpus' should be 'outputs'.
  2. [Section 5, first paragraph] The sentence 'OptiSeq achieves an average improvement of 10.5% points over random selection, 9.05% points over Top-K, 6.5% over LocalE and 5.5% over Influence score' reports aggregates, but Table 1 shows per-cell differences that vary substantially; please specify whether these averages are computed over datasets, models, or both, and report the per-cell raw differences in the main text.
  3. [Appendix Tables 5-13] Several displayed generated sequences appear malformed, for example 'SearchMovie, Mcosy.credits', 'SearchMovie, MovieCredits, ortrailing', and 'SearchMovie, B' in Table 7. If these are tokenization artifacts or truncated outputs, they should be explained; as presented, they undermine the qualitative evidence for the selection mechanism.
  4. [Section 2, last paragraph] The claim that 'the model is influenced by example order than by the examples themselves' appears to have a missing comparative word (likely 'more'), and the claim is not supported by a statistical test; please rephrase and provide quantitative support.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: OptiSeq is a fixed, parameter-free inference-time heuristic evaluated against external ground-truth benchmarks.

full rationale

The paper's derivation chain is self-contained. OptiSeq enumerates all permutations of the in-context examples, generates an output for each, removes the examples, and ranks the generated outputs by the sum of token log-probabilities conditioned only on the task instruction (Section 3.1, Phi_k definition). The selected ordering is k* = argmax_k Phi_k. This selection rule is a fully specified algorithm with no fitted parameters, no training data, and no use of the ground-truth labels in constructing the score; the reported accuracy gains are measured against external datasets (ToolBench, RestGPT, AG News, SST-5, TREC) and therefore do not reduce to the selection criterion by construction. The ranking premise that zero-shot log-likelihood is a useful proxy for correctness is an empirical heuristic, supported by distributions in Figures 6 and 9; whether it is adequately validated is a correctness-risk question, not circularity. EOptiSeq's anchoring heuristic is attributed to an external prior result (Liu et al., 2024b), and the only self-adjacent citation is the IBM model card, which is not load-bearing. No equation in the paper is equivalent to its input by definition, and no fitted parameter is renamed as a prediction. Score 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the empirical premise that zero-shot log-likelihood ranks output correctness, plus access to token-level log probabilities and to multiple forward passes per instance. No numeric parameters are fitted; the method is a fixed, inference-time heuristic.

assumptions (3)
  • domain assumption Zero-shot log-likelihood under instruction-only prompt ranks output correctness.
    Core premise of OptiSeq (Section 3.1, Eq. Phi_k and selection step); supported only by the empirical distributions in Figure 9, not by a derivation.
  • domain assumption The LLM's ordering sensitivity is instance-specific and cannot be predicted from inputs alone.
    Motivates the exhaustive search; established empirically in Section 2 (Figures 3-4), not theoretically.
  • domain assumption Token-level log-probabilities are available from the deployed LLM.
    Required by Algorithm 1 step 6; acknowledged as a limitation in Section 8.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OptiSeq: Ordering Examples On-The-Fly for In-Context Learning." pith.science (2026). https://pith.science/paper/BWULN2KF

@misc{pith2026250115030,
  author       = {Pith},
  title        = {Pith review of: OptiSeq: Ordering Examples On-The-Fly for In-Context Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BWULN2KF}},
  note         = {Machine review of arXiv:2501.15030}
}
read the original abstract

Developers using LLMs and LLM-based agents in their applications have provided plenty of anecdotal evidence that in-context-learning (ICL) is fragile. In this paper, we show that in addition to the quantity and quality of examples, the order in which the in-context examples are listed in the prompt affects the output of the LLM and, consequently, their performance. While prior work has explored improving ICL through dataset-dependent techniques, we introduce OptiSeq, a purely inference-time, dataset-free optimization method that efficiently determines the best example order. OptiSeq leverages log probabilities of LLM-generated outputs to systematically prune the search space of possible orderings and recommend the best order(s) by distinguishing orderings that yield high levels of accuracy and those that underperform. Extensive empirical evaluation on multiple LLMs, datasets, and prompts demonstrate that OptiSeq improves accuracy by 5.5 - 10.5 percentage points across multiple tasks.

Figures

Figures reproduced from arXiv: 2501.15030 by the authors.

Figure 1
Figure 1. llama-3-8b-instruct performance variation with 6 in-context example orderings using 3 examples. where limited data is available (Perez et al., 2021). They also develop mutual information- or entropy￾based heuristics (Sorensen et al., 2022; Lu et al., 2021; Guo et al., 2024) on these validation datasets, but are computationally limited to tasks like single￾class classification assuming label-balanced tasks and do not… view at source ↗
Figure 4
Figure 4. Variation in the average accuracy across all [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 2
Figure 2. Naive ICL exhibits higher overlap in log [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (7 more)
Figure 3
Figure 3. Figure 3: AG News classification accuracy across dif [PITH_FULL_IMAGE:figures/full_fig_p003_3.png]
Figure 5
Figure 5. Figure 5: OptiSeq Overview 1 2 3 4 5 6 Order 20 15 10 5 Log Probabilities Hard to distinguish Naive ICL Correct Incorrect (a) 1 2 3 4 5 6 Order 20 15 10 5 Log Probabilities Better distinguishability OptiSeq Correct Incorrect (b) [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Comparison of log probabilities with and [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Comparing 3, 4, and 5 shot random/Top-K selection strategy with 3 shot OptiSeq (in red) . The strategic ordering of a smaller number of examples in OptiSeq can significantly enhance performance compared to using a larger set of examples in a random/Top-K order. As high…
Figure 9
Figure 9. Figure 9: Probability distribution for correct and incor [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 12
Figure 12. Figure 12: Comparing 3, 4, and 5 shot random/Top-K selection strategy with 3 shot OptiSeq (in red) for the classification task. Dataset Labels Type ToolBench 100 Multi-class RestGPT 75 Multi-class AG_news 4 Single-class SST-5 5 Single-class TREC 6 Single-class [PITH_FULL_IMAGE:…
Figure 11
Figure 11. Figure 11: Distribution of Recall values for ToolBench [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 11 canonical work pages

  1. [1]

    Rishabh Agarwal, Avi Singh, Lei M Zhang, Bernd Bohnet, Luis Rosias, Stephanie Chan, Biao Zhang, Ankesh Anand, Zaheer Abbas, Azade Nova, et al. 2024. Many-shot in-context learning. arXiv preprint arXiv:2404.11018

    arXiv 2024

  2. [2]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  3. [3]

    Manish Chandra, Debasis Ganguly, and Iadh Ounis. 2024. One size doesn’t fit all: Predicting the number of examples for in-context learning

    work page 2024

  4. [4]

    Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. 2021. Meta-learning via language model in-context tuning. arXiv preprint arXiv:2110.07814

    arXiv 2021

  5. [5]

    Qi Guo, Leiyu Wang, Yidong Wang, Wei Ye, and Shikun Zhang. 2024. What makes a good order of examples in in-context learning. In Findings of the Association for Computational Linguistics: ACL 2024, pages 14892--14904

    2024

  6. [6]

    Shivanshu Gupta, Matt Gardner, and Sameer Singh. 2023. Coverage-based example selection for in-context learning. arXiv preprint arXiv:2305.14907

    arXiv 2023

  7. [7]

    Eduard H Hovy, Laurie Gerber, Ulf Hermjakob, Michael Junk, and Chin-Yew Lin. 2000. Question answering in webclopedia. In TREC, volume 52, pages 53--56

    work page 2000

  8. [8]

    IBM. 2023. Granite: Scaling language models with ibm's efficient architecture. IBM Research Journal. Available at https://research.ibm.com/granite

    work page 2023

Show all 38 references
  1. [9]

    Haoyu Liu, Jianfeng Liu, Shaohan Huang, Yuefeng Zhan, Hao Sun, Weiwei Deng, Furu Wei, and Qi Zhang. 2024 a . se2: Sequential example selection for in-context learning. In Findings of the Association for Computational Linguistics ACL 2024, pages 5262--5284

    2024

  2. [10]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. What makes good in-context examples for gpt- 3 ? arXiv preprint arXiv:2101.06804

    2021 arXiv

  3. [11]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024 b . Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157--173

    2024

  4. [12]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1--35

    2023

  5. [13]

    Yinpeng Liu, Jiawei Liu, Xiang Shi, Qikai Cheng, Yong Huang, and Wei Lu. 2024 c . Let's learn step by step: Enhancing in-context learning ability with curriculum learning. arXiv preprint arXiv:2402.10738

    2024 arXiv

  6. [14]

    Do Long, Yiran Zhao, Hannah Brown, Yuxi Xie, James Zhao, Nancy Chen, Kenji Kawaguchi, Michael Shieh, and Junxian He. 2024. Prompt optimization via adversarial in-context learning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volum...

    2024

  7. [15]

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786

    2021 arXiv

  8. [16]

    MistralAI. 2023. Mixtral: A diverse and scalable instruction-tuned language model. Mistral AI Technical Report. Available at https://mistral.ai/mixtral

    2023

  9. [17]

    Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021. True few-shot learning with language models. Advances in neural information processing systems, 34:11054--11070

    2021

  10. [18]

    Kiran Purohit, Raghuram Devalla, Krishna Mohan Yerragorla, Sourangshu Bhattacharya, Avishek Anand, et al. 2024. Explora: Efficient exemplar subset selection for complex reasoning. arXiv preprint arXiv:2411.03877

    2024 arXiv

  11. [19]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789

    2023 arXiv

  12. [20]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  13. [21]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  14. [22]

    N Reimers. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084

    2019 arXiv

  15. [23]

    Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980

    2020 arXiv

  16. [24]

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language...

    2013

  17. [25]

    Yifan Song, Weimin Xiong, Dawei Zhu, Wenhao Wu, Han Qian, Mingbo Song, Hailiang Huang, Cheng Li, Ke Wang, Rong Yao, et al. 2023. Restgpt: Connecting large language models with real-world restful apis. arXiv preprint arXiv:2306.06624

    2023 arXiv

  18. [26]

    Taylor Sorensen, Joshua Robinson, Christopher Michael Rytting, Alexander Glenn Shaw, Kyle Jeffrey Rogers, Alexia Pauline Delorey, Mahmoud Khalil, Nancy Fulda, and David Wingate. 2022. An information-theoretic approach to prompt engineering without ground truth labels. arXiv pr...

    2022 arXiv

  19. [27]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama 3: Open and efficient foundation language models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  20. [28]

    Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. 2022. Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering. arXiv preprint arXiv:2212.10375

    2022 arXiv

  21. [29]

    Zhichao Xu, Daniel Cohen, Bei Wang, and Vivek Srikumar. 2024. https://arxiv.org/abs/2402.11447 In-context example ordering guided by label distributions . Preprint, arXiv:2402.11447

    2024 arXiv

  22. [30]

    Zhao Yang, Yuanzhe Zhang, Dianbo Sui, Cao Liu, Jun Zhao, and Kang Liu. 2023. Representative demonstration selection for in-context learning with two-stage determinantal point process. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pa...

    2023

  23. [31]

    Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2023. Compositional exemplars for in-context learning. In International Conference on Machine Learning, pages 39818--39833. PMLR

    2023

  24. [32]

    Kaiyi Zhang, Ang Lv, Yuhan Chen, Hansen Ha, Tao Xu, and Rui Yan. 2024. Batch-icl: Effective, efficient, and order-agnostic in-context learning. arXiv preprint arXiv:2401.06469

    2024 arXiv

  25. [33]

    Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28

    2015

  26. [34]

    Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. 2023. What makes good examples for visual in-context learning? Advances in Neural Information Processing Systems, 36:17773--17794

    2023

  27. [35]

    Hao Zhao, Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2024. Is in-context learning sufficient for instruction following in llms? arXiv preprint arXiv:2405.19874

    2024 arXiv

  28. [36]

    Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning, pages 12697--12706. PMLR

    2021

  29. [37]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.