Pith. sign in

REVIEW 5 major objections 4 minor 81 references

The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a training-free four-step recipe—embedding, clustering, scoring, and voting—lets ten open-source ~7B models match or beat GPT-4.1's average across 15 benchmarks, including an 18.21% edge on mathematics.

desk verdict A genuinely useful training-free routing recipe whose headline 'beats GPT-4.1' claim dissolves under matched-evaluation scrutiny. read the letter →

arxiv 2505.19797 v3 pith:3OE23X4B submitted 2025-05-26 cs.CL

Yiqun Zhang , Hao Li , Chenxu Wang , Linyao Chen , Qiaosheng Zhang , Peng Ye , Shi Feng , Daling Wang
show 6 more authors
Zhen Wang Xinrun Wang Jia Xu Lei Bai Wanli Ouyang Shuyue Hu
This is my paper
classification cs.CL
keywords collectiveintelligencemodelroutingclustering-basedrouterself-consistencyvotingsmalllanguagemodelstraining-freeensembleopen-sourceLLMs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that collective intelligence among small open-source models can match or beat flagship proprietary models without any neural-network training. The recipe is four lightweight steps: embed each query, cluster queries by semantic similarity, score every candidate model on validation queries within each cluster, and route each test query to the best-scoring model or models, then sample repeatedly and take a majority vote. Across 15 datasets spanning mathematics, code, logic, knowledge, and affective tasks, ten ~7B models reach an average of 70.54, above GPT-4.1's 69.20, and the method also generalizes to five held-out out-of-distribution datasets. The paper further claims the approach is robust to the choice of embedding model, clustering algorithm, and cluster count, and that it can automatically select a complementary team of models from a larger candidate pool.

What carries the argument

The load-bearing object is the cluster-wise capability profile: a vector over K clusters recording each model's validation accuracy on queries embedded into that cluster. With K=64 K-means clusters and the gte-qwen2-7B-instruct embedding model, the profile turns routing into a nearest-centroid lookup: a test query's embedding lands in one cluster, and the profile says which model or models are strongest there. The second mechanism is repeated sampling and voting: Self-Consistency for a single selected model, or Model-Switch when two models are selected, converting per-cluster routing into a stable final answer. The chosen cluster count sits in a stability plateau between under-clustering and over-clustering, which the paper identifies as the reason training-free routing generalizes where trained routers overfit.

What would settle it

Take a test set, assign each query to its validation-built cluster, and compute each model's per-cluster accuracy on the test set; if the per-cluster ranking of models on the test set frequently disagrees with the ranking on the validation set for the same cluster, the routing gain should vanish or reverse, and a simpler check is whether randomly shuffled cluster assignments drop the average score to about the level of a random router.

Watch

Extended reading notes

Core claim

The central discovery the paper tries to establish is that a clustering-based router, not trained at all, plus repeated sampling with majority voting, can raise a team of ten open ~7B models above the average of GPT-4.1 and above GPT-4o and GPT-4.5 as well. The mechanism is a per-cluster capability profile: validation queries are embedded and K-means clustered, each model is scored on each cluster, and at inference each test query is assigned to the nearest cluster centroid, where the top model or top two models generate multiple samples that are voted into a final answer. The paper reports an 18.21% improvement over GPT-4.1 on mathematics tasks and a 7.46% improvement on code tasks, while conceding the proprietary model still leads on specialized knowledge tasks such as GPQA and MedQA. On out-of-distribution benchmarks the routing advantage is larger, exceeding the best trained router baseline by at least 8%. This is the first demonstration, the authors argue, that a training-free router can lift open small models to the overall performance of a proprietary flagship model.

Load-bearing premise

The whole gain rests on the assumption that the model that performs best on validation queries in a cluster is also the best model on new test queries that land in that same cluster; if cluster membership does not carry model competence from validation to test, the routing advantage disappears.

Editorial extensions

If this is right

  • If correct, any open-source team with ~7B models can compete with proprietary flagships by plugging models into this recipe, with no training involved.
  • New models can be added by evaluating them on the same validation set and computing a new capability profile, without re-clustering.
  • New tasks and datasets can be absorbed by re-running offline calibration, requiring only incremental re-evaluation of existing models.
  • Automatic model selection means the composition of the team matters less than the diversity of specializations, with finance- and medical-tuned models being chosen for queries in their niches.
  • The cluster count K is the only hyperparameter, and performance stays stable across a wide range of K values, with stronger embeddings and ensembles only improving results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If cluster-level model rankings transfer across datasets of the same type, the recipe could be applied to a closed test set without labeled validation by transferring profiles from a proxy dataset; this transferability is not tested in the paper.
  • The paper's over-clustering explanation suggests trained routers fail because they overfit cluster idiosyncrasies; a direct test would be to train a router with early stopping on validation and compare its generalization curve to the K-sweep curve.
  • As the open-source ecosystem grows, the ceiling of the recipe should rise with model diversity rather than model size; one could test this by adding more specialized small models and watching whether average performance continues to climb.
  • Because the paper only claims average superiority, users who care about worst-case or knowledge-heavy tasks might not benefit; testing on tasks where one proprietary model dominates could reveal where the routing ceiling sits below GPT-4.1.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes the Avengers, a training-free recipe for combining multiple small language models. The method embeds validation queries, clusters them with K-means, scores each candidate model per cluster, and at inference time routes each query to the top-performing model(s) in its nearest cluster, followed by repeated sampling and majority voting. With 10 open-source models of roughly 7B parameters, the paper reports an average score of 70.54 across 15 benchmarks versus 69.20 for GPT-4.1, claims advantages on mathematics and code tasks, and reports robustness across embeddings, clustering algorithms, ensemble strategies, and the number of clusters. The paper also introduces automatic model-set construction and evaluates five additional datasets under a protocol it calls out-of-distribution.

Significance. If the evaluation were properly controlled, the paper would make a useful and striking contribution: a simple, training-free, open-sourced recipe for assembling open 7B models into a system that rivals a proprietary flagship model. The paper also has concrete strengths: it releases code, averages results over five seeds, includes detailed ablations over clustering methods and embeddings, and openly discusses limitations. However, the central comparison with proprietary models is currently not controlled, the MBPP score for GPT-4.1 is internally anomalous, and the out-of-distribution claim is overstated by the protocol actually used. These issues are load-bearing because the headline advantage over GPT-4.1 is only about 1.3 points and depends on a single questionable benchmark number.

major comments (5)
  1. [§4.1, §4.2, Table 2] The comparison that supports the headline 'surpasses GPT-4.1' is not controlled for test-time compute. Section 4.1 states that all SLM and router-based baselines use the Avengers’ 10-sample Self-Consistency, but it does not state that GPT-4o, GPT-4.5, or GPT-4.1 receive the same treatment. Since the average margin is only 1.34 points (70.54 vs 69.20) and Self-Consistency at temperature 0.7 can substantially improve math and code accuracy, the claimed advantage may reflect sampling budget rather than routing. Please either evaluate the proprietary models with the same 10-sample Self-Consistency protocol, or compare all methods in the single-sample CoT setting, and report both sets of numbers.
  2. [§4.2, Table 2 (MBPP row)] GPT-4.1’s MBPP score of 57.70 is internally inconsistent with GPT-4o’s 82.64 and GPT-4.5’s 86.69 in the same table, and it is also inconsistent with GPT-4.1’s own HumanEval score of 92.07. Because MBPP is one of only two code datasets, this single value drives the reported 7.46% code advantage and contributes substantially to the 1.34-point overall margin. If the MBPP value were corrected to a level near the other GPT models, the overall average comparison would reverse (for example, replacing 57.70 with 82.64 would raise GPT-4.1’s average to approximately 70.86). Please verify this number and report the corrected table; if this is an evaluation artifact, the headline claim needs to be revised.
  3. [§3 (New Datasets and Newly Available Models), Table 3] The 'OOD' evaluation is not out-of-distribution in the usual sense. The procedure in Section 3 explicitly permits labeled validation data from each new dataset D′ to be used for re-clustering and re-evaluation, so Table 3 measures held-out test performance after in-distribution calibration on the same benchmark. The abstract’s claim of 'superior out-of-distribution generalization' is therefore overstated. Please either remove D′ labels from the calibration (for example, route OOD queries using clusters built solely from source data) or relabel the experiment as held-out generalization.
  4. [§4.1 (Enhancement to the Baselines), Tables 2–3] Reporting trainable routers at their peak test-set performance (marked with †) is not a valid evaluation protocol, because it uses test labels to select the checkpoint. This applies to RouterDC, EmbedLLM, and MODEL-SAT. The comparison should select checkpoints by validation performance or by a fixed training budget; otherwise the claim of achieving the best average performance against trained routers is an upper-envelope comparison rather than a comparison of achievable methods. Please report the checkpoint-selection criterion and, ideally, validation/test curves with the chosen point marked.
  5. [§3 (Offline Calibration), Figure 4] The central routing mechanism assumes that the per-cluster model performance rankings measured on the validation split transfer to test queries assigned to the same clusters. The paper does not provide direct evidence for this transfer; Figure 4 demonstrates aggregate stability of average scores over K but not cluster-level ranking stability. Please report a measure such as rank correlation between validation and test per-cluster model scores, or the fraction of clusters where the selected top model is also the test-best model, to substantiate the 'horses for courses' mechanism.
minor comments (4)
  1. [§3, opening paragraph] There is a duplicated phrase: 'Given a set of models and a dataset a dataset' should read 'Given a set of models and a dataset.' Also, 'theAvengers' is missing a space in multiple places.
  2. [§4.2, Table 2] The text says the Avengers outperforms GPT-4.1 on 10 of the 15 datasets, but the arrows in Table 2 show only nine such wins (AIME, MATH500, LiveMathBench, MBPP, KORBench, MMLUPro, FinQA, EmoryNLP, MELD). The Introduction and Conclusion correctly say nine; please reconcile the count.
  3. [Abstract and §3] The abstract describes K as the 'sole parameter,' but Section 3 also includes the number of selected models n, the number of sampling rounds, and the sampling temperature as hyperparameters. Please qualify the claim by saying these are fixed by default rather than absent.
  4. [Table 4 and Appendix B.3] Table 4 reports the gte-qwen2-7B embedding dimension as 3,584, while Appendix A.2 reports model parameters in inconsistent formats; please unify units and abbreviations for model sizes and embedding dimensions.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the only self-citation is optional and non-load-bearing.

full rationale

The paper presents an empirical routing recipe rather than a formal derivation chain, so most circularity patterns do not apply. The capability profile p_i is measured on a validation split and used to select models for a disjoint test split; the selected model's test score is not defined in terms of the profile, so there is no self-definition. The headline comparison to GPT-4.1 is an external benchmark, not an input to the method. The only self-citation is Model-Switch [21], used when n>1; the default ensemble is Self-Consistency [37], an external method, so this citation is not load-bearing. The paper itself flags the need for a labeled validation set (Appendix B.6), and the 'OOD' evaluation re-calibrates on each OOD dataset's validation split, which weakens the OOD claim but is a soundness issue rather than internal circularity. Accordingly, no specific circular step can be exhibited with a quote, and the appropriate finding is no significant circularity with a minor self-citation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new entities and performs no theoretical derivation. The free parameters are experimental settings, most of which are shown to be robust. The load-bearing assumptions are the transferability of cluster-wise capabilities from validation to test and the uniform benefit of self-consistency voting.

free parameters (5)
  • Number of clusters K = 64
    Chosen by the authors; paper shows robustness across K from about 14 to 140, but the main results use K=64.
  • Self-consistency sample count = 10
    Number of repeated samples for voting; affects compute and performance.
  • Sampling temperature = 0.7
    Temperature for SC sampling; set to 0.7 per standard SC practice.
  • Number of selected models n = 1 (default)
    Number of top models selected per cluster; default single model with SC, optional multi-model with Model-Switch.
  • Validation split ratio = 0.7/0.3
    70% of each dataset used for calibration, 30% for testing.
assumptions (5)
  • domain assumption Semantic embedding (gte-qwen2-7B-instruct) places queries with similar model-performance rankings close together, so cluster assignment transfers from validation to test.
    Section 3, offline calibration and online inference; this is the core inductive assumption of routing.
  • domain assumption Cluster-wise performance on the validation split is representative of test-split performance for the same cluster.
    Section 4.1, validation/test splits; standard supervised assumption.
  • domain assumption Majority voting over 10 self-consistent samples improves accuracy for all models and tasks used.
    Section 4.2, Table 6; based on prior SC literature [37], but the uniform benefit is assumed.
  • domain assumption The proprietary GPT baselines were evaluated correctly with their standard interfaces; the paper's reported numbers are accurate.
    Section 4.1 baseline description; we cannot verify API calls.
  • domain assumption The 22-model candidate pool is diverse enough to cover the task space.
    Section 4.1 and Appendix A.2; the pool is a convenience sample of open models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants." pith.science (2026). https://pith.science/paper/3OE23X4B

@misc{pith2026250519797,
  author       = {Pith},
  title        = {Pith review of: The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3OE23X4B}},
  note         = {Machine review of arXiv:2505.19797}
}
read the original abstract

Proprietary giants are increasingly dominating the race for ever-larger language models. Can open-source, smaller models remain competitive across a broad range of tasks? In this paper, we present the Avengers -- a simple recipe that leverages the collective intelligence of these smaller models. The Avengers builds upon four lightweight operations: (i) embedding: encode queries using a text embedding model; (ii) clustering: group queries based on their semantic similarity; (iii) scoring: scores each model's performance within each cluster; and (iv) voting: improve outputs via repeated sampling and voting. At inference time, each query is embedded and assigned to its nearest cluster. The top-performing model(s) within that cluster are selected to generate the response with repeated sampling. Remarkably, with 10 open-source models (~7B parameters each), the Avengers surpasses GPT-4o, 4.1, and 4.5 in average performance across 15 diverse datasets spanning mathematics, coding, logical reasoning, general knowledge, and affective tasks. In particular, it surpasses GPT-4.1 on mathematics tasks by 18.21% and on code tasks by 7.46%. Furthermore, the Avengers delivers superior out-of-distribution generalization, and remains robust across various embedding models, clustering algorithms, ensemble strategies, and values of its sole parameter -- the number of clusters.

Figures

Figures reproduced from arXiv: 2505.19797 by the authors.

Figure 1
Figure 1. Comparison of proprietary models with our method (the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The Avengers, a simple recipe for collective intelligence of smaller language models. 1 Introduction The current landscape of language models (LMs) is dominated by few proprietary giants. The race to build ever-larger models is far from over; however, it is becoming a closed competition, accessible only to organizations with massive resources. This trend risks marginalizing the broader research community and concent… view at source ↗
Figure 3
Figure 3. Impact of selected models’ number. 1 10 100 1000 10000 Number of Clusters (K) - Log Scale 50 55 60 65 70 75 80 85 Average Score (%) Under-clustering Stability Over-clustering Oracle GPT-4.1 Average Valid Test [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: The training and test accuracy curves on RouterDC. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: The training and test accuracy curves on MODEL-SAT. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: The training and test accuracy curves on EmbedLLM. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: The impact of test set proportion on the [PITH_FULL_IMAGE:figures/full_fig_p023_8.png]
Figure 9
Figure 9. Figure 9: Distribution of the routing results of the [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

81 extracted references · 16 canonical work pages

  1. [36]

    Universal model routing for efficient llm inference.arXiv preprint arXiv:2502.08773, 2025

    Wittawat Jitkrittum, Harikrishna Narasimhan, Ankit Singh Rawat, Jeevesh Juneja, Zifeng Wang, Chen-Yu Lee, Pradeep Shenoy, Rina Panigrahy, Aditya Krishna Menon, and Sanjiv Kumar. Universal model routing for efficient llm inference.arXiv preprint arXiv:2502.08773, 2025

    arXiv 2025

  2. [3]

    Ultramedical: Building specialized generalists in biomedicine.Advances in Neural Information Processing Systems, 37:26045–26081, 2024

    Kaiyan Zhang, Sihang Zeng, Ermo Hua, Ning Ding, Zhang-Ren Chen, Zhiyuan Ma, Haoxin Li, Ganqu Cui, Biqing Qi, Xuekai Zhu, et al. Ultramedical: Building specialized generalists in biomedicine.Advances in Neural Information Processing Systems, 37:26045–26081, 2024

    2024

  3. [4]

    Process reinforcement through implicit rewards.arXiv preprint arXiv:2502.01456, 2025

    Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. Process reinforcement through implicit rewards.arXiv preprint arXiv:2502.01456, 2025

    arXiv 2025

  4. [5]

    Building community–centered ai collaborations.Stanford Social Innovation Review, 2025

    Michelle Flores Vryn and Meena Das. Building community–centered ai collaborations.Stanford Social Innovation Review, 2025. doi: 10.48558/KF5R-C507. URL https://ssir.org/ articles/entry/community-centered-ai-collaborations

    work page doi:10.48558/kf5r-c507 2025

  5. [6]

    Green ai.Communications of the ACM, 63(12):54–63, 2020

    Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. Green ai.Communications of the ACM, 63(12):54–63, 2020

    2020

  6. [7]

    Llm-blender: Ensembling large language models with pairwise ranking and generative fusion

    Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. Llm-blender: Ensembling large language models with pairwise ranking and generative fusion. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14165–14178, 2023

    2023

  7. [8]

    Routerdc: Query-based router by dual contrastive learning for assembling large language models.Advances in Neural Information Processing Systems, 37:66305–66328, 2024

    Shuhao Chen, Weisen Jiang, Baijiong Lin, James Kwok, and Yu Zhang. Routerdc: Query-based router by dual contrastive learning for assembling large language models.Advances in Neural Information Processing Systems, 37:66305–66328, 2024

    2024

  8. [9]

    Embedllm: Learning compact representations of large language models.arXiv preprint arXiv:2410.02223, 2024

    Richard Zhuang, Tianhao Wu, Zhaojin Wen, Andrew Li, Jiantao Jiao, and Kannan Ramchan- dran. Embedllm: Learning compact representations of large language models.arXiv preprint arXiv:2410.02223, 2024

    arXiv 2024

Show all 81 references
  1. [10]

    Capability instruction tuning: A new paradigm for dynamic llm routing.arXiv preprint arXiv:2502.17282, 2025

    Yi-Kai Zhang, De-Chuan Zhan, and Han-Jia Ye. Capability instruction tuning: A new paradigm for dynamic llm routing.arXiv preprint arXiv:2502.17282, 2025

    2025 arXiv

  2. [11]

    Mixture-of-agents enhances large language model capabilities.arXiv preprint arXiv:2406.04692, 2024

    Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agents enhances large language model capabilities.arXiv preprint arXiv:2406.04692, 2024. 10

    2024 arXiv

  3. [12]

    Smoa: Improving multi-agent large language models with sparse mixture-of-agents.arXiv preprint arXiv:2411.03284, 2024

    Dawei Li, Zhen Tan, Peijia Qian, Yifan Li, Kumar Satvik Chaudhary, Lijie Hu, and Jiayi Shen. Smoa: Improving multi-agent large language models with sparse mixture-of-agents.arXiv preprint arXiv:2411.03284, 2024

    2024 arXiv

  4. [13]

    Rethinking mixture-of-agents: Is mixing different large language models beneficial?arXiv preprint arXiv:2502.00674, 2025

    Wenzhe Li, Yong Lin, Mengzhou Xia, and Chi Jin. Rethinking mixture-of-agents: Is mixing different large language models beneficial?arXiv preprint arXiv:2502.00674, 2025

    2025 arXiv

  5. [14]

    Symbolic mixture-of-experts: Adaptive skill-based routing for heterogeneous reasoning, 2025

    Justin Chih-Yao Chen, Sukwon Yun, Elias Stengel-Eskin, Tianlong Chen, and Mohit Bansal. Symbolic mixture-of-experts: Adaptive skill-based routing for heterogeneous reasoning, 2025. URLhttps://arxiv.org/abs/2503.05641

    2025 arXiv

  6. [15]

    Least squares quantization in pcm.IEEE transactions on information theory, 28 (2):129–137, 1982

    Stuart Lloyd. Least squares quantization in pcm.IEEE transactions on information theory, 28 (2):129–137, 1982

    1982

  7. [16]

    Some methods for classification and analysis of multivariate observations

    James MacQueen. Some methods for classification and analysis of multivariate observations. InProceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics, volume 5, pages 281–298. University of California press, 1967

    1967

  8. [17]

    Hierarchical grouping to optimize an objective function.Journal of the American statistical association, 58(301):236–244, 1963

    Joe H Ward Jr. Hierarchical grouping to optimize an objective function.Journal of the American statistical association, 58(301):236–244, 1963

    1963

  9. [18]

    Maximum likelihood from incomplete data via the em algorithm.Journal of the royal statistical society: series B (methodological), 39 (1):1–22, 1977

    Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm.Journal of the royal statistical society: series B (methodological), 39 (1):1–22, 1977

    1977

  10. [19]

    A tutorial on spectral clustering.Statistics and computing, 17:395–416, 2007

    Ulrike V on Luxburg. A tutorial on spectral clustering.Statistics and computing, 17:395–416, 2007

    2007

  11. [20]

    Birch: an efficient data clustering method for very large databases.ACM sigmod record, 25(2):103–114, 1996

    Tian Zhang, Raghu Ramakrishnan, and Miron Livny. Birch: an efficient data clustering method for very large databases.ACM sigmod record, 25(2):103–114, 1996

    1996

  12. [21]

    Do we truly need so many samples? multi-llm repeated sampling efficiently scales test-time compute, 2025

    Jianhao Chen, Zishuo Xun, Bocheng Zhou, Han Qi, Qiaosheng Zhang, Yang Chen, Wei Hu, Yuzhong Qu, Wanli Ouyang, and Shuyue Hu. Do we truly need so many samples? multi-llm repeated sampling efficiently scales test-time compute, 2025. URL https://arxiv.org/ abs/2504.00762

    2025

  13. [22]

    Merge, ensemble, and cooperate! a survey on collaborative strategies in the era of large language models.arXiv preprint arXiv:2407.06089, 2024

    Jinliang Lu, Ziliang Pang, Min Xiao, Yaochen Zhu, Rui Xia, and Jiajun Zhang. Merge, ensemble, and cooperate! a survey on collaborative strategies in the era of large language models.arXiv preprint arXiv:2407.06089, 2024

    2024 arXiv

  14. [23]

    Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680, 2024

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680, 2024

    2024 arXiv

  15. [24]

    If multi-agent debate is the answer, what is the question?arXiv preprint arXiv:2502.08788, 2025

    Hangfan Zhang, Zhiyao Cui, Xinrun Wang, Qiaosheng Zhang, Zhen Wang, Dinghao Wu, and Shuyue Hu. If multi-agent debate is the answer, what is the question?arXiv preprint arXiv:2502.08788, 2025

    2025 arXiv

  16. [25]

    Multiagent finetuning: Self improvement with diverse reasoning chains.arXiv preprint arXiv:2501.05707, 2025

    Vighnesh Subramaniam, Yilun Du, Joshua B Tenenbaum, Antonio Torralba, Shuang Li, and Igor Mordatch. Multiagent finetuning: Self improvement with diverse reasoning chains.arXiv preprint arXiv:2501.05707, 2025

    2025 arXiv

  17. [26]

    Rema: Learning to meta-think for llms with multi-agent reinforcement learning.arXiv preprint arXiv:2503.09501, 2025

    Ziyu Wan, Yunxiang Li, Yan Song, Hanjing Wang, Linyi Yang, Mark Schmidt, Jun Wang, Weinan Zhang, Shuyue Hu, and Ying Wen. Rema: Learning to meta-think for llms with multi-agent reinforcement learning.arXiv preprint arXiv:2503.09501, 2025

    2025 arXiv

  18. [27]

    Decouple and orthogonalize: A data-free framework for lora merging.arXiv preprint arXiv:2505.15875, 2025

    Shenghe Zheng, Hongzhi Wang, Chenyu Huang, Xiaohui Wang, Tao Chen, Jiayuan Fan, Shuyue Hu, and Peng Ye. Decouple and orthogonalize: A data-free framework for lora merging.arXiv preprint arXiv:2505.15875, 2025

    2025 arXiv

  19. [28]

    Frugalgpt: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research, 2023

    Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance.Transactions on Machine Learning Research, 2023. 11

    2023

  20. [29]

    Large language model routing with benchmark datasets

    Tal Shnitzer, Anthony Ou, Mírian Silva, Kate Soule, Yuekai Sun, Justin Solomon, Neil Thomp- son, and Mikhail Yurochkin. Large language model routing with benchmark datasets. InFirst Conference on Language Modeling, 2023

    2023

  21. [30]

    Routellm: Learning to route llms from preference data

    Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route llms from preference data. In The Thirteenth International Conference on Learning Representations, 2024

    2024

  22. [31]

    Graphrouter: A graph-based router for LLM selections

    Tao Feng, Yanzhen Shen, and Jiaxuan You. Graphrouter: A graph-based router for LLM selections. InThe Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum?id=eU39PDsZtT

    2025

  23. [32]

    Learning to decode collaboratively with multiple language models

    Zejiang Shen, Hunter Lang, Bailin Wang, Yoon Kim, and David Sontag. Learning to decode collaboratively with multiple language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12974–12990, 2024

    2024

  24. [33]

    Routereval: A comprehensive benchmark for routing llms to explore model-level scaling up in llms.arXiv preprint arXiv:2503.10657, 2025

    Zhongzhan Huang, Guoming Ling, Vincent S Liang, Yupei Lin, Yandong Chen, Shanshan Zhong, Hefeng Wu, and Liang Lin. Routereval: A comprehensive benchmark for routing llms to explore model-level scaling up in llms.arXiv preprint arXiv:2503.10657, 2025

    2025 arXiv

  25. [34]

    Routing to the expert: Efficient reward-guided ensemble of large language models

    Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. Routing to the expert: Efficient reward-guided ensemble of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational ...

    2024

  26. [35]

    Routerbench: A benchmark for multi-llm routing system

    Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. Routerbench: A benchmark for multi-llm routing system. InAgentic Markets Workshop at ICML 2024, 2024

    2024

  27. [37]

    Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXiv preprint arXiv:2203.11171, 2022

    2022 arXiv

  28. [38]

    Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step.arXiv preprint arXiv:2305.20050, 2023

    2023 arXiv

  29. [39]

    Are your llms capable of stable reasoning?arXiv preprint arXiv:2412.13147, 2024

    Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, and Kai Chen. Are your llms capable of stable reasoning?arXiv preprint arXiv:2412.13147, 2024

    2024 arXiv

  30. [40]

    Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732, 2021

    2021 arXiv

  31. [41]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott G...

    2021

  32. [42]

    Kor-bench: Benchmarking language models on knowledge-orthogonal reasoning tasks, 2024

    Kaijing Ma, Xinrun Du, Yunran Wang, Haoran Zhang, Zhoufutu Wen, Xingwei Qu, Jian Yang, Jiaheng Liu, Minghao Liu, Xiang Yue, Wenhao Huang, and Ge Zhang. Kor-bench: Benchmarking language models on knowledge-orthogonal reasoning tasks, 2024. URL https: //arxiv.org/abs/2410.06526

    2024 arXiv

  33. [43]

    On memorization of large language models in logical reasoning

    Chulin Xie, Yangsibo Huang, Chiyuan Zhang, Da Yu, Xinyun Chen, Bill Yuchen Lin, Bo Li, Badih Ghazi, and Ravi Kumar. On memorization of large language models in logical reasoning. arXiv preprint arXiv:2410.23123, 2024

    2024 arXiv

  34. [44]

    Challenging big-bench tasks and whether chain-of-thought can solve them.arXiv preprint arXiv:2210.09261, 2022

    Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them.arXiv preprint arXiv:2210.09261, 2022

    arXiv

  35. [45]

    Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  36. [46]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark.arXiv preprint arXiv:2406.01574, 2024

    2024 arXiv

  37. [47]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google- proof q&a benchmark. InFirst Conference on Language Modeling, 2024. URL https: //openreview.net/forum?id=Ti67584b98

    2024

  38. [48]

    Finqa: A dataset of numerical reasoning over financial data.Proceedings of EMNLP 2021, 2021

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. Finqa: A dataset of numerical reasoning over financial data.Proceedings of EMNLP 2021, 2021

    2021

  39. [49]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams.Applied Sciences, 11(14):6421, 2021

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams.Applied Sciences, 11(14):6421, 2021

    2021

  40. [50]

    Ternary twitter sentiment classification with distant supervision and sentiment-specific word embeddings

    Mats Byrkjeland, Frederik Gørvell de Lichtenberg, and Björn Gambäck. Ternary twitter sentiment classification with distant supervision and sentiment-specific word embeddings. In Proceedings of the 9th workshop on computational approaches to subjectivity, sentiment and social m...

    2018

  41. [51]

    Meld: A multimodal multi-party dataset for emotion recognition in conversa- tions, 2019

    Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. Meld: A multimodal multi-party dataset for emotion recognition in conversa- tions, 2019. URLhttps://arxiv.org/abs/1810.02508

    2019 arXiv

  42. [52]

    Mathbench: Evaluating the theory and application proficiency of llms with a hierarchical mathematics benchmark.arXiv preprint arXiv:2405.12209, 2024

    Hongwei Liu, Zilong Zheng, Yuxuan Qiao, Haodong Duan, Zhiwei Fei, Fengzhe Zhou, Wenwei Zhang, Songyang Zhang, Dahua Lin, and Kai Chen. Mathbench: Evaluating the theory and application proficiency of llms with a hierarchical mathematics benchmark.arXiv preprint arXiv:2405.12209, 2024

    2024 arXiv

  43. [53]

    Studenteval: A benchmark of student-written prompts for large language models of code.arXiv preprint arXiv:2306.04556, 2023

    Hannah McLean Babe, Sydney Nguyen, Yangtian Zi, Arjun Guha, Molly Q Feldman, and Carolyn Jane Anderson. Studenteval: A benchmark of student-written prompts for large language models of code.arXiv preprint arXiv:2306.04556, 2023

    2023 arXiv

  44. [54]

    Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale.Communications of the ACM, 64(9):99–106, 2021

    2021

  45. [55]

    Nimz at semeval-2024 task 9: Evaluating methods in solving brainteasers defying commonsense

    Zahra Rahimi, Mohammad Moein Shirzady, Zeinab Taghavi, and Hossein Sameti. Nimz at semeval-2024 task 9: Evaluating methods in solving brainteasers defying commonsense. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), pages 148–154, 2024. 13

    2024

  46. [56]

    Dailydialog: A manually labelled multi-turn dialogue dataset.arXiv preprint arXiv:1710.03957, 2017

    Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. Dailydialog: A manually labelled multi-turn dialogue dataset.arXiv preprint arXiv:1710.03957, 2017

    2017 arXiv

  47. [57]

    Bge m3- embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3- embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024

    2024

  48. [58]

    New embedding models and api updates

    OpenAI. New embedding models and api updates. https://openai.com/index/ new-embedding-models-and-api-updates/, January 2024. Accessed: 2025-05-10

    2024

  49. [59]

    Towards general text embeddings with multi-stage contrastive learning.arXiv preprint arXiv:2308.03281, 2023

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning.arXiv preprint arXiv:2308.03281, 2023

    2023 arXiv

  50. [60]

    Ties-merging: Resolving interference when merging models.Advances in Neural Information Processing Systems, 36:7093–7115, 2023

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models.Advances in Neural Information Processing Systems, 36:7093–7115, 2023

    2023

  51. [61]

    Language models are super mario: Absorbing abilities from homologous models as a free lunch

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. Language models are super mario: Absorbing abilities from homologous models as a free lunch. InForty-first International Conference on Machine Learning, 2024

    2024

  52. [62]

    Model swarms: Col- laborative search to adapt llm experts via swarm intelligence.arXiv preprint arXiv:2410.11163, 2024

    Shangbin Feng, Zifeng Wang, Yike Wang, Sayna Ebrahimi, Hamid Palangi, Lesly Miculicich, Achin Kulshrestha, Nathalie Rauschmayr, Yejin Choi, Yulia Tsvetkov, et al. Model swarms: Col- laborative search to adapt llm experts via swarm intelligence.arXiv preprint arXiv:2410.11163, 2024

    arXiv

  53. [63]

    Lorahub: Efficient cross-task generalization via dynamic lora composition

    Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. Lorahub: Efficient cross-task generalization via dynamic lora composition. InFirst Conference on Language Modeling, 2023

    2023

  54. [64]

    Nature-inspired population-based evolution of large language models.arXiv preprint arXiv:2503.01155, 2025

    Yiqun Zhang, Peng Ye, Xiaocui Yang, Shi Feng, Shufei Zhang, Lei Bai, Wanli Ouyang, and Shuyue Hu. Nature-inspired population-based evolution of large language models.arXiv preprint arXiv:2503.01155, 2025

    2025 arXiv

  55. [65]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, 2022

    2022

  56. [66]

    What matters for model merging at scale?arXiv preprint arXiv:2410.03617, 2024

    Prateek Yadav, Tu Vu, Jonathan Lai, Alexandra Chronopoulou, Manaal Faruqui, Mohit Bansal, and Tsendsuren Munkhdalai. What matters for model merging at scale?arXiv preprint arXiv:2410.03617, 2024

    2024 arXiv

  57. [67]

    Fin-r1: A large language model for financial reasoning through reinforcement learning, 2025

    Zhaowei Liu, Xin Guo, Fangqi Lou, Lingfeng Zeng, Jinyi Niu, Zixuan Wang, Jiajie Xu, Weige Cai, Ziwei Yang, Xueqian Zhao, Chao Li, Sheng Xu, Dezhi Chen, Yun Chen, Zuo Bai, and Liwen Zhang. Fin-r1: A large language model for financial reasoning through reinforcement learning, 20...

    2025

  58. [68]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  59. [69]

    Qwen2.5-coder technical report, 2024

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou...

    2024 arXiv

  60. [70]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 14

    2025 arXiv

  61. [71]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  62. [72]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  63. [73]

    Chatglm: A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793, 2024

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793, 2024

    2024 arXiv

  64. [74]

    Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras, 2025

    Microsoft, :, Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, Dong Chen, Dongdong Chen, Junkun Chen, Weizhu Chen, Yen-Chun Chen, Yi ling Chen, Qi Dai, Xiyang Dai, Ruchao ...

    2025 arXiv

  65. [75]

    The falcon 3 family of open models, December 2024

    TII Team. The falcon 3 family of open models, December 2024

    2024

  66. [76]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  67. [77]

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Ak- shita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, ...

    2024 arXiv

  68. [78]

    Internlm2 technical report, 2024

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang,...

    2024

  69. [79]

    Hermes 3 technical report, 2024

    Ryan Teknium, Jeffrey Quesnelle, and Chen Guang. Hermes 3 technical report, 2024. URL https://arxiv.org/abs/2408.11857

    2024 arXiv

  70. [80]

    Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs, 2025

    Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, Yihan Cao, Hui Ren, Xiang Li, Xiaoxiao Li, and Yuyin Zhou. Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs, 2025. URLhttp...

    2025 arXiv

  71. [81]

    Llama-nemotron: Efficient reasoning models, 2025

    Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, Ido Shahaf, Oren Tropp, Ehud Karpas, Ran Zilberstein, Jiaqi Zeng, Soumye Singhal, Alexander Bukharin, Yian Zhang, Tugrul Konuk, Ger- ald Shen,...

    2025

  72. [82]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V . Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Lu...

    2024 arXiv

  73. [83]

    thinker” rather than a “calculator

    01. AI, :, Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, Kaidong Yu, Peng Liu, Qiang Liu, Shawn Yue, Senbin Yang, Shiming Yang, Wen Xie, Wenhao Huang, Xiaohui Hu, Xiaoyi Ren, Xinyao Niu, P...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.