Pith. sign in

REVIEW 4 major objections 5 minor 89 references

Compass-V2 Technical Report

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 5B-active-parameter model claims state-of-the-art Southeast Asian language and e-commerce performance among sub-30B models.

desk verdict Real MoE engineering, no artifacts; the SOTA claim is a claim, not a result. read the letter →

arxiv 2504.15527 v1 pith:JM5JH5FN submitted 2025-04-22 cs.CL

Sophia Maria This is my paper · ORCID
classification cs.CL
keywords Mixture-of-ExpertsSoutheastAsianlanguagese-commercehybridreasoninglow-resourceNLPtokenizationdirectpreferenceoptimizationmodelquantization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Compass-v2 is a Mixture-of-Experts language model with 30B total parameters but only 5B active per token, trained from scratch specifically for Southeast Asian languages and e-commerce. The paper argues that careful data curation—a 12T-token corpus emphasizing Indonesian, Thai, Vietnamese, Malay, Tagalog, and Portuguese plus hundreds of billions of e-commerce tokens—and a tokenizer optimized for SEA compression let a small model beat larger dense rivals on SEA and shopping tasks. It also introduces a hybrid reasoning design that gives both fast and deep thinking from one model, instead of deploying two separate systems. If the results hold, sub-30B models trained from scratch on region-specific data can outperform general-purpose models at a fraction of the inference cost.

What carries the argument

The engine of the design is a fine-grained Mixture-of-Experts (MoE) transformer in which every token routes through 2 shared experts and 4 specialized experts drawn from 48, giving 30B total parameters with only 5B active per token; a load-balancing auxiliary loss and a router Z-loss keep expert usage even during training. Supporting machinery includes a three-way sub-tokenizer merged into a 180k-vocabulary BPE tokenizer with the best reported compression on SEA languages, a three-stage pretraining schedule (8T tokens, 4T high-quality annealing, then extension from 4k to 32k context), and post-training with two SFT stages plus Online-DPO. The hybrid reasoning capability comes from long chain-of-thought data with thinking tags added in the second SFT stage, selectively downsampled to avoid degrading general skills.

What would settle it

Release the in-house evaluation prompts and answers, then run a near-duplicate search against the 12T-token pretraining corpus and the SFT and DPO data; finding substantial overlap would show the reported in-house scores measure memorization rather than capability. Alternatively, a rerun of the open-source benchmarks (Shopping MMLU, ECInstruct, and the general set) under identical generation settings without the custom tokenizer would test whether the advantage survives fair comparison.

Watch

Extended reading notes

Core claim

On its own terms, the paper reports that Compass-v2 achieves the highest average scores among comparable sub-30B models on open-source e-commerce benchmarks (55.74% average, beating the runner-up by a relative 10.6%) and on general English benchmarks (72.21% average), while activating only 5B parameters per token. It also reports in-house multilingual scores averaging 76.64, within about two points of GPT-4o and Qwen2.5-72b despite its smaller scale. The model unifies fast and deep reasoning through a single checkpoint: a long chain-of-thought prompt template triggers step-by-step thinking, and the authors report the hybrid training preserves general performance within a 1% drop. These claims rest on internal evaluation sets built from real Shopee scenarios, scored with GPT-4.

Load-bearing premise

The claim of state-of-the-art performance depends on internal evaluation datasets that are not publicly released and have not been checked for overlap with the training data; if those datasets leak into pretraining or fine-tuning, or if the GPT-4 judge is biased, the reported advantage may not be real.

Editorial extensions

If this is right

  • A region-focused MoE trained from scratch can match or beat general-purpose dense models several times its active size on low-resource language tasks.
  • One checkpoint can serve both lightweight chat and long step-by-step reasoning, removing the operational cost of maintaining two models.
  • 4-bit AWQ quantization preserves benchmark accuracy while delivering a 1.58x throughput gain over FP16 at high concurrency, making such models deployable on modest GPU fleets.
  • A tokenizer built by merging language-group sub-tokenizers improves compression for SEA languages, directly cutting per-token inference cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the tokenizer compression advantage generalizes, the same sub-tokenizer merging recipe could be applied to other low-resource language families (for example, South Asian or African languages) with similar efficiency gains.
  • The reported results suggest that for domain-specific low-resource deployments, investing in cleaned native corpora and a bespoke tokenizer may yield more value than starting from an English-centric checkpoint, though matched compute budgets would be needed to confirm this.
  • A direct test of the hybrid reasoning claim would be to measure whether the same checkpoint, with the general template active, matches a dedicated fast model on latency-sensitive e-commerce queries while the LongCoT template matches a dedicated reasoning model on math benchmarks.
  • Contamination is the key threat: because the in-house evaluation uses unreleased Shopee-derived prompts, publishing them and scanning the 12T-token corpus for near-duplicates would let the community verify the claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This technical report describes Compass-v2, a fine-grained Mixture-of-Experts (MoE) language model with 30B total and 5B active parameters, designed for Southeast Asian (SEA) languages and e-commerce applications. The authors present a three-stage pretraining pipeline on 12T tokens (generic pretraining, high-quality annealing, and long-context extension), a two-stage SFT on about 6M instructions, and an online-DPO alignment stage. They also introduce a hybrid reasoning design that supports both fast-thinking and deep-thinking responses within one model. Evaluation is reported on open multilingual, e-commerce, and general English benchmarks, and on in-house Shopee datasets scored by GPT-4. The abstract claims state-of-the-art SEA multilingual and e-commerce performance among sub-30B models with significantly lower inference cost.

Significance. If the reported results are accurate, Compass-v2 is a practically relevant contribution: a compact MoE trained from scratch for SEA languages and e-commerce, with a tokenizer that achieves strong compression on SEA languages (Figure 6) and a hybrid reasoning mode that reportedly preserves general performance within 1% (Section 5.3). The open-benchmark tables (Tables 6-8) provide some external grounding and show a broadly competitive model, and the data-quality comparison (Table 1) is a useful reference point. However, the absence of a public model, data, or evaluation harness, together with the reliance on a private GPT-4-scored evaluation for the strongest claims, limits the immediate impact of the work until those artifacts are released.

major comments (4)
  1. [Abstract, §1, §7.4, Table 8] The abstract and Section 1 claim 'state-of-the-art SEA multilingual and e-commerce performance among sub-30B models.' This is not fully supported by the paper's own open-benchmark tables. In Table 8, Compass-v2 scores 0.7801 on MMLU, below Qwen2.5-14b's 0.7991; 0.4091 on GPQA, below Qwen2.5-14b's 0.4394; 0.5913 on ARC, below Qwen2.5-14b's 0.6220; and 0.8150 on GSM8K, below Qwen2.5-7b's 0.8347. Since Qwen2.5-14b and Qwen2.5-7b are both sub-30B models, the claim either needs a clearly defined aggregation rule under which Compass-v2 is indeed the best, or a more modest phrasing such as 'competitive with' or 'among the best.'
  2. [§7.5, Table 9] The in-house evaluation is load-bearing for the claims of state-of-the-art SEA multilingual and e-commerce performance and for the statement that Compass-v2 is 'comparable' to GPT-4o and Qwen2.5-72b. However, this evaluation is not externally verifiable: the prompts are not released, no contamination analysis is performed against the SFT and DPO data described in Sections 3 and 4, and the judge is GPT-4, which is also one of the baselines in Table 9. Moreover, no error bars or statistical tests are reported for any of the tables. To support the central claims, the authors should release the evaluation prompts (or a substantial sample), provide a contamination check, describe the judge protocol in detail, and report variance across repeated evaluations.
  3. [Abstract, §5.1] The abstract claims that the authors 'pioneered a hybrid reasoning model that supports both fast thinking and deep thinking within a unified framework.' However, Section 5.1 explicitly cites Anthropic (2025) as a prior hybrid reasoning model of this kind. The word 'pioneered' is therefore inaccurate and should be replaced with a more measured term such as 'introduced' or 'presented.' The novelty of the hybrid reasoning design relative to Anthropic (2025) should also be stated explicitly.
  4. [§2.1.2, Table 1] The claim that the Compass-v2 (EN) dataset 'surpasses the best open-source datasets in the industry' rests on a single evaluation run with no reported variance. The difference between Compass-v2 (EN) (0.6397) and Combined (EN) (0.6366) is only 0.0031 in average score, which is within typical run-to-run noise for a 210B-token pretraining comparison. The paper should report multiple seeds, confidence intervals, or a significance test before making this strong claim.
minor comments (5)
  1. [Throughout] There are several typos: 'Comapss-v2' (Section 1), 'Quantizaiton' (Section 1), 'Qwem2.5-72b' (Section 1), 'wtih' (Section 7.4), 'pionneering' (Section 8.1), 'instend' (Section 4.1), and 'reuslt' (Section 3.1.1). Please copyedit the manuscript.
  2. [§2.2] The auxiliary loss formula for load balancing is typeset as Laux = N · Σ_i (p_i/B · c_i/B · K), which is dimensionally unclear. Please provide the correct mathematical expression or explicitly define p_i, c_i, B, and K so that the formula is unambiguous.
  3. [§2.3.3] The acronym 'ABF' is used without expansion. Please define it on first use (e.g., 'adjusted base frequency,' citing Xiong et al., 2023).
  4. [§7.1] The evaluation settings for baselines are incomplete; for example, Sailor2 and Moonlight are given as 'huggingface opensource generation config.' For reproducibility, consistent generation parameters should be listed for all baselines.
  5. [§10] The disclaimer in Section 10 is a legal boilerplate and is not a substitute for a scientific limitations statement. Please add a short limitations paragraph addressing the internal-evaluation constraints and the absence of public artifacts.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central performance claims rest on external open benchmarks and controlled comparisons, not on inputs by construction.

full rationale

The paper's central claims (SEA multilingual and e-commerce performance among sub-30B models at 5B active parameters) are supported primarily by Tables 6-8, which use external open benchmarks such as XCOPA, MMLU, HellaSwag, Shopping MMLU, ECInstruct, GSM8K, GPQA, and Belebele, with fixed generation settings. These are independent, externally grounded measurements rather than quantities derived from the model's own training objective. Table 1's dataset-quality comparison is a controlled experiment: 210B-token samples from each corpus are used to train a 7B model under identical settings, so the comparison is not a fit renamed as a prediction. The in-house Table 9 evaluation is self-reported and lacks released prompts, contamination analysis, and error bars, but that is a verification limitation, not a circular reduction: the model is trained and then measured on same-domain Shopee tasks, which is normal empirical evaluation rather than an equation forcing the outcome. Self-citations to the prior Compass report (Maria 2024) describe the predecessor pipeline and dataset scale, but none of the load-bearing performance claims reduces to those citations; the current data-quality comparison is independently executed. The hybrid-reasoning claim is an architectural and data choice, not a result derived from its own premises. No equation in the paper equates a predicted quantity to a fitted input, and no uniqueness or ansatz is imported by self-citation. Accordingly, no specific circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the quality and exclusivity of Shopee's internal data and on the fairness of internal evaluations; none of the resources are public, so these are assumptions rather than verified inputs. No physical or formal entities are introduced.

free parameters (4)
  • Pretraining data mixture proportions
    The paper states the corpus is 12T tokens with emphasis on SEA and e-commerce, but the exact mixture ratios per language and domain are not reported; the final model performance depends directly on them.
  • Auxiliary loss coefficient α and Z-loss coefficient β = max 0.01 and 0.001, decaying
    Chosen by hand; no ablation or formula determines them; the architecture's training stability claim depends on them.
  • Long-CoT data downsampling ratio = tuned to keep general metrics within a 1% drop
    Section 5.3 states the ratio is set to achieve a 1% drop in general metrics; this is a fitted target, not predicted.
  • Reasoning-data pass-rate thresholds [α, β]
    Section 5.2 uses an interval [α, β] to filter queries by pass rate; values are not given, making the Long-CoT dataset construction irreproducible.
assumptions (4)
  • domain assumption The 12T-token corpus was curated as described (language identification, deduplication, filtering, resampling).
    Section 2.1 asserts pipeline outcomes without releasing data; the model's multilingual ability is attributed to this corpus.
  • domain assumption GPT-4-as-judge scoring of in-house free-form answers is a valid and unbiased proxy for quality.
    Section 7 states GPT-4 is used as evaluator; no agreement analysis with human ratings is provided.
  • domain assumption Baselines are evaluated under fair, near-optimal settings (e.g., hyperparameters in Section 7.1).
    The comparison tables (Tables 6-9) rely on the listed generation settings being representative; some baselines use default configs.
  • domain assumption The open SEA benchmarks (XCOPA, MMLU, HellaSwag, OpenBookQA) measure meaningful multilingual capability in SEA languages.
    These benchmarks are English-centric or limited; the paper uses them as evidence of SEA expertise without a dedicated SEA-native benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Compass-V2 Technical Report." pith.science (2026). https://pith.science/paper/JM5JH5FN

@misc{pith2026250415527,
  author       = {Pith},
  title        = {Pith review of: Compass-V2 Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JM5JH5FN}},
  note         = {Machine review of arXiv:2504.15527}
}
read the original abstract

Predominant LLMs focus on high-resource languages while leaving low-resource languages, particularly those in Southeast Asia (SEA), underrepresented. In addition, those models are general-purpose and pay limited attention to the e-commerce domain. To overcome these limitations, we introduce Compass-v2, a lightweight Mixture-of-Experts (MoE) model specifically designed for Southeast Asian languages and e-commerce applications. To balance model performance and inference cost, the model is designed with 30B total parameters and 5B active parameters, incorporating both fine-grained and shared expert modules. To enhance multilingual performance, we curated and constructed a high-quality, industry-leading SEA dataset, to the best of our knowledge. To boost performance in the e-commerce domain, we built a dataset comprising hundreds of billions of tokens, sourced through external data mining and internal platform collection. Besides, we pioneered a hybrid reasoning model that supports both fast thinking and deep thinking within a unified framework to enhance the reasoning capabilities, diverging from the conventional industry practice of deploying two separate models. Through extensive experimental evaluations, our model demonstrates state-of-the-art SEA multilingual and e-commerce performance among sub-30B models, while maintaining significantly lower inference cost.

Figures

Figures reproduced from arXiv: 2504.15527 by the authors.

Figure 2
Figure 2. Ecommerce comparison of LLaMa3-8b, Qwen2.5-14b, and Compass-v2 1 arXiv:2504.15527v1 [cs.CL] 22 Apr 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_2.png] view at source ↗
Figure 3
Figure 3. Industry Overview of Pretraining Dataset Scale. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Pretraining Data Processing Overview [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figures from the paper (11 more)
Figure 6
Figure 6. Figure 6: Compression Ratio on different languages of Compass-v2. Compass-v2 tokenizer achieves [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Data volume of Compass-v2 dataset, versus previous generations of Compass training data [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Model Architecture of Compass-v2. Compass-v2 applies the fine-grained expert design [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: “Needle In A HayStack” Performance of Compass-v2. [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Overview of SFT data process pipeline The current SFT instructions encompass approximately 6 million high-quality samples, encom￾passing the necessary SEA languages and a variety of e-commerce tasks within the industry. Compass-v2 SFT dataset is sourced from three cha…
Figure 11
Figure 11. Figure 11: Overview of the multilingual dataset construction pipeline. [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Overview of the e-commerce dataset construction pipeline. [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: E-commerce Tasks Distribution. E-commerce Tasks Distribution The Figure13 illustrates the E-commerce data tasks distribution. The data is predominantly composed of three primary categories: Query-Product Matching, Product Questioning and Product Similarity Judgment, c…
Figure 14
Figure 14. Figure 14: Overview of Online-DPO Pipeline in Compass-v2. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Comparison of inference speed between Compass-v2 and Compass-v1. [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: We compare the average inference latency across models of similar [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 10 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Gpt-4 technical report

    OpenAI Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, and et al. Gpt-4 technical report. 2023. URL https://api.semanticscholar.org/CorpusID:257532815

    2023

  3. [3]

    GQA : Training generalized multi-query transformer models from multi-head checkpoints

    Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. GQA : Training generalized multi-query transformer models from multi-head checkpoints. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4895--4901, Singa...

    doi:10.18653/v1/2023.emnlp-main.298 2023

  4. [4]

    Sea-lion (southeast asian languages in one network): A family of large language models for southeast asia

    AISingapore. Sea-lion (southeast asian languages in one network): A family of large language models for southeast asia. https://github.com/aisingapore/sealion, 2024

    2024

  5. [5]

    Claude 3.7 sonnet and claude code

    Anthropic. Claude 3.7 sonnet and claude code. 2025. URL https://www.anthropic.com/news/claude-3-7-sonnet

    2025

  6. [6]

    Aya 23: Open weight releases to further multilingual progress

    Viraat Aryabumi, John Dang, Dwarak Talupuru, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Jon Ander Campos, Yi Chern Tan, et al. Aya 23: Open weight releases to further multilingual progress. arXiv preprint arXiv:2405.15032, 2024

    arXiv 2024

  7. [7]

    The belebele benchmark: a parallel reading comprehension dataset in 122 language variants

    Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa. The belebele benchmark: a parallel reading comprehension dataset in 122 language variants. arXiv preprint arXiv:2308.16884, 2023

    arXiv 2023

  8. [8]

    M 3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M 3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318--2335, Bangkok, Thailand, August 2024. Association for Computational Lin...

    doi:10.18653/v1/2024.findings-acl.137 2024

Show all 89 references
  1. [9]

    Benchmarking large language models in retrieval-augmented generation

    Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmented generation. arxiv abs/2309.01431 (2023), 2023 a

    2023 arXiv

  2. [10]

    MultilingualSIFT: Multilingual Supervised Instruction Fine-tuning , July 2023 b

    Zhihong Chen, Shuo Yan, Juhao Liang, Feng Jiang, Xiangbo Wu, Fei Yu, Guiming Hardy Chen, Junying Chen, Hongbo Zhang, Li Jianquan, Wan Xiang, and Benyou Wang. MultilingualSIFT: Multilingual Supervised Instruction Fine-tuning , July 2023 b . URL https://github.com/FreedomIntelli...

    2023

  3. [11]

    Brown, Miljan Martic, Shane Legg, and Dario Amodei

    Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences, 2023. URL https://arxiv.org/abs/1706.03741

    2023 arXiv

  4. [12]

    Boolq: Exploring the surprising difficulty of natural yes/no questions

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019

    1905 arXiv

  5. [13]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018

    2018 arXiv

  6. [14]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. 2021. URL https://arxiv.org/abs/2110.14168

    2021 arXiv

  7. [15]

    Commoncrawl, 2007

    CommonCrawl. Commoncrawl, 2007. URL https://commoncrawl.org

    2007

  8. [16]

    Unsupervised cross-lingual representation learning at scale

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. ArXiv, abs/1911.02116, 2019. URL https://a...

    1911 arXiv

  9. [17]

    Ultrafeedback: Boosting language models with scaled ai feedback, 2024

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. Ultrafeedback: Boosting language models with scaled ai feedback, 2024. URL https://arxiv.org/abs/2310.01377

    2024 arXiv

  10. [18]

    Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y. K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. Deepseekmoe: Towards ultimate expert specialization in mixture-of-ex...

    2024 arXiv

  11. [19]

    Dauphin, Angela Fan, Michael Auli, and David Grangier

    Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, page 933–941. JMLR.org, 2017

    2017

  12. [20]

    Deepseek-v3 technical report

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bing-Li Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dong-Li Ji, and et al. Deepseek-v3 technical report. ArXiv, abs/2412.19437, 2024. URL https://a...

    2024 arXiv

  13. [21]

    Rlhf workflow: From reward modeling to online rlhf, 2024

    Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. Rlhf workflow: From reward modeling to online rlhf, 2024. URL https://arxiv.org/abs/2405.07863

    2024 arXiv

  14. [22]

    Tran, Mike Zhang, Shiqi Chen, Tianyu Pang, Chao Du, Xinyi Wan, Wei Lu, and Min Lin

    Longxu Dou, Qian Liu, Fan Zhou, Changyu Chen, Zili Wang, Ziqi Jin, Zichen Liu, Tongyao Zhu, Cunxiao Du, Penghui Yang, Haonan Wang, Jiaheng Liu, Yongchi Zhao, Xiachong Feng, Xin Mao, Man Tsung Yeung, Kunat Pipatanakul, Fajri Koto, Min Si Thu, Hynek Kydl \' c ek, Zeyi Liu, Qunsh...

    2025 arXiv

  15. [23]

    Alan Schelten

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, and et al. Alan Schelten. The llama 3 herd of models. ArXiv, abs/2407.21783, 2024. URL https://api.semanticscholar.org/CorpusID:271571434

    2024 arXiv

  16. [24]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23 0 (120): 0 1--39, 2022

    2022

  17. [25]

    A framework for few-shot language model evaluation, 07 2024

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024

  18. [26]

    Gemma 3 technical report

    Gemma, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ram \'e , Morgane Rivi \`e re, et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025

    2025 arXiv

  19. [27]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  20. [28]

    Direct language model alignment from online ai feedback, 2024

    Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, and Mathieu Blondel. Direct language model alignment from online ai feedback, 2024. URL https://arxiv.org/abs/2402.04792

    2024 arXiv

  21. [29]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  22. [30]

    Amazon-m2: A multilingual multi-locale shopping session dataset for recommendation and text generation, 2023

    Wei Jin, Haitao Mao, Zheng Li, Haoming Jiang, Chen Luo, Hongzhi Wen, Haoyu Han, Hanqing Lu, Zhengyang Wang, Ruirui Li, Zhen Li, Monica Xiao Cheng, Rahul Goutam, Haiyang Zhang, Karthik Subbian, Suhang Wang, Yizhou Sun, Jiliang Tang, Bing Yin, and Xianfeng Tang. Amazon-m2: A mul...

    2023 arXiv

  23. [31]

    Shopping mmlu: A massive multi-task online shopping benchmark for large language models

    Yilun Jin, Zheng Li, Chenwei Zhang, Tianyu Cao, Yifan Gao, Pratik Jayarao, Mao Li, Xin Liu, Ritesh Sarkhel, Xianfeng Tang, et al. Shopping mmlu: A massive multi-task online shopping benchmark for large language models. arXiv preprint arXiv:2410.20745, 2024

    arXiv

  24. [32]

    T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension. In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational L...

    2017 doi

  25. [33]

    S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing

    Taku Kudo and John Richardson. S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Eduardo Blanco and Wei Lu, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: Syste...

    2018 doi

  26. [34]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention, 2023. URL https://arxiv.org/abs/2309.06180

    2023 arXiv

  27. [35]

    RACE : Large-scale R e A ding comprehension dataset from examinations

    Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. RACE : Large-scale R e A ding comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785--794, Copenhagen, Denmark, September 20...

    2017 doi

  28. [36]

    Chatharuhi: Reviving anime character in reality via large language model, 2023

    Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, and Haozhen Sun. Chatharuhi: Reviving anime character in reality via large language model, 2023. URL https://arxiv.org/ab...

    2023 arXiv

  29. [37]

    Dimakis, Yair Carmon, Achal Dave, Ludwig Schmidt, and Vaishaal Shankar

    Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan ...

    2024

  30. [38]

    Reinhard Heckel

    Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Yitzhak Gadre, Hritik Bansal, Etash Kumar Guha, Sedrick Scott Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, and et al. Reinhard Heckel. Datacomp-lm: In search of the next generation of train...

    2024 arXiv

  31. [39]

    Numinamath

    Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath...

    2024

  32. [40]

    Numinamath

    Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath...

    2024

  33. [41]

    Awq: Activation-aware weight quantization for llm compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv, 2023

    2023

  34. [42]

    Few-shot learning with multilingual language models

    Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, et al. Few-shot learning with multilingual language models. arXiv preprint arXiv:2112.10668, 2021

    2021 arXiv

  35. [43]

    Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model

    Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434, 2024

    2024 arXiv

  36. [44]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  37. [45]

    Compass: Large multilingual language model for south-east asia

    Sophia Maria. Compass: Large multilingual language model for south-east asia. ArXiv, abs/2404.09220, 2024. URL https://api.semanticscholar.org/CorpusID:269149349

    2024 arXiv

  38. [46]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018

    2018 arXiv

  39. [47]

    Crosslingual generalization through multitask finetuning

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning. In The 61st Annual Meeting Of The Association For Com...

    2023

  40. [48]

    s1: Simple test-time scaling, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025. URL https://arxiv.org/abs/2501.19393

    2025 arXiv

  41. [49]

    MS MARCO: A human generated machine reading comprehension dataset

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. MS MARCO: A human generated machine reading comprehension dataset. CoRR, abs/1611.09268, 2016. URL http://arxiv.org/abs/1611.09268

    2016 arXiv

  42. [50]

    Seallms - large language models for southeast asia

    Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, and Li Bing. Seallms - large language models for southeast asia. ArXiv, abs/2312.00738, 2023. URL https://api.semanticscholar.org/Cor...

    2023 arXiv

  43. [51]

    Murray, Crystal Nam, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Chris Wilhelm, Michael Wilson, Luke S

    Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Mi...

    2024 arXiv

  44. [52]

    Learning to reason with llms

    OpenAI. Learning to reason with llms. 2024. URL https://openai.com/index/learning-to-reason-with-llms/

    2024

  45. [53]

    Shyamal Anadkat

    OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and et al. Shyamal Anadkat. Gpt-4 technical report, 2024. URL https://arxiv.org/abs/2303.08774

    2024 arXiv

  46. [54]

    The fineweb datasets: Decanting the web for the finest text data at scale

    Guilherme Penedo, Hynek Kydl\' c ek, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. ...

    2024

  47. [55]

    ece LLM : Generalizing large language models for e-commerce from large-scale, high-quality instruction data

    Bo Peng, Xinyi Ling, Ziru Chen, Huan Sun, and Xia Ning. ece LLM : Generalizing large language models for e-commerce from large-scale, high-quality instruction data. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=LWRI4uPG2X

    2024

  48. [56]

    Xcopa: A multilingual dataset for causal commonsense reasoning

    Edoardo Maria Ponti, Goran Glava s , Olga Majewska, Qianchu Liu, Ivan Vuli \'c , and Anna Korhonen. Xcopa: A multilingual dataset for causal commonsense reasoning. arXiv preprint arXiv:2005.00333, 2020

    2005 arXiv

  49. [57]

    Qwq-32b: Embracing the power of reinforcement learning

    Qwen. Qwq-32b: Embracing the power of reinforcement learning. 2025. URL https://qwenlm.github.io/blog/qwq-32b/

    2025

  50. [58]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model, 2024. URL https://arxiv.org/abs/2305.18290

    2024 arXiv

  51. [59]

    Squad: 100,000+ questions for machine comprehension of text

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250, 2016

    2016 arXiv

  52. [60]

    Siva Reddy, Danqi Chen, and Christopher D. Manning. C o QA : A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7: 0 249--266, 2019. doi:10.1162/tacl_a_00266. URL https://aclanthology.org/Q19-1016/

    2019 doi

  53. [61]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024

    2024

  54. [62]

    Neural machine translation of rare words with subword units

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Katrin Erk and Noah A. Smith, editors, Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715-...

    2016 doi

  55. [63]

    Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V

    Noam M. Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. ArXiv, abs/1701.06538, 2017. URL https://api.semanticscholar.org/CorpusID:12462234

    2017 arXiv

  56. [64]

    Nemotron-cc: Transforming common crawl into a refined long-horizon pretraining dataset

    Dan Su, Kezhi Kong, Ying Lin, Joseph Jennings, Brandon Norick, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Nemotron-cc: Transforming common crawl into a refined long-horizon pretraining dataset. ArXiv, abs/2412.02595, 2024 a . URL https://api.semanti...

    2024 arXiv

  57. [65]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomput., 568 0 (C), February 2024 b . ISSN 0925-2312. doi:10.1016/j.neucom.2023.127063. URL https://doi.org/10.1016/j.neucom.2023.127063

    2024

  58. [66]

    Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent

    Xingwu Sun, Yanfeng Chen, Yiqing Huang, Ruobing Xie, Jiaqi Zhu, Kai Zhang, Shuaipeng Li, Zhen Yang, Jonny Han, Xiaobo Shu, et al. Hunyuan-large: An open-source moe model with 52 billion activated parameters by tencent. arXiv preprint arXiv:2411.02265, 2024

    2024 arXiv

  59. [67]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https://qwenlm.github.io/blog/qwen2.5/

    2024

  60. [68]

    N ews QA : A machine comprehension dataset

    Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. N ews QA : A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP , pages 191--200, Vancouver, Canada, August 2017. As...

    2017 doi

  61. [69]

    Multilingual language model pretraining using machine-translated data

    Jiayi Wang, Yao Lu, Maurice Weber, Max Ryabinin, David Adelani, Yihong Chen, Raphael Tang, and Pontus Stenetorp. Multilingual language model pretraining using machine-translated data. arXiv preprint arXiv:2502.13252, 2025 a

    2025 arXiv

  62. [70]

    Coser: Coordinating llm-based persona simulation of established roles

    Xintao Wang, Heng Wang, Yifei Zhang, Xinfeng Yuan, Rui Xu, Jen-tse Huang, Siyu Yuan, Haoran Guo, Jiangjie Chen, Wei Wang, et al. Coser: Coordinating llm-based persona simulation of established roles. arXiv preprint arXiv:2502.09082, 2025 b

    2025

  63. [71]

    Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint, 2024

    Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint, 2024. URL https://arxiv.org/abs/2312.11456

    2024 arXiv

  64. [72]

    Effective long-context scaling of foundation models

    Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models. arXiv preprint arXiv:2309.16039, 2023

    2023 arXiv

  65. [73]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 arXiv

  66. [74]

    Bowen Yu

    Qwen An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, and et al. Bowen Yu. Qwen2.5 technical report. ArXiv, abs/2412.15115, 2024 b . URL https://api.semanticscholar.org/CorpusID:274859421

    2024 arXiv

  67. [75]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. H otpot QA : A dataset for diverse, explainable multi-hop question answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun ' ichi Tsujii, editors,...

    2018 doi

  68. [76]

    Limo: Less is more for reasoning, 2025

    Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025. URL https://arxiv.org/abs/2502.03387

    2025 arXiv

  69. [77]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  70. [78]

    Root mean square layer normalization

    Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019

    2019

  71. [79]

    Chinese open instruction generalist: A preliminary release, 2023

    Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, and Jie Fu. Chinese open instruction generalist: A preliminary release, 2023

    2023

  72. [80]

    Map-neo: Highly capable and transparent bilingual large language model series

    Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Esther Cheng, Jie Liu, Qunshu Lin, Raven Yuan, Tuney Zheng, Wei Pang, Xinrun Du, Yiming Liang, Yi Ma, Yizhi Li, Ziyang Ma, Bill Lin, Emmanouil Benetos, Huan Yang, Junting Zhou, Kaijing Ma...

    2024 arXiv

  73. [81]

    Gender bias in coreference resolution: Evaluation and debiasing methods

    Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. arXiv preprint arXiv:1804.06876, 2018

    2018 arXiv

  74. [82]

    Trade-offs in large reasoning models: An empirical analysis of deliberative and adaptive reasoning over foundational capabilities

    Weixiang Zhao, Xingyu Sui, Jiahe Guo, Yulin Hu, Yang Deng, Yanyan Zhao, Bing Qin, Wanxiang Che, Tat-Seng Chua, and Ting Liu. Trade-offs in large reasoning models: An empirical analysis of deliberative and adaptive reasoning over foundational capabilities. arXiv preprint arXiv:...

    2025

  75. [83]

    Pytorch fsdp: experiences on scaling fully sharded data parallel

    Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, et al. Pytorch fsdp: experiences on scaling fully sharded data parallel. arXiv preprint arXiv:2304.11277, 2023

    2023 arXiv

  76. [84]

    St-moe: Designing stable and transferable sparse expert models

    Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. St-moe: Designing stable and transferable sparse expert models. arXiv preprint arXiv:2202.08906, 2022

    2022 arXiv

  77. [85]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  78. [86]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  79. [87]

    Please summarize the following passage in Chinese, using a formal tone, no more than 50 words, and ensuring that the keyword ‘artificial intelligence’ is included

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

  80. [88]

    , " * write output.state after.block = add.period write

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...

  81. [89]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.