Pith. sign in

REVIEW 4 major objections 5 minor 63 references

Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a pass-rate-driven curriculum in reinforcement learning lifts abnormal-event detection F1 by an average of 17.19% and out-of-domain transfer by 9.59%.

desk verdict Plausible industrial F1 gains for an RLVR post-training recipe, but the paper's central claim about its adaptive sampler is unsupported as written: the headline ablation changes the KL coefficient and the sampler at once, and the 'perplexity' in the title is never computed. read the letter →

arxiv 2507.01327 v1 pith:ZA7VTBUF submitted 2025-07-02 cs.LG cs.AI

Xiaoyun Zhang , Jingqing Ruan , Xing Ma , Yawen Zhu , Jiansong Chen , Ke Zeng , Xunliang Cai This is my paper
classification cs.LGcs.AI
keywords abnormaleventdetectionreinforcementlearninglargelanguagemodelscurriculumperplexity-awaresamplingout-of-domaingeneralizationcustomerservicedialoguerule-basedrewards
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that abnormal-event detection in customer-service dialogues can be made substantially better by training a large language model with a two-loop reinforcement-learning procedure that adjusts which examples the model practices on as the model improves. The outer loop measures each sample's current difficulty by how often the model answers it correctly in a few quick tries, then makes the training batch favor samples near the model's current average success rate. The inner loop is a rule-based RL update that rewards correct answers and well-formatted reasoning output without requiring human-written chain-of-thought labels. On an industrial food-delivery dialogue benchmark, the authors report the best F1 scores, with an average improvement of 17.19% over baselines and an average improvement of 9.59% in out-of-domain transfer tests across three business scenarios. If these results hold, the recipe offers a practical way to turn general reasoning models into domain-specific detectors under label and annotation constraints.

What carries the argument

The load-bearing object is the adaptive perplexity-aware sampling probability in Eq. (2), a power function of the ratio between each example's empirical success rate $p_i$ and the batch proficiency $\mu_p$. Its peak sits at $p_i=\mu_p$, so it assigns the highest selection probability to samples whose difficulty just matches the model's current ability, and it zeroes out samples the model always gets right or always gets wrong. That filter is what turns a fixed training set into an evolving curriculum; the inner loop, a rule-rewarded policy update with a stability term that penalizes movement away from the previous policy, supplies the learning signal. The claimed effect is that the model consolidates easy and moderate skills early, then spends its sampling budget on borderline cases, which is what the authors say removes the plateau and improves transfer.

What would settle it

Run APARL with two different estimation budgets (for example, $k=1$ versus $k=16$) and vary the sharpness parameter $t$ from 0.05 to 0.5: if in-domain F1 and out-of-domain F1 move by less than a point, the curriculum is robust, while swings of several points would show the reported gains are fragile. A second decisive experiment is to train with the sampling filter removed but the same stability-regularized update; if that matches APARL's F1, the outer loop is not the cause of the gains.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a dynamic curriculum defined by the model's empirical per-sample success rate, rather than by static features of the data, removes the two failure modes that block RL training on customer-service tasks: early accuracy plateaus and a bimodal split in which easy samples get mastered while hard samples remain unsolved. At each step the method draws $k$ Monte Carlo responses per prompt, sets $p_i$ to the fraction of correct responses, sets $\mu_p$ to the batch average of the $p_i$, and samples training examples with probability $P(x_i)=(p_i/(\mu_p+\epsilon))^{t\mu_p}((1-p_i)/(1-\mu_p+\epsilon))^{t(1-\mu_p)}$. This concentrates selection on samples near the model's current proficiency, filters out samples with $p_i=0$ or $p_i=1$, and shifts toward harder examples as $\mu_p$ grows. Applied to Qwen-14B-Instruct and DeepSeek-R1-Distill-Qwen-14B, the full system reaches F1 of 83.38% and 82.28% on the in-domain test set and 78.65% and 79.01% averaged over three out-of-domain tasks, surpassing API baselines, small-model classifiers, supervised fine-tuning, and the base RL updates.

Load-bearing premise

The whole gain rests on assuming that a sample's difficulty, measured by how often the model answers it correctly in a few quick tries, is stable enough to steer training, and that this steering, rather than the extra stability term in the learning update, is what lifts performance.

Editorial extensions

If this is right

  • Abnormal-event detection can be delivered by a single reasoning language model rather than by many task-specific classification heads, cutting annotation and maintenance overhead per event type.
  • The same model trained once transfers to new business domains with only a small drop in F1 (roughly four to five points), so rapid deployment to a new scenario may not require a new dataset.
  • Because the curriculum is driven by the model's own pass rates, it applies without chain-of-thought annotations, keeping the reward signal rule-based and cheap to implement.
  • On out-of-domain tasks, the method's average F1 of about 79% exceeds strong general-purpose API models by several points, suggesting RL post-training can beat prompting on vertical classification tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the gains are genuinely from pass-rate-based reweighting, the same outer loop should transfer to any verifiable categorical reward task, such as named-entity recognition or relation extraction, not just customer-service event detection.
  • A cheaper variant would replace the $k$ Monte Carlo generations per sample with confidence scores from a single forward pass; the paper does not test this, and showing the two curricula behave similarly would indicate the difficulty signal, not the sampling procedure, is what matters.
  • The fixed sharpness parameter $t=0.1$ and the unreported value of $k$ leave an open sensitivity analysis: if the method is robust across a wide range of these settings, it is a practical recipe, while narrow robustness would make deployment require per-task tuning.
  • The ablations compare APARL with the base RL updates but do not separately isolate the sampling filter from the added stability term, so part of the reported gain could come from the inner-loop regularization rather than the curriculum.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes APARL, a dual-loop reinforcement learning framework for abnormal event detection in customer service dialogues. The outer loop adaptively samples training mini-batches via a 'perplexity-aware' probability computed from per-sample pass rates, and the inner loop uses a DAPO-based rule-guided RL objective with a KL regularizer. Experiments on a Meituan food-delivery dataset (55k training samples, 20 event types) report large F1 gains over baselines, including OOD transfer to three anonymized business domains. The claimed headline results are that APARL reaches 83.38% F1 on the in-domain test set with Qwen-14B-Instruct and 78.65% F1 on OOD domains, with average improvements of 17.19% and 9.59% over all baselines.

Significance. If the reported gains are robust and correctly attributed, the paper offers a practical recipe for applying reasoning-focused RL to a real industrial text-classification task, with a plausible curriculum mechanism that could transfer to other vertical domains. Strengths include the use of a large real-world dataset, the dual-loop architecture idea, the inclusion of detailed per-domain OOD results in Table 5, and a stated intention to release code. However, the central claims are currently not well supported because the sampling mechanism is not cleanly isolated from the inner-loop KL change, the key hyperparameter k is missing, and no variance estimates are provided. The contribution is therefore conditional on additional ablations and reporting.

major comments (4)
  1. [§3.2, Eq (1)-(2)] The method is introduced as 'Adaptive Perplexity-Aware Sampling,' but Eq (2) defines P(x_i) purely as a power function of the empirical success rate p_i and the batch average proficiency μ_p. No perplexity, likelihood, or entropy term appears anywhere in the sampling criterion. This mismatch between the claimed mechanism and the actual equation is load-bearing: it affects how readers interpret the cause of any observed gains. Please either rename the method to 'success-rate-aware' or provide an explicit perplexity-based computation and justify its role.
  2. [§3.3, Tables 2 and 4] The ablation that is supposed to demonstrate the value of the outer loop actually changes two variables at once. DAPO is trained with kl=0.0 while APARL uses kl=0.001 and adds the outer sampler. Section 3.3 explicitly states that omitting the KL constraint causes 'significant instability,' meaning the KL term is a known performance contributor. Without a control run of DAPO+KL(0.001) that keeps the outer sampling off, the F1 increments (78.91→82.28 for DeepSeek-R1-Distill-Qwen-14B and 78.48→83.38 for Qwen-14B-Instruct) cannot be attributed to the adaptive sampling. Please run this missing ablation.
  3. [§3.2, Eq (2); §B.2, Table 4] The number of Monte Carlo generations per sample, k, is never reported. This is not a cosmetic omission: for k=1, p_i is binary and Eq (2) assigns P(x_i)=0 to every sample (either the first factor or the second factor is zero), which would make the sampling filter select an empty batch. For small k, the support of p_i is coarse and its variance is high, so the stability of the estimated difficulty is questionable. Please report k, provide a sensitivity analysis for t (including t=0, which reduces Eq (2) to uniform sampling), and analyze the variance of p_i to show that the curriculum, rather than the inner-loop KL term, drives the reported improvements.
  4. [Tables 1, 2, 3, 5] All experimental results are reported as single point estimates with no error bars or run-to-run variance. Given that the key gains over DAPO are 4.89 and 7.93 F1 points, and RL training is stochastic, the absence of variance information makes it impossible to assess whether the observed differences are meaningful. Please report mean±std over at least three independent training runs for the main comparisons, or explicitly justify why single-seed reporting is reliable for this industrial setup.
minor comments (5)
  1. [§3.3, Eq (6) and §B.2, Table 4] The reward constants Ca and Cf appear in Eq (6) but are not listed in the hyperparameter table; please specify their numerical values.
  2. [§4.2 and Table 1] The claim of a 17.19% 'average improvement over all the baselines' mixes very different baselines (API models, small models, SFT, RL). Please clarify the exact set of baselines and the formula used for the average, since the interpretation differs if the average is computed over F1 differences or over relative improvements.
  3. [§4.4 and Table 3] The OOD claim would be stronger if the three anonymized domains were shown to differ from the training distribution in a measurable way (e.g., in label distribution or language style). Please add a description of how these domains were selected and what makes them out-of-distribution.
  4. [Ethics Statement] The ethics statement says 'All data and materials utilized are suitable for public release,' but the dataset is described as sourced from Meituan's online logs and the OOD tasks are anonymized for business confidentiality. This appears contradictory; please clarify whether the released code includes only scripts or also data, and how confidentiality is maintained.
  5. [Throughout] There are several typos and inconsistencies: 'controlßling' in §3.2, 'we explores' in §3.1, 'ASRR' instead of 'APARL' in the Figure 6 caption, and duplicated references for DeepSeek-R1 (Guo et al. 2025a/2025b) and DeepSeek-Math (Shao et al. 2024a/2024b). Please correct these.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: APARL's headline F1 gains are measured on external held-out and OOD test sets, and the only overlapping-author citation is a non-load-bearing related-work reference.

full rationale

The paper's derivation chain does not reduce its predictions to its inputs. The outer-loop sampling probability in Eq. (2) is a function of the current policy's empirical pass rates pi and batch proficiency mu_p, but the paper's central claims (Tables 1 and 3) are F1 scores on a 9,000-sample held-out test set and on three anonymized OOD domains; those metrics are not constructed from Eq. (2) and are not fitted quantities. The KL-regularized DAPO update in Eq. (3) is borrowed from DAPO and DeepSeek-Math with explicit citations, and no load-bearing uniqueness theorem or fitted parameter is imported from the authors' own prior work. The only overlapping-author citation is CodeV-R1 (Zhu et al., 2025) in the Related Work survey of vertical-domain RL; it is used as an example, not as justification for APARL's design. The absence of an ablation that isolates the outer sampler from the KL term is an attribution weakness, and the 'perplexity-aware' name is a misnomer for pass-rate-based sampling, but neither makes the evaluation circular: the reported F1 gains could be wrong or confounded without being equivalent to the method's inputs by construction. The Limitations section's acknowledged narrow architecture testing is also a scope limitation, not a circularity. Therefore no circular steps are identified; the score reflects only the minor non-load-bearing self-citation.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claim rests on one hand-tuned sampling sharpness t, an unreported Monte Carlo count k, and unreported reward constants Ca and Cf. The main axioms are reliability of pass-rate estimates as difficulty, exclusion of pi=0 or 1 samples, correctness of the 20-event annotation, representativeness of the three OOD domains, and standard RL background. The paper introduces no invented entities, particles, forces, or latent dimensions; the dual-loop is a training architecture, not a new physical or model entity.

free parameters (3)
  • t (sampling sharpness) = 0.1
    Hand-set in Table 4; controls the concentration of the sampling distribution in Eq (2). No sensitivity analysis is reported, yet it directly shapes the curriculum.
  • k (Monte Carlo generations per sample)
    Used in Eq (1) to estimate the empirical pass rate pi. Its value is never reported, and the stability of the difficulty signal depends on it.
  • Ca and Cf (reward constants)
    In Eq (6), reward magnitudes are initialized as negative constants and switched to positive values; the exact values are not given, which affects advantage normalization and final performance.
assumptions (6)
  • ad hoc to paper The empirical pass rate pi from k rollouts is a reliable and stable measure of sample difficulty and model proficiency.
    Section 3.2, Eq (1). The entire outer-loop curriculum rests on this proxy; the paper provides no analysis of its variance or of the choice of k.
  • ad hoc to paper Samples with pi=0 or pi=1 carry no useful gradient signal and should be excluded.
    Section 3.2, following DAPO; the claim is borrowed and not re-verified for this domain.
  • domain assumption The 20 predefined abnormal event categories and human annotations in the Meituan dataset are correct, complete, and consistent.
    Section 2 and Section 4.1; the reward function in Eq (6) treats labels as ground truth.
  • domain assumption The three anonymized OOD domains are representative of cross-business distribution shift.
    Section 4.4 and Appendix C; no details of label distributions or category overlap are given.
  • standard math Standard RL and GRPO background assumptions about policy gradients, KL penalty, and advantage normalization hold for 14B models.
    Section 3.3, Eq (3)-(5); the paper inherits convergence and stability properties from DAPO and DeepSeek-Math without proving them.
  • ad hoc to paper The power-function sampling distribution peaked at mu_p causes the model to focus on appropriately challenging samples and improves final performance.
    Section 3.2; the core design claim is asserted without formal analysis and without a sensitivity study on t.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy." pith.science (2026). https://pith.science/paper/ZA7VTBUF

@misc{pith2026250701327,
  author       = {Pith},
  title        = {Pith review of: Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZA7VTBUF}},
  note         = {Machine review of arXiv:2507.01327}
}
read the original abstract

Detecting abnormal events in real-world customer service dialogues is highly challenging due to the complexity of business data and the dynamic nature of customer interactions. Moreover, models must demonstrate strong out-of-domain (OOD) generalization to enable rapid adaptation across different business scenarios and maximize commercial value. In this work, we propose a novel Adaptive Perplexity-Aware Reinforcement Learning (APARL) framework that leverages the advanced reasoning capabilities of large language models for abnormal event detection. APARL introduces a dual-loop dynamic curriculum learning architecture, enabling the model to progressively focus on more challenging samples as its proficiency increases. This design effectively addresses performance bottlenecks and significantly enhances OOD transferability. Extensive evaluations on food delivery dialogue tasks show that our model achieves significantly enhanced adaptability and robustness, attaining the highest F1 score with an average improvement of 17.19\%, and an average improvement of 9.59\% in OOD transfer tests. This method provides a superior solution for industrial deployment of anomaly detection models, contributing to improved operational efficiency and commercial benefits.

Figures

Figures reproduced from arXiv: 2507.01327 by the authors.

Figure 1
Figure 1. (a) Training Plateau effect on accuracy. (b) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Abnormal event detection in the dialogue. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. (a) SFT, (b) traditional RL, and (c) overview of our proposed APARL: integrating rule-guided RL in the [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: highlight the effectiveness of our adaptive strategy. In the left panel, the reward trajectory of the adaptive strategy exhibits a more rapid upward trajectory and ultimately converges to a higher re￾ward region. This characteristic stems from the dy￾namic adjustment o…
Figure 5
Figure 5. Figure 5: The prompt used for abnormal event detection in food delivery tasks. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The output generated after ASRR reinforcement learning. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 19 canonical work pages

  1. [1]

    Arash Ahmadian, Chris Cremer, Matthias Gall \'e , Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet \"U st \"u n, and Sara Hooker. 2024. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms. arXiv preprint arXiv:2402.14740

    arXiv 2024

  2. [2]

    Aitor Arrieta, Miriam Ugarte, Pablo Valle, Jos \'e Antonio Parejo, and Sergio Segura. 2025. o3-mini vs deepseek-r1: Which one is safer? arXiv preprint arXiv:2501.18438

    arXiv 2025

  3. [3]

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  4. [4]

    Laurence A Baxter. 1995. Markov decision processes: Discrete stochastic dynamic programming

    work page 1995

  5. [5]

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216

    arXiv 2024

  6. [6]

    Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, and 1 others. 2025. Rm-r1: Reward modeling as reasoning. arXiv preprint arXiv:2505.02387

    arXiv 2025

  7. [7]

    Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161

    arXiv 2025

  8. [8]

    Tao Dai, Yidan Lu, Nikola Zivlak, Danijela \'C iri \'c Lali \'c , and Bojan Lalic. 2024. How does queueing information in pre-sales call centers affect customer repurchase behavior. Asia-Pacific Journal of Operational Research, (41 (4)):26--p

    work page 2024

Show all 63 references
  1. [9]

    Google DeepMind. 2024. Gemini 2.0 flash thinking. URL https://deepmind.google/technologies/gemini/flash-thinking/

    2024

  2. [10]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  3. [11]

    Evgenii Evstafev. 2025. Token-hungry, yet precise: Deepseek r1 highlights the need for multi-step reasoning over speed in math. arXiv preprint arXiv:2501.18576

    2025 arXiv

  4. [12]

    Chengliang Gao, Fan Zhang, Guanqun Wu, Qiwan Hu, Qiang Ru, Jinghua Hao, Renqing He, and Zhizhao Sun. 2021. A deep learning method for route and time prediction in food delivery service. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, page...

    2021

  5. [14]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025 b . Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  6. [15]

    Ronald A Howard. 1960. Dynamic programming and markov processes

    1960

  7. [16]

    Jian Hu. 2025. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262

    2025 arXiv

  8. [17]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. arXiv preprint arXiv:2412.16720

    2024 arXiv

  9. [18]

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

    2020 arXiv

  10. [19]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating S...

    2023

  11. [20]

    Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. 2025. Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models. arXiv preprint arXiv:2503.13939

    2025

  12. [21]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023 a . Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281

    2023 arXiv

  13. [22]

    Ziniu Li, Tian Xu, Yushun Zhang, Zhihang Lin, Yang Yu, Ruoyu Sun, and Zhi-Quan Luo. 2023 b . Remax: A simple, effective, and efficient reinforcement learning method for aligning large language models. arXiv preprint arXiv:2310.10505

    2023 arXiv

  14. [23]

    Zhaowei Liu, Xin Guo, Fangqi Lou, Lingfeng Zeng, Jinyi Niu, Zixuan Wang, Jiajie Xu, Weige Cai, Ziwei Yang, Xueqian Zhao, and 1 others. 2025 a . Fin-r1: A large language model for financial reasoning through reinforcement learning. arXiv preprint arXiv:2503.16252

    2025

  15. [24]

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025 b . Understanding r1-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783

    2025 arXiv

  16. [25]

    Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 37:124198--124235

    2024

  17. [26]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837

    2022 arXiv

  18. [27]

    Gianluca Mondillo, Simone Colosimo, Alessandra Perrotta, Vittoria Frattolillo, and Mariapia Masino. 2025. Comparative evaluation of advanced ai reasoning models in pediatric clinical decision support: Chatgpt o1 vs. deepseek-r1. medRxiv, pages 2025--01

    2025

  19. [28]

    Dishita Naik, Ishita Naik, and Nitin Naik. 2024. Leveraging the use of chatgpt: exploring its real-world applications including their related ethical and regulatory considerations. In The International Conference on Computing, Communication, Cybersecurity & AI, pages 649--667....

    2024

  20. [29]

    Eric WT Ngai, Maggie CM Lee, Mei Luo, Patrick SL Chan, and Tenglu Liang. 2021. An intelligent knowledge-based chatbot for customer service. Electronic Commerce Research and Applications, 50:101098

    2021

  21. [30]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2023 arXiv

  22. [31]

    OpenAI. 2024. https://openai.com/index/learning-to-reason-with-llms Learning to reason with language models

    2024

  23. [32]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Sys...

    2022

  24. [33]

    Sumanth Prabhu, Moosa Mohamed, and Hemant Misra. 2021. Multi-class text classification using bert-based active learning. arXiv preprint arXiv:2104.14289

    2021 arXiv

  25. [34]

    Rukhma Qasim, Waqas Haider Bangyal, Mohammed A Alqarni, and Abdulwahab Ali Almazroi. 2022. A fine-tuned bert-based transfer learning approach for text classification. Journal of healthcare engineering, 2022(1):3498123

    2022

  26. [35]

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. https://arxiv.org/abs/2412.15115 Qwe...

    2025 arXiv

  27. [36]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728--53741

    2023

  28. [37]

    Anitha Ravikumar, Donalie H Cabral, Sheikha Said AlSubhi, and Gopalan Puthukulam. 2025. A study on the application of ai chatbots for customized product recommendations and uninterrupted order fulfillment for enhanced customer experience using kano model. In Innovative and Int...

    2025

  29. [38]

    Tom Schaul, Diana Borsa, Joseph Modayil, and Razvan Pascanu. 2019. Ray interference: a source of plateaus in deep reinforcement learning. arXiv preprint arXiv:1904.11455

    2019 arXiv

  30. [39]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  31. [41]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024 b . Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300

    2024 arXiv

  32. [42]

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256

    2024 arXiv

  33. [43]

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053

    2019 arXiv

  34. [44]

    P Slade and Tam \'a s D Gedeon. 1993. Bimodal distribution removal. In International Workshop on Artificial Neural Networks, pages 249--254. Springer

    1993

  35. [45]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314

    2024 arXiv

  36. [46]

    Jingyuan Sun and Marie-Francine Moens. 2023. Fine-tuned vs. prompt-tuned supervised representations: Which better account for brain language representations? arXiv preprint arXiv:2310.01854

    2023 arXiv

  37. [47]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599

    2025 arXiv

  38. [48]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  39. [49]

    Naoki Wake, Atsushi Kanehira, Kazuhiro Sasabuchi, Jun Takamatsu, and Katsushi Ikeuchi. 2023. Chatgpt empowered long-step robot control in various environments: A case application. arXiv preprint arXiv:2304.03893

    2023 arXiv

  40. [50]

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652

    2021 arXiv

  41. [51]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022 a . Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  42. [52]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022 b . Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  43. [53]

    Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. https://arxiv.org/abs/2502.14768 Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning . Preprint, arXiv:2502.14768

    2025 arXiv

  44. [54]

    Jingfeng Yang, Hoong Chuin Lau, and Hai Wang. 2024. Optimization of customer service and driver dispatch areas for on-demand food delivery. Transportation Research Part C: Emerging Technologies, 165:104653

    2024

  45. [55]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809--11822

    2023

  46. [56]

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476

    2025 arXiv

  47. [57]

    Shanshan Yu, Jindian Su, and Da Luo. 2019. Improving bert-based text classification with auxiliary sentence and domain knowledge. IEEE Access, 7:176600--176612

    2019

  48. [58]

    Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, and 1 others. 2025. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint arXiv:2504.05118

    2025 arXiv

  49. [59]

    Chong Zhang, Yue Deng, Xiang Lin, Bin Wang, Dianwen Ng, Hai Ye, Xingxuan Li, Yao Xiao, Zhanfeng Mo, Qi Zhang, and 1 others. 2025. 100 days after deepseek-r1: A survey on replication studies and more directions for reasoning language models. arXiv preprint arXiv:2505.00551

    2025 arXiv

  50. [60]

    Jixiao Zhang and Chunsheng Zuo. 2025. Grpo-lead: A difficulty-aware reinforcement learning approach for concise mathematical reasoning in language models. arXiv preprint arXiv:2504.09696

    2025

  51. [61]

    Denny Zhou, Nathanael Sch \"a rli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and 1 others. 2022. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625

    2022 arXiv

  52. [62]

    Yaoyu Zhu, Di Huang, Hanqi Lyu, Xiaoyun Zhang, Chongxiao Li, Wenxuan Shi, Yutong Wu, Jianan Mu, Jinghua Wang, Yang Zhao, and 1 others. 2025. Codev-r1: Reasoning-enhanced verilog generation. arXiv preprint arXiv:2505.24183

    2025

  53. [63]

    Yicheng Zou, Lujun Zhao, Yangyang Kang, Jun Lin, Minlong Peng, Zhuoren Jiang, Changlong Sun, Qi Zhang, Xuanjing Huang, and Xiaozhong Liu. 2021. Topic-oriented spoken dialogue summarization for customer service with saliency-aware topic modeling. In Proceedings of the AAAI Conf...

    2021

  54. [64]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  55. [65]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.