Pith. sign in

REVIEW 4 major objections 5 minor 34 references

A Novel Self-Evolution Framework for Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A dual-phase self-evolution framework lets an LLM keep improving itself from interaction signals, beating SFT, preference-optimization, and memory-augmented baselines on general and long-term dialogue benchmarks.

desk verdict DPSE is a plausible self-evolution framework whose central empirical claim is undermined by a train/test leak on LoCoMo and an unvalidated satisfaction scorer. read the letter →

arxiv 2507.15281 v1 pith:GNEQBHF2 submitted 2025-07-21 cs.CL cs.AI

Haoran Sun , Zekun Zhang , Shaoning Zeng This is my paper
classification cs.CLcs.AI
keywords self-evolutiondual-phasefine-tuningpreferenceoptimizationdirectsupervisedlong-termdialoguesatisfactionestimationdataexpansion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes DPSE, a two-phase post-training framework meant to give large language models a form of continual self-evolution: the model collects its own interaction data, estimates how satisfied users were, and uses those estimates to build new training sets, so it can keep improving without fresh human annotation. The central claim is that this loop improves both user alignment and domain competence, which the paper argues SFT, preference optimization, and memory-augmented prompting each improve only separately. On general NLP benchmarks DPSE raises the AlpacaEval length-controlled win rate from 13.04% (UPO) to 14.26% and MT-Bench from 7.02 to 8.46, and on the LoCoMo long-term dialogue benchmark it beats ReadAgent, MemoryBank, and A-MEM by large F1 margins. The authors take this as evidence that endogenous parameter updates outperform exogenous memory retrieval for long-term dialogue.

What carries the argument

The load-bearing object is the Censor module's satisfaction score, a scalar in [-1,1] computed from five normalized signals by a gated fusion network (a gate vector G and a credibility vector C) subject to four physical constraints: similarity weight capped at 10%, sentiment-modulated similarity penalty, explicit-feedback weight floor, and renormalization. This score does double duty: it decides which dialogues are kept as preference examples and sets the duplication frequency used both in data expansion and in the weighted DPO loss (Eq. 8), thereby concentrating training on interactions the Censor judges satisfying. The second mechanism is the two-phase training order—SFT on topic-expanded data first, then frequency-weighted DPO—which is meant to prevent preference optimization from drifting into fluent but factually wrong outputs.

What would settle it

Collect a corpus of real user–model dialogues, obtain genuine user satisfaction ratings, run the same dialogues through the Censor, and compute rank correlation or agreement; if the Censor cannot separate satisfied from dissatisfied conversations at well above chance, the self-evolution loop is optimizing an unvalidated surrogate and the framework's core mechanism fails.

Watch

Extended reading notes

Core claim

DPSE claims that post-training methods split into two camps—alignment via preference optimization and domain grounding via supervised fine-tuning—and that neither alone achieves what the paper calls self-evolution: improvement of the model's intrinsic competence, not just its surface alignment. DPSE unites them with a Censor module that estimates user satisfaction from five interaction signals (explicit feedback, dwell time, coherence, similarity, sentiment), a dual expansion mechanism that turns stored interactions into a topic-balanced SFT dataset and a satisfaction-weighted preference dataset, and a two-stage training pipeline in which supervised fine-tuning for domain grounding is followed by Direct Preference Optimization with satisfaction-frequency weighting. In the reported results, the 1,000-sample trigger threshold performs best, and DPSE consistently outperforms SFT, DPO, UPO, and memory-augmented baselines on AlpacaEval 2.0, MT-Bench, and LoCoMo.

Load-bearing premise

The whole loop depends on the Censor's satisfaction score being a trustworthy stand-in for real user satisfaction, yet the paper never checks that score against actual user ratings.

Editorial extensions

If this is right

  • If DPSE works as claimed, an LLM can continue improving during deployment without fresh human preference annotations, because the satisfaction score replaces them.
  • Domain cognition and preference alignment are not necessarily conflicting objectives; the SFT-then-DPO order lets the same interaction data feed both.
  • External memory systems (ReadAgent, MemoryBank, A-MEM) are not a substitute for parameter updates: on LoCoMo, DPSE beats them even though the memory baselines have retrieval available at inference time.
  • The trigger threshold matters: updating every 1,000 accumulated interactions balances noise and signal better than 500 or 2,000 in the reported experiments.
  • The WildChat simulation suggests the loop keeps improving over many rounds in this paper's setup, with the MT-Bench absolute score rising from 3.65 to 8.97.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test the paper does not run is to compare Censor's satisfaction scores against genuine user ratings on the same conversations; if the score is miscalibrated, the whole loop optimizes the wrong target.
  • The framework's domain-cognition gain is measured only through general and dialogue benchmarks; a knowledge-intensive test such as medical or legal QA would make the domain-grounding claim directly falsifiable.
  • The weighted DPO loss resembles reward-weighted training, so an ablation could clarify whether the gain comes from the satisfaction signal itself or simply from oversampling high-scoring examples.
  • The Censor's explicit-feedback and sentiment rules are fixed heuristics; deploying the same rules in another language or interaction style could shift scores and break the claimed robustness.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes Dual-Phase Self-Evolution (DPSE), a post-training framework in which a Censor module extracts five interaction signals, estimates a satisfaction score, and classifies topics; two expansion strategies then build supervised and preference datasets, and a two-stage pipeline applies SFT followed by satisfaction-weighted DPO. The authors evaluate DPSE on AlpacaEval 2.0, MT-Bench, and LoCoMo, reporting consistent gains over SFT/PO baselines and over memory-augmented baselines, plus ablations and a WildChat-based longitudinal simulation.

Significance. The design idea of jointly optimizing domain grounding and preference alignment in an autonomous loop is a plausible direction, and the paper is clearly structured, with an ablation study (Table 4) that isolates the Censor, data-construction, and self-evolution modules. However, the empirical evidence for the central claim is not valid: the LoCoMo evaluation leaks test data into the evolution process, and the satisfaction score that drives the learning signal is never validated. The claimed consistent outperformance is therefore not established, and the current manuscript would need a fully re-run evaluation protocol and a validation study to be publishable.

major comments (4)
  1. [Experiment, Implementation Details; Table 2] The LoCoMo comparison is invalid because the evolution data and the evaluation data come from the same dataset. The Implementation Details section states that "DPSE evolves on-line during deployment by collecting real-time data from the LoCoMo dataset," while Table 2 evaluates on LoCoMo Single-hop and Multi-hop questions. Since the memory baselines (ReadAgent, MemoryBank, A-MEM) are not fine-tuned on LoCoMo, the comparison is not controlled; the F1 and BLEU-1 gains in Table 2 are confounded by direct test-distribution exposure. This undermines the paper's central claim of consistent outperformance on long-term dialogue tasks.
  2. [Methodology, Censor Module; Eqs. (3)-(7); Table 3] The satisfaction score is the objective that shapes the expanded data and the DPO weights, but it is never validated against human ground truth. Table 3 presents only four illustrative examples and no correlation or agreement metric with actual user satisfaction. Because the gate and credibility networks and the physical constraints are either learned without a supervision signal or hand-set, the paper needs independent validation of the score before the self-evolution loop can be claimed to optimize user satisfaction.
  3. [Experiment, Optimal Trigger Threshold Analysis; Tables 1-2; Figure 3] The trigger threshold is selected based on the same benchmark results that are then reported as evidence. The paper tests three thresholds and identifies 1000 as optimal from Tables 1 and 2 and Figure 3, with no hold-out validation; with only three thresholds this is a form of post hoc selection, and the absence of significance testing across thresholds makes the "optimal setting" claim less reliable.
  4. [Further Analysis, WildChat simulation] The longitudinal experiment using 100,000 WildChat entries and MT-Bench validation lacks any control condition, so the reported rise from 3.65 to 8.97 cannot be attributed to DPSE's self-evolution mechanisms rather than to generic continued training on in-the-wild user queries. A comparison against standard SFT or DPO on the same WildChat data is needed to support the claim that self-evolution, rather than data exposure, drives the improvement.
minor comments (5)
  1. [Eq. (1)] The text says the mapping has a value range of [0, 2], but substituting the discretized dwell values 0, 1, 2 yields 0, 0.5, 0, so the range is actually [0, 0.5]; please correct the description.
  2. [Table 2 and 'Comparison to Memory baselines'] The term 'BLUE-1' is used instead of 'BLEU-1' in several places; please correct these typos.
  3. [Related Work] The notation 'Rrhf' should be 'RRHF' for the method of Yuan et al. (2023).
  4. [Experiment, 'Comparison to SFT and PO baselines'] The abbreviation 'SF' should be 'SFT' in the sentence 'The comparison results with SF and PO baselines...'.
  5. [References] The reference to Tunstall et al. (The Alignment Handbook) is missing the publication year and venue; please complete it.

Circularity Check

1 steps flagged · score 6.0 of 10

Partial circularity: DPSE is fine-tuned on LoCoMo and then evaluated on LoCoMo, so the memory-baseline outperformance claim is statistically forced rather than predicted.

  1. fitted input called prediction [Experiment / Implementation Details; Table 2 comparison to Memory baselines]
    "Unlike non-finetuned baselines, DPSE evolves on-line during deployment by collecting real-time data from the LoCoMo dataset, which targets long-term, multi-turn dialogues."

    The paper presents Table 2 as evidence that DPSE outperforms memory baselines on LoCoMo Single-hop and Multi-hop questions. However, the same LoCoMo dataset is used as the online self-evolution data source for DPSE's weight updates, while the memory baselines 'strictly follow their original training protocols' and receive no LoCoMo exposure. The evaluation is therefore not a prediction from an independent model but a measurement on a distribution the model was fine-tuned on. The reported F1 and BLEU gains can be explained by direct test-distribution exposure, making the memory-baseline superiority claim statistically forced rather than derived from the framework's components.

full rationale

The general-NLP results (Table 1) are self-contained: DPSE evolves on UltraFeedback and UltraChat200K, self-evolution is disabled during benchmark evaluation, and AlpacaEval 2.0 and MT-Bench are external to the training data. Those results are not circular. The circular step is confined to the memory-baseline pillar (Table 2), where DPSE is fine-tuned on LoCoMo and then evaluated on LoCoMo, breaking the control condition against non-finetuned baselines. This is a fitted-input-called-prediction pattern: the LoCoMo numbers are not an independent prediction but a trained outcome on the evaluation distribution. The Censor's satisfaction score is a self-defined proxy without ground-truth validation, but that is a calibration/validity concern rather than a derivation that reduces to its inputs, because the headline external benchmarks do not depend on the Censor's scores. Threshold selection after inspecting results is a model-selection concern, not circularity. Overall, one of the two central claims is substantially compromised, yielding a partial circularity score of 6.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central contribution rests on a satisfaction scoring function with many unvalidated, hand-set parameters, plus assumptions that the five signals and the physical constraints reflect genuine user satisfaction. No independent evidence is provided for these assumptions beyond a four-example case study. The free parameters are not fitted to ground-truth labels and could substantially alter the results.

free parameters (6)
  • dwell transformation coefficients = -0.5, 0.5
    The quadratic in Eq. (1) maps dwell time to an attention signal with an arbitrary inverted-U shape; no empirical basis is provided and the description as U-shaped is incorrect.
  • similarity weight cap = 0.1
    Eq. (3) caps the similarity signal's weight at 10% of the total, a hand-set constraint.
  • sentiment sensitivity beta = not specified
    Eq. (4) uses beta > 0 to modulate the similarity weight under negative sentiment; the value is never given.
  • explicit feedback minimum tau = not specified
    Eq. (5) enforces a minimum weight for explicit feedback; the value is never given.
  • expansion scale K = 10 (example)
    Satisfaction scores are scaled by K and floored to obtain expansion frequency; the text suggests K = 10.
  • gate, credibility, and fusion network weights = unknown
    These neural sub-networks are described but no training procedure or loss is specified, so their weights are unconstrained and effectively free parameters.
assumptions (4)
  • domain assumption The five extracted signals (explicit feedback, dwell time, coherence, similarity, sentiment) capture user satisfaction.
    Section 'Signal Extraction and Preprocessing' asserts these signals represent satisfaction without validation against human labels or an independent reward model.
  • ad hoc to paper The physically inspired constraints in Eqs. (3)-(6) encode human intuition about satisfaction.
    Section 'Incorporating Physical Constraints' introduces weighting rules without empirical support or derivation; they are specific to this paper.
  • domain assumption The topic classifier and uncertainty-based filtering ensure data quality for fine-tuning.
    Section 'Methodology' claims hallucination mitigation via prompts and validation, but no metrics on filter accuracy are provided.
  • standard math The DPO loss formula (Eq. 8) is a valid objective for preference optimization.
    Borrowed from Rafailov et al. 2023, this is a standard result and does not need re-derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Novel Self-Evolution Framework for Large Language Models." pith.science (2026). https://pith.science/paper/GNEQBHF2

@misc{pith2026250715281,
  author       = {Pith},
  title        = {Pith review of: A Novel Self-Evolution Framework for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GNEQBHF2}},
  note         = {Machine review of arXiv:2507.15281}
}
read the original abstract

The capabilities of Large Language Models (LLMs) are limited to some extent by pre-training, so some researchers optimize LLMs through post-training. Existing post-training strategies, such as memory-based retrieval or preference optimization, improve user alignment yet fail to enhance the model's domain cognition. To bridge this gap, we propose a novel Dual-Phase Self-Evolution (DPSE) framework that jointly optimizes user preference adaptation and domain-specific competence. DPSE introduces a Censor module to extract multi-dimensional interaction signals and estimate satisfaction scores, which guide structured data expansion via topic-aware and preference-driven strategies. These expanded datasets support a two-stage fine-tuning pipeline: supervised domain grounding followed by frequency-aware preference optimization. Experiments across general NLP benchmarks and long-term dialogue tasks demonstrate that DPSE consistently outperforms Supervised Fine-Tuning, Preference Optimization, and Memory-Augmented baselines. Ablation studies validate the contribution of each module. In this way, our framework provides an autonomous path toward continual self-evolution of LLMs.

Figures

Figures reproduced from arXiv: 2507.15281 by the authors.

Figure 1
Figure 1. Overview of Dual-Phase Self-Evolution frame [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of DPSE Framework. Censor extracts multidimensional signals from user–model interactions, computes satisfaction scores, and performs topic classification to construct structured preference memory. Based on this memory, DPSE introduces two data expansion strategies, preference-driven expansion guided by satisfaction scores, and topic-aware expansion based on topic distribution. Subsequently, DPSE conduct… view at source ↗
Figure 3
Figure 3. The curve of training loss on UltraFeedback, Ul [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: We input the WildChat dataset into DPSE to simulate long-term usage by real users and validate it on the MT-Bench [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 9 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bai, Y.; Jones, A.; Ndousse, K.; Askell, A.; Chen, A.; DasSarma, N.; Drain, D.; Fort, S.; Ganguli, D.; Henighan, T.; et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862

    arXiv 2022

  4. [4]

    Chen, T.; Liu, S.; Chang, S.; Cheng, Y.; Amini, L.; and Wang, Z. 2020. Adversarial robustness: From self-supervised pre-training to fine-tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 699--708

    work page 2020

  5. [5]

    Cui, G.; Yuan, L.; Ding, N.; Yao, G.; He, B.; Zhu, W.; Ni, Y.; Xie, G.; Xie, R.; Lin, Y.; Liu, Z.; and Sun, M. 2024. UltraFeedback: Boosting Language Models with Scaled AI Feedback. arXiv:2310.01377

    arXiv 2024

  6. [6]

    Ding, N.; Chen, Y.; Xu, B.; Qin, Y.; Zheng, Z.; Hu, S.; Liu, Z.; Sun, M.; and Zhou, B. 2023. Enhancing Chat Language Models by Scaling High-quality Instructional Conversations. arXiv:2305.14233

    arXiv 2023

  7. [7]

    Dong, G.; Yuan, H.; Lu, K.; Li, C.; Xue, M.; Liu, D.; Wang, W.; Yuan, Z.; Zhou, C.; and Zhou, J. 2023. How abilities in large language models are affected by supervised fine-tuning data composition. arXiv preprint arXiv:2310.05492

    arXiv 2023

  8. [8]

    Dubois, Y.; Galambosi, B.; Liang, P.; and Hashimoto, T. B. 2025. Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. arXiv:2404.04475

    arXiv 2025

Show all 34 references
  1. [9]

    Gu, J.; Jiang, X.; Shi, Z.; Tan, H.; Zhai, X.; Xu, C.; Li, W.; Shen, Y.; Ma, S.; Liu, H.; et al. 2024. A survey on llm-as-a-judge. arXiv preprint arXiv:2411.15594

    2024 arXiv

  2. [10]

    Huang, X.; Liu, W.; Chen, X.; Wang, X.; Wang, H.; Lian, D.; Wang, Y.; Tang, R.; and Chen, E. 2024. Understanding the planning of LLM agents: A survey. arXiv preprint arXiv:2402.02716

    2024 arXiv

  3. [11]

    Lee, K.-H.; Chen, X.; Furuta, H.; Canny, J.; and Fischer, I. 2024. A human-inspired reading agent with gist memory of very long contexts. arXiv preprint arXiv:2402.09727

    2024 arXiv

  4. [12]

    Liu, S.; Fang, W.; Hu, Z.; Zhang, J.; Zhou, Y.; Zhang, K.; Tu, R.; Lin, T.-E.; Huang, F.; Song, M.; et al. 2025. A survey of direct preference optimization. arXiv preprint arXiv:2503.11701

    2025 arXiv

  5. [13]

    Maharana, A.; Lee, D.-H.; Tulyakov, S.; Bansal, M.; Barbieri, F.; and Fang, Y. 2024. Evaluating very long-term conversational memory of llm agents. arXiv preprint arXiv:2402.17753

    2024 arXiv

  6. [14]

    Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C. L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P.; Leike, J.; and Lowe, R. 2022. Training language model...

    2022 arXiv

  7. [15]

    D.; Ermon, S.; and Finn, C

    Rafailov, R.; Sharma, A.; Mitchell, E.; Manning, C. D.; Ermon, S.; and Finn, C. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36: 53728--53741

    2023

  8. [16]

    Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; and Liu, P. J. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. In Journal of Machine Learning Research. T5 Paper

    2020

  9. [17]

    Shen, Z. 2024. Llm with tools: A survey. arXiv preprint arXiv:2409.18807

    2024 arXiv

  10. [18]

    Song, F.; Yu, B.; Li, M.; Yu, H.; Huang, F.; Li, Y.; and Wang, H. 2024. Preference ranking optimization for human alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 18990--18998

    2024

  11. [19]

    Sun, H.; Bian, H.; Zeng, S.; Rao, Y.; Xu, X.; Mei, L.; and Gou, J. 2025. DatasetAgent: A Novel Multi-Agent System for Auto-Constructing Datasets from Real-World Images. arXiv:2507.08648

    2025 arXiv

  12. [20]

    Sun, H.; and Zeng, S. 2025. Introspection of Thought Helps AI Agents. arXiv:2507.08664

    2025 arXiv

  13. [21]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; Rodriguez, A.; Joulin, A.; Grave, E.; and Lample, G. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971

    2023 arXiv

  14. [22]

    Rush, A.; and Wolf, T

    Tunstall, L.; Beeching, E.; Lambert, N.; Rajani, N.; Huang, S.; Rasul, K.; Bartolome, A.; M. Rush, A.; and Wolf, T. ???? The Alignment Handbook

  15. [23]

    Wang, J.; Zhou, Y.; Zhang, X.; Bao, M.; and Yan, P. 2025. Self-evolutionary large language models through uncertainty-enhanced preference optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 25362--25370

    2025

  16. [24]

    S.; and Wong, D

    Wu, J.; Yang, S.; Zhan, R.; Yuan, Y.; Chao, L. S.; and Wong, D. F. 2025. A survey on LLM-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 1--66

    2025

  17. [25]

    Xu, W.; Liang, Z.; Mei, K.; Gao, H.; Tan, J.; and Zhang, Y. 2025. A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110

    2025 arXiv

  18. [26]

    Yang, A.; Yang, B.; Hui, B.; Zheng, B.; Yu, B.; Zhou, C.; Li, C.; Li, C.; Liu, D.; Huang, F.; Dong, G.; Wei, H.; Lin, H.; Tang, J.; Wang, J.; Yang, J.; Tu, J.; Zhang, J.; Ma, J.; Xu, J.; Zhou, J.; Bai, J.; He, J.; Lin, J.; Dang, K.; Lu, K.; Chen, K.; Yang, K.; Li, M.; Xue, M.;...

    2024 arXiv

  19. [27]

    Yao, Y.; Duan, J.; Xu, K.; Cai, Y.; Sun, Z.; and Zhang, Y. 2024. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, 100211

    2024

  20. [28]

    Yi, Z.; Ouyang, J.; Liu, Y.; Liao, T.; Xu, Z.; and Shen, Y. 2024. A survey on recent advances in llm-based multi-turn dialogue systems. arXiv preprint arXiv:2402.18013

    2024 arXiv

  21. [29]

    Yuan, H.; Yuan, Z.; Tan, C.; Wang, W.; Huang, S.; and Huang, F. 2023. Rrhf: Rank responses to align language models with human feedback. Advances in Neural Information Processing Systems, 36: 10935--10950

    2023

  22. [30]

    Zeng, Y.; Cui, X.; Jin, X.; Liu, G.; Sun, Z.; Li, D.; Yang, N.; Hao, J.; Zhang, H.; and Wang, J. 2025. Evolving LLMs' Self-Refinement Capability via Iterative Preference Optimization. arXiv preprint arXiv:2502.05605

    2025

  23. [31]

    Zhang, Z.; Bo, X.; Ma, C.; Li, R.; Chen, X.; Dai, Q.; Zhu, J.; Dong, Z.; and Wen, J.-R. 2024. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404.13501

    2024 arXiv

  24. [32]

    Zhao, W.; Ren, X.; Hessel, J.; Cardie, C.; Choi, Y.; and Deng, Y. 2024. WildChat: 1M ChatGPT Interaction Logs in the Wild. arXiv:2405.01470

    2024 arXiv

  25. [33]

    P.; Zhang, H.; Gonzalez, J

    Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E. P.; Zhang, H.; Gonzalez, J. E.; and Stoica, I. 2023. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv:2306.05685

    2023 arXiv

  26. [34]

    Zhong, W.; Guo, L.; Gao, Q.; Ye, H.; and Wang, Y. 2024. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 19724--19731

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.