Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RAIDEN-R1 claims GRPO with verifiable keyword rewards lifts role-play memory metrics to about 88 percent.

desk verdict A plausible GRPO-plus-keyword-reward recipe for role-playing training, but the paper's own table undercuts the robustness claim and the reward is too entangled with the metric to support broad role-awareness conclusions. read the letter →

arxiv 2505.10218 v1 pith:PCDSULHS submitted 2025-05-15 cs.CL

Zongsheng Wang , Kaili Sun , Bowen Wu , Qun Yu , Ying Li , Baoxun Wang This is my paper · ORCID
classification cs.CL
keywords role-playingconversationalagentsGRPOverifiablerewardkeywordmatchingchain-of-thoughtRAIDENbenchmarkrole-awarenessreinforcementlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that role-awareness in role-playing conversational agents can be treated as a verifiable reward signal rather than an open-ended quality judgment. It trains a 14B instruction-tuned model with GRPO, scoring each response by whether it contains a mined role-specific keyword (single-term validation) or passes an auto-generated Python parsing function (multi-term dynamic parsing). On the RAIDEN benchmark, the resulting model reaches 88.04 percent on Script-Based Knowledge and 88.65 percent on Conversation Memory, beating the untrained base model and avoiding the degradation seen with plain SFT. The authors also build a cold-start chain-of-thought dataset and show that SFT on it alone does not improve role-play, while the GRPO stage does. If correct, this shows that a rule-based, quantifiable reward can bridge the gap that made reinforcement learning hard to apply to open-ended role-play.

What carries the argument

The carrying mechanism is VRAR, the Verifiable Role-Awareness Reward, which converts role-consistency into a binary score. Single-Term Validation extracts exactly one validated keyword per WH-question and gives a reward of 1 iff the response contains that keyword. Multi-Term Dynamic Parsing expands keywords into semantically equivalent variants, filters them for legitimacy, and generates a Python function that parses the response, giving a reward of 1 iff the function returns True. A format reward wraps the chain-of-thought in think tags with Chinese-character and repetition constraints, and a cold-start SFT stage supplies first-person chain-of-thought data intended to teach persona-consistent internal monologue before the GRPO stage.

What would settle it

Take the trained 14B-GRPO model and evaluate it on a held-out set where each response contains the target keyword but contradicts a fact in the character profile or an earlier dialogue turn; if human raters judge these responses as role-unaware at roughly the same rate as the base model, the reward measured keyword emission rather than role-awareness.

Watch

Extended reading notes

Core claim

The central discovery is that a verifiable keyword-based reward, paired with GRPO, improves role-awareness metrics on the RAIDEN benchmark while leaving conversational quality largely intact. The reward design has two tiers: single-term validation for questions with one unambiguous keyword, and multi-term dynamic parsing for cases with multiple acceptable phrasings, where semantic expansion plus generated Python parsers decide correctness. Trained on 1,000 Script-Based Knowledge and Conversation Memory samples plus 1,000 filtered general role-play samples, the 14B-GRPO model scores 88.04 percent SBK and 88.65 percent CM, surpassing the 14B-Instruct baseline while maintaining secondary metrics such as Script-Contradictory Knowledge and Role-Cognition Boundary. The paper also finds that SFT on the GRPO training data or on a multi-LLM cold-start chain-of-thought corpus degrades most role-play metrics, while subsequent GRPO training recovers and improves them; case analysis shows first-person chain-of-thought reasoning emerging, including detection of contradictory profile information.

Load-bearing premise

The load-bearing premise is that a response containing the mined keyword, or passing the generated Python check, is genuinely role-aware; if keyword occurrence can be satisfied without true role consistency, GRPO can exploit that shortcut.

Editorial extensions

If this is right

  • Role-play improvement can be optimized with a fixed, deterministic reward, making RL viable for role-playing agents without relying on noisy LLM-as-judge feedback.
  • GRPO on a focused reward preserves base-model conversational skills better than SFT on the same training data, since RL nudges only the scored behavior.
  • Cold-start chain-of-thought SFT alone is not a sufficient bridge; the verifiable RL stage is what moves the primary metrics.
  • The method extends to other role dimensions with definitive answers: any metric reducible to keyword or parsing checks can be added as another reward term.
  • The observed short chain-of-thought outputs suggest role-aware reasoning emerges in compact, first-person form rather than long deliberation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same reward template could be applied to emotional-consistency or action-consistency checks, converting any aspect of a character sheet into pass/fail verifiers.
  • A risk the paper does not fully settle: because correctness is defined by keyword presence, GRPO can optimize for keyword insertion. The degraded secondary metrics in some conditions hint that role-boundary behaviors may not be captured by the reward.
  • A testable extension: feed the GRPO model adversarial queries where the mined keyword is contextually irrelevant and measure whether it still emits the keyword; if it does, the reward is being gamed.
  • Another extension: anneal or reweight the accuracy reward over training so the model first learns keyword coverage, then is pushed to justify it in chain-of-thought; the current paper does not explore reward shaping.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes RAIDEN-R1, a reinforcement-learning framework for role-playing conversational agents. The method defines a Verifiable Role-Awareness Reward (VRAR) built from keyword-based checks: Single-Term Validation (STV), which rewards responses containing a target keyword, and Multi-Term Dynamic Parsing (MTDP), which uses generated Python checker functions that test for expanded keyword variants; a format reward enforces <think> tags and a Chinese-character ratio. A cold-start SFT corpus is constructed with DeepSeek-R1 and Claude 3.5, and the system is trained with GRPO on Qwen2.5-14B-Instruct. On the RAIDEN benchmark test set, the 14B-GRPO model reaches 88.04% on Script-Based Knowledge and 88.65% on Conversation Memory, exceeding the 14B-Instruct baseline, while showing lower scores on SCK, RCB, TA, and TS. The paper claims this demonstrates effective and robust role-aware RL training and 'bridges the non-quantifiability gap' in RPCA training; case studies illustrate improved contextual memory and self-correction in misleading-query scenarios.

Significance. If the central claim were fully supported, the paper would be a useful contribution: it applies GRPO with a rule-based reward to a domain usually considered non-quantifiable, and it reports a transparent training recipe, including an honest negative result for cold-start SFT. The multi-LLM data-generation pipeline and the case analyses are informative. However, the significance is currently limited by the reward-validity question: because the reward is a surface-form keyword-containment test built from the same RAIDEN evaluation categories used as the primary outcome, the reported SBK/CM gains may reflect keyword-emission optimization rather than improved role-awareness. The robustness claim is also contradicted by the paper's own Table 1, and no independent validation, error bars, or significance tests are provided.

major comments (3)
  1. [§3.3, Table 1] The claim that 14B-GRPO maintains 'comparable performance on other metrics' is not supported by the table it refers to: relative to 14B-Instruct, SCK drops from 84.75% to 81.36% and RCB from 62.77% to 51.44%, with TA and TS also lower. These are role-cognition metrics, not conversational extras, so the robustness conclusion in the Abstract and Section 3.3 is undercut. Additionally, the table reports a single run with no confidence intervals, error bars, or significance tests, so the primary-metric gains (+1.45 on SBK, +8.40 on CM) cannot be distinguished from noise. Please provide multiple seeds with dispersion measures, or revise the robustness claim to match the observed declines.
  2. [§2.1.2, Algorithm 1 and Figure 2] The accuracy reward is a surface-form keyword-containment test: STV assigns 1 iff the target keyword is a substring of the response, and the MTDP Python checker in the Figure 2 example is itself a containment check over expanded keyword variants. Under this reward, a response that contains the keyword in a negated or role-violating context is scored 1, while a semantically correct paraphrase that omits the keyword is scored 0. GRPO can therefore improve the reward by keyword insertion rather than by improving role-awareness. The paper offers no control evaluation (e.g., keyword-free human judgments, removal of reward keywords at test time, or measurements of keyword-stuffing behavior) to rule out this failure mode. Since the central claim depends on VRAR being a genuine verifiable role-awareness signal, this issue is load-bearing and must be addressed.
  3. [§2.1.1, §3.2, §3.3] The reward is constructed from the same RAIDEN evaluation categories (SBK and CM keyword checks) used as the primary outcome, and the GRPO training data are drawn from the RAIDEN training split. Training on a benchmark and testing on its held-out split is standard, but here the reward literally encodes the evaluation criterion, so the SBK/CM improvements may reflect overfitting to the benchmark's keyword distribution rather than transferable role-awareness. No independent validation is provided: no second role-playing benchmark, no human evaluation, and no held-out keyword-free assessment. Section 4 also concedes that the rewards cover only memory and profile-consistency dimensions. Please add an external validation set or human study to support the general claim that the framework 'bridges the non-quantifiability gap'.
minor comments (5)
  1. [§3.2] The phrase 'irrelevant metrics' is misleading: SCK and RCB are directly relevant to role-awareness, and TA/TS are conversational quality metrics. Please use 'remaining' or 'supplementary' metrics instead.
  2. [Figure 4 caption and Figure 1] The Figure 4 caption contains the typo 'Camparisions' for 'comparisons', and Figure 1 contains 'Referneces' for 'References'; both should be corrected.
  3. [§3.1] Please state the exact composition of the 1,000 RAIDEN samples (how many SBK versus CM) and the breakdown for the 1,000 general role-playing samples; the current wording is ambiguous about whether 1,000 is the total or per category.
  4. [§3.1, reproducibility] No code, data, or trained checkpoints are provided, and some GRPO hyperparameters (e.g., KL coefficient, clipping range, sampling temperature, random seed) are missing; specifying these would materially improve reproducibility.
  5. [Section 1 and 4] The footnote marker after 'VRAR' is typeset as an inline number rather than a superscript, and 'Raiden' / 'RAIDEN' are used inconsistently; also, the Section 4 admission that the rewards are limited to memory and profile-consistency dimensions should be reflected in the abstract's broad 'bridges the non-quantifiability gap' phrasing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the keyword-containment reward is a stated training proxy, and the reported SBK/CM accuracy is evaluated by an independent LLM judge, so the measured gains are not entailed by the reward definition.

full rationale

The paper's derivation chain is not circular. The training reward is explicitly a substring or parsed-keyword test: Algorithm 1 assigns STV reward iff X is in R and MTDP reward iff the generated Python checker returns True, with the Figure 2 example being another containment check over expanded keywords. The primary outcome, however, is RAIDEN SBK/CM accuracy as assessed by an LLM judge: Section 3.2 states, "For assessment, we adopted the LLM-as-a-judge approach using Claude 3.5 for correctness evaluation." The test metric is therefore not identical to the reward by construction; a response can satisfy the keyword reward and still fail the judge, or satisfy the judge without containing the exact keyword. The reward is a heuristic proxy, not the evaluation itself, and the paper does not fit any parameter to the test set. The RAIDEN benchmark citation [16] involves overlapping authors, but RAIDEN is a published, externally usable benchmark with independently applicable metrics, so citing it as the evaluation instrument is a normal use of an external artifact rather than a self-citation chain that forces the result. The observed SCK and RCB declines in Table 1 are a validity and robustness concern about the keyword proxy, not evidence that the SBK/CM improvements reduce to the reward definition. No uniqueness theorem is imported from the authors' prior work, and no ansatz is hidden behind a citation. The acknowledged limitation in Section 4 concerns coverage of additional role-playing metrics, not a circular derivation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that keyword containment is a valid reward signal for role-awareness, on the RAIDEN metric definitions, and on the reliability of Claude 3.5 as judge. No new entities are introduced. The hand-set thresholds in the reward and data pipeline are the main free parameters.

free parameters (4)
  • Chinese character ratio threshold = > 0.7
    Algorithm 2 requires a Chinese-character ratio above 0.7 to receive format reward; the threshold is chosen by hand and no sensitivity analysis is reported.
  • MTDP consistency threshold = 70%
    Data instances are kept when QwQ-32B judgment and Python execution agree on more than 70% of 10 model responses; the cutoff is hand-set.
  • Training data size = 1,000 SBK/CM + 1,000 general
    The paper fixes the RL training set at 2,000 samples and does not ablate this choice.
  • GRPO generation count = 7
    The vLLM generation quantity is set to 7 during GRPO; a hyperparameter, not shown to have been tuned or varied.
assumptions (4)
  • domain assumption RAIDEN benchmark's SBK and CM metrics are valid measures of role-awareness.
    The paper selects training data from these dimensions and evaluates on them, assuming they capture the construct of role-awareness.
  • ad hoc to paper Keyword containment is a reliable proxy for response correctness.
    The accuracy reward in Section 2.1.2 assigns full credit when the target keyword appears, with no evidence that role-appropriate responses require those exact strings.
  • domain assumption Claude 3.5 as LLM-as-a-judge provides accurate correctness labels.
    The evaluation in Section 3.2 relies on Claude 3.5 without reporting agreement with human labels or other judges.
  • standard math GRPO and Open-R1 implementation are correct and stable at the chosen hyperparameters.
    The RL method is imported from prior work; training stability is asserted but not verified with ablations or seeds.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward." pith.science (2026). https://pith.science/paper/PCDSULHS

@misc{pith2026250510218,
  author       = {Pith},
  title        = {Pith review of: RAIDEN-R1: Improving Role-awareness of LLMs via GRPO with Verifiable Reward},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PCDSULHS}},
  note         = {Machine review of arXiv:2505.10218}
}
read the original abstract

Role-playing conversational agents (RPCAs) face persistent challenges in maintaining role consistency. To address this, we propose RAIDEN-R1, a novel reinforcement learning framework that integrates Verifiable Role-Awareness Reward (VRAR). The method introduces both singular and multi-term mining strategies to generate quantifiable rewards by assessing role-specific keys. Additionally, we construct a high-quality, role-aware Chain-of-Thought dataset through multi-LLM collaboration, and implement experiments to enhance reasoning coherence. Experiments on the RAIDEN benchmark demonstrate RAIDEN-R1's superiority: our 14B-GRPO model achieves 88.04% and 88.65% accuracy on Script-Based Knowledge and Conversation Memory metrics, respectively, outperforming baseline models while maintaining robustness. Case analyses further reveal the model's enhanced ability to resolve conflicting contextual cues and sustain first-person narrative consistency. This work bridges the non-quantifiability gap in RPCA training and provides insights into role-aware reasoning patterns, advancing the development of RPCAs.

Figures

Figures reproduced from arXiv: 2505.10218 by the authors.

Figure 1
Figure 1. Workflow of VRAR, followed by the multi-stage training strategy of DeepSeek R1, we create two datasets for sft and reinforcement learning. work like DeepSeek-R1[6], which activates self-reasoning capability through Group Relative Policy Optimization (GRPO)[9] with quantifiable rewards. Intuitively, the CoT reasoning exhibits natural compatibility with RPCAs, as its step-by-step reasoning mirrors the human-like narra… view at source ↗
Figure 2
Figure 2. Pseudocode and example of one parsing function in accuracy reward. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An example of the generated cold-start CoT dataset [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Camparisions between responses from 14B-Instruct and 14B-GRPO under different query [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LLMs vs. Chinese Anime Enthusiasts: A Comparative Study on Emotionally Supportive Role-Playing

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    ChatAnime, a new emotionally supportive anime role-play benchmark, reports top LLMs outperforming human enthusiasts on role-playing and emotional support metrics while humans keep the diversity edge.

Reference graph

Works this paper leans on

22 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Roleinteract: Evaluating the social interaction of role-playing agents

    Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Xing Gao, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, Fei Huang, et al. Roleinteract: Evaluating the social interaction of role-playing agents. arXiv e-prints, pages arXiv–2403, 2024

    work page 2024

  3. [3]

    Socialbench: Sociality evaluation of role-playing conversational agents

    Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Gao Xing, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, and Fei Huang. Socialbench: Sociality evaluation of role-playing conversational agents. In Findings of the Association for Computational Linguistics ACL 2024, pages 2108–2126, 2024

    work page 2024

  4. [4]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025

    2025

  5. [5]

    Reasoning does not necessarily improve role-playing ability

    Xiachong Feng, Longxu Dou, and Lingpeng Kong. Reasoning does not necessarily improve role-playing ability. arXiv preprint arXiv:2502.16940, 2025

    arXiv 2025

  6. [6]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  7. [7]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    arXiv 2024

  8. [8]

    Large language models are superpo- sitions of all characters: Attaining arbitrary role-play via self-alignment

    Keming Lu, Bowen Yu, Chang Zhou, and Jingren Zhou. Large language models are superpo- sitions of all characters: Attaining arbitrary role-play via self-alignment. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume1: Long Papers), pages 7828–7840, 2024. 8

    work page 2024

Show all 22 references
  1. [9]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  2. [10]

    Openai o1 system card, 2024

    OpenAI Team. Openai o1 system card, 2024

    2024

  3. [11]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025

    2025

  4. [12]

    CharacterEval: A Chinese benchmark for role-playing conversational agent evaluation

    Quan Tu, Shilong Fan, Zihang Tian, Tianhao Shen, Shuo Shang, Xin Gao, and Rui Yan. CharacterEval: A Chinese benchmark for role-playing conversational agent evaluation. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Asso...

    2024

  5. [13]

    Iteratively prompt pre-trained language models for chain of thought

    Boshi Wang, Xiang Deng, and Huan Sun. Iteratively prompt pre-trained language models for chain of thought. arXiv preprint arXiv:2203.08383, 2022

    2022 arXiv

  6. [14]

    Rolellm: Benchmarking, eliciting, and en- hancing role-playing abilities of large language models

    Noah Wang, Zy Peng, Haoran Que, Jiaheng Liu, Wangchunshu Zhou, Yuhan Wu, Hongcheng Guo, Ruitong Gan, Zehao Ni, Jian Yang, et al. Rolellm: Benchmarking, eliciting, and en- hancing role-playing abilities of large language models. In Findings of the Association for Computational ...

    2024

  7. [15]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  8. [16]

    RAIDEN benchmark: Evaluating role-playing conversational agents with measurement-driven custom dialogues

    Bowen Wu, Kaili Sun, Ziwei Bai, Ying Li, and Baoxun Wang. RAIDEN benchmark: Evaluating role-playing conversational agents with measurement-driven custom dialogues. In Proceedings of the 31st International Conference on Computational Linguistics, pages 11086–11106, Abu Dhabi, U...

    2025

  9. [17]

    Baichuan 2: Open large-scale language models

    Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Ce Bian, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, et al. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305, 2023

    2023 arXiv

  10. [18]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  11. [19]

    Beyond dialogue: A profile-dialogue alignment framework towards general role-playing language model

    Yeyong Yu, Runsheng Yu, Haojie Wei, Zhanqiu Zhang, and Quan Qian. Beyond dialogue: A profile-dialogue alignment framework towards general role-playing language model. arXiv preprint arXiv:2408.10903, 2024

    2024 arXiv

  12. [20]

    Cot-based synthe- sizer: Enhancing llm performance through answer synthesis

    Bohan Zhang, Xiaokang Zhang, Jing Zhang, Jifan Yu, Sijia Luo, and Jie Tang. Cot-based synthe- sizer: Enhancing llm performance through answer synthesis. arXiv preprint arXiv:2501.01668, 2025

    2025 arXiv

  13. [21]

    Characterglm: Customizing chinese conversational ai characters with large language models

    Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Lib- iao Peng, Jiaming Yang, Xiyao Xiao, et al. Characterglm: Customizing chinese conversational ai characters with large language models. arXiv preprint arXiv:2311.16832, 2023

    2023 arXiv

  14. [22]

    Characterbench: Benchmarking character customization of large language models

    Jinfeng Zhou, Yongkang Huang, Bosi Wen, Guanqun Bi, Yuxuan Chen, Pei Ke, Zhuang Chen, Xiyao Xiao, Libiao Peng, Kuntian Tang, et al. Characterbench: Benchmarking character customization of large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, ...

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.