Pith. sign in

REVIEW 12 cited by

RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.00746 v3 pith:KSBRDAI6 submitted 2023-10-01 cs.CL cs.AI

Zekun Moore Wang , Zhongyuan Peng , Haoran Que , Jiaheng Liu , Wangchunshu Zhou , Yuhan Wu , Hongcheng Guo , Ruitong Gan
show 9 more authors
Zehao Ni Jian Yang Man Zhang Zhaoxiang Zhang Wanli Ouyang Ke Xu Stephen W. Huang Jie Fu Junran Peng
This is my paper · ORCID
classification cs.CLcs.AI
keywords role-playingmodelsabilitiesllmsrolerolegptrolellmbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The advent of Large Language Models (LLMs) has paved the way for complex tasks such as role-playing, which enhances user interactions by enabling models to imitate various characters. However, the closed-source nature of state-of-the-art LLMs and their general-purpose training limit role-playing optimization. In this paper, we introduce RoleLLM, a framework to benchmark, elicit, and enhance role-playing abilities in LLMs. RoleLLM comprises four stages: (1) Role Profile Construction for 100 roles; (2) Context-Based Instruction Generation (Context-Instruct) for role-specific knowledge extraction; (3) Role Prompting using GPT (RoleGPT) for speaking style imitation; and (4) Role-Conditioned Instruction Tuning (RoCIT) for fine-tuning open-source models along with role customization. By Context-Instruct and RoleGPT, we create RoleBench, the first systematic and fine-grained character-level benchmark dataset for role-playing with 168,093 samples. Moreover, RoCIT on RoleBench yields RoleLLaMA (English) and RoleGLM (Chinese), significantly enhancing role-playing abilities and even achieving comparable results with RoleGPT (using GPT-4).

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs

    cs.CL 2026-07 conditional novelty 6.0 of 10

    HSS-Synth generates 230k instruction-tuning samples for 14 humanities/social-science fields and reports state-of-the-art fine-tuning results on 16 benchmarks.

  2. SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

    cs.DC 2026-02 unverdicted novelty 6.0 of 10

    SPEED-Bench is a new standardized benchmark for speculative decoding that supplies semantically diverse qualitative data and throughput-oriented splits across concurrency levels, integrated with vLLM and TensorRT-LLM.

  3. LLMs vs. Chinese Anime Enthusiasts: A Comparative Study on Emotionally Supportive Role-Playing

    cs.CL 2025-08 unverdicted novelty 6.0 of 10

    ChatAnime, a new emotionally supportive anime role-play benchmark, reports top LLMs outperforming human enthusiasts on role-playing and emotional support metrics while humans keep the diversity edge.

  4. CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards

    cs.CL 2025-07 conditional novelty 6.0 of 10

    A role-playing LLM that reasons about the scene and its own state before responding, trained with two semantic rewards, beats stronger baselines on role-play benchmarks.

  5. Which Prompting Technique Should I Use? An Empirical Investigation of Prompting Techniques for Software Engineering Tasks

    cs.SE 2025-06 conditional novelty 6.0 of 10

    Across ten software engineering tasks and four LLMs, no prompting technique wins consistently; ES-KNN is best on many tasks, some techniques underperform the baseline, and USC is best for code QA and code generation.

  6. X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

    cs.LG 2026-07 conditional novelty 5.0 of 10

    X3-OPD improves audio-grounded reasoning by training the audio student on its own rollouts with token-level teacher feedback, using a three-tier paired text-audio corpus.

  7. VoxRole: A Comprehensive Benchmark for Evaluating Speech-Based Role-Playing Agents

    cs.CL 2025-09 reject novelty 5.0 of 10

    A 65.6-hour movie-dialogue benchmark for spoken role-playing agents, with an evaluation framework whose main judge is also an evaluated model.

  8. Statistical Hypothesis Testing for Auditing Robustness in Language Models

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A permutation-based hypothesis test on pairwise semantic similarities detects whether LLM outputs shift under arbitrary input or model perturbations.

  9. OmniCharacter: Towards Immersive Role-Playing Agents with Seamless Speech-Language Personality Interaction

    cs.CL 2025-05 conditional novelty 5.0 of 10

    OmniCharacter is a speech-language role-playing agent that generates character-specific voice responses with low latency, trained on a new 10K-dialogue, 135K-audio dataset of 20 game characters.

  10. Concept Incongruence: An Exploration of Time and Death in Role Playing

    cs.CL 2025-05 conditional novelty 5.0 of 10

    LLMs asked to role-play dead historical figures rarely abstain from answering post-death questions, and their factual accuracy drops due to poorly encoded death states and role-playing-induced shifts in temporal repre...

  11. A Stepwise Questioning Expert-Editor Multi-Agent Framework for Long-Document Summarization

    cs.CL 2026-07 conditional novelty 4.0 of 10

    An expert-editor stepwise-questioning multi-agent pipeline improves ROUGE/BERTScore/FactCC for long scientific summarization on two datasets relative to direct generation and HERA.

  12. Great Power Brings Great Responsibility: Personalizing Conversational AI for Diverse Problem-Solvers

    cs.SE 2025-02 conditional novelty 4.0 of 10

    A vision paper suggests tailoring ChatGPT guidance to OSS newcomers using GenderMag personas, with only an illustrative example and no empirical test.

Pith tools