REVIEW 5 cited by
Rewarding Chatbots for Real-World Engagement with Millions of Users
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The emergence of pretrained large language models has led to the deployment of a range of social chatbots for chitchat. Although these chatbots demonstrate language ability and fluency, they are not guaranteed to be engaging and can struggle to retain users. This work investigates the development of social chatbots that prioritize user engagement to enhance retention, specifically examining the use of human feedback to efficiently develop highly engaging chatbots. The proposed approach uses automatic pseudo-labels collected from user interactions to train a reward model that can be used to reject low-scoring sample responses generated by the chatbot model at inference time. Intuitive evaluation metrics, such as mean conversation length (MCL), are introduced as proxies to measure the level of engagement of deployed chatbots. A/B testing on groups of 10,000 new daily chatbot users on the Chai Research platform shows that this approach increases the MCL by up to 70%, which translates to a more than 30% increase in user retention for a GPT-J 6B model. Future work aims to use the reward model to realise a data fly-wheel, where the latest user conversations can be used to alternately fine-tune the language model and the reward model.
Forward citations
Cited by 5 Pith papers
-
ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling
Per-step retrieval of solved exemplars injected into the reasoning trace improves test-time scaling accuracy, with up to 13.4 absolute points gained on AIME 2025.
-
Beyond Borrowed Histories: Person-Aligned User Simulation for Interactive Role-Playing Evaluation
PALATE trains five per-user simulators and personalized rubrics from real chat histories, then evaluates 16 role-playing agents on free multi-turn conversations, finding user-dependent winners.
-
It's Not That Simple. An Analysis of Simple Test-Time Scaling
The apparent scaling behavior of the s1 model is mostly an artifact of truncating long outputs, while appending 'Wait' does not reliably improve answers.
-
Thinking Short and Right Over Thinking Long: Serving LLM Reasoning Efficiently and Accurately
SART reduces LLM serving latency by combining redundant branch sampling with early stopping and PRM-based dynamic branch pruning, while keeping answer accuracy comparable to baseline methods.
-
Efficiently Scaling LLM Reasoning with Certaindex
Certaindex measures when an LLM's intermediate answers stop changing, enabling early exit and dynamic token allocation that cuts token usage by up to 50% with no accuracy drop in tested workloads.
Discussion (0). Continue with ORCID to comment.