REVIEW 8 cited by
Pearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Powerful large language models have facilitated the development of writing assistants that promise to significantly improve the quality and efficiency of composition and communication. However, a barrier to effective assistance is the lack of personalization in LLM outputs to the author's communication style, specialized knowledge, and values. In this paper, we address this challenge by proposing Pearl, a LLM writing assistant personalized with a retriever that is trained to be generation-calibrated for personalization. Generation calibration ensures that our retriever selects historic user authored documents to augment an LLM prompt such that they are likely to help an LLM generation better adhere to a users' preferences. We propose two key novelties for training such a retriever: (1) A training data selection method that identifies user requests likely to benefit from personalization and documents that provide that benefit; and (2) A scale-calibrating KL-divergence objective that ensures that our retriever scores remain proportional to the downstream generation quality from using the document for personalized generation. In a series of holistic evaluations, we demonstrate the effectiveness of Pearl in generating long-form texts on multiple social media datasets. Finally, we demonstrate how a generation-calibrated retriever can double as a performance predictor -- detecting low quality retrieval, and improving potentially under-performing outputs via revision with LLMs.
Forward citations
Cited by 8 Pith papers
-
Learning Dynamic User Personas from Implicit Interaction Streams via Iterative Refinement
IRIS learns and iteratively refines natural-language user personas from implicit interaction streams; on 100 Reddit AITA commenters it predicts held-out verdicts at 61% accuracy, within a statistically untested 56-61% band.
-
Evaluating Style-Personalized Text Generation: Challenges and Directions
A new style-discrimination benchmark for personalized text generation shows ensemble metrics give only a marginal, possibly test-fitted, edge over the best single judge.
-
PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization
PersonaFeedback provides a human-labeled benchmark showing current LLMs, including strong reasoners, score only about 65-70 percent on hard personalization choices, and explicit persona information helps more than retrieval.
-
Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning
TICL improves style personalization by iteratively adding model-generated negative examples and explanations to an in-context prompt, beating fine-tuned baselines in LLM-judged comparisons without any parameter updates.
-
Personas within Parameters: Fine-Tuning Small Language Models with Low-Rank Adapters to Mimic User Behaviors
Persona-level LoRA fine-tuning lets a 3.8B small language model simulate MovieLens users about as accurately as a much larger frozen LLM, at lower cost.
-
EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
EdgeLoRA combines automatic adapter routing, LRU caching with a memory pool, and grouped LoRA batching to serve thousands of LoRA adapters on edge devices with up to 4x higher throughput than llama.cpp.
-
PrefReward: Learning User Preference Matrix for Personalized Text Generation
PrefReward selects the most style-aligned LLM output via a KL-divergence reward against an explicit user preference matrix, beating retrieval baselines on LongLaMP.
-
T-POP: Test-Time Personalization with Online Preference Feedback
T-POP uses dueling-bandit token selection to learn a reward function online from pairwise user feedback, enabling test-time personalization of a frozen LLM without fine-tuning.
Discussion (0). Continue with ORCID to comment.