REVIEW 4 cited by
Uncovering ChatGPT's Capabilities in Recommender Systems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The debut of ChatGPT has recently attracted the attention of the natural language processing (NLP) community and beyond. Existing studies have demonstrated that ChatGPT shows significant improvement in a range of downstream NLP tasks, but the capabilities and limitations of ChatGPT in terms of recommendations remain unclear. In this study, we aim to conduct an empirical analysis of ChatGPT's recommendation ability from an Information Retrieval (IR) perspective, including point-wise, pair-wise, and list-wise ranking. To achieve this goal, we re-formulate the above three recommendation policies into a domain-specific prompt format. Through extensive experiments on four datasets from different domains, we demonstrate that ChatGPT outperforms other large language models across all three ranking policies. Based on the analysis of unit cost improvements, we identify that ChatGPT with list-wise ranking achieves the best trade-off between cost and performance compared to point-wise and pair-wise ranking. Moreover, ChatGPT shows the potential for mitigating the cold start problem and explainable recommendation. To facilitate further explorations in this area, the full code and detailed original results are open-sourced at https://github.com/rainym00d/LLM4RS.
Forward citations
Cited by 4 Pith papers
-
Can Large Language Models Understand Preferences in Personalized Recommendation?
A new grouped-ranking benchmark finds current LLMs score near chance on personalized preference ranking once user rating bias and item quality are controlled.
-
Full-Stack Optimized Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation
ReLLaX combines semantic behavior retrieval, collaborative soft prompts, and a new fully interactive LoRA variant to improve LLM-based CTR prediction on long user histories.
-
PaSa: An LLM Agent for Comprehensive Academic Paper Search
PaSa, a two-agent LLM system trained with session-level RL, reports substantially higher recall than existing academic search baselines on complex paper-finding queries.
-
Personas within Parameters: Fine-Tuning Small Language Models with Low-Rank Adapters to Mimic User Behaviors
Persona-level LoRA fine-tuning lets a 3.8B small language model simulate MovieLens users about as accurately as a much larger frozen LLM, at lower cost.
Discussion (0). Continue with ORCID to comment.