{"id":"5c383ad8-914e-43a7-864c-c8817cb51825","arxiv_id":"2411.13415","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LLMGPR adapts Llama3-8b with QLoRA, semantic POI tokens, member aggregation, and purpose SSL, beating prior group POI recommenders on Foursquare, Weeplace, and Gowalla.","lead":"This paper presents LLMGPR, a framework that uses large language models to recommend places a group should visit next. It combines semantic text about places, member preference aggregation, and trip-purpose predictions to outperform existing group POI recommenders on three datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unclear whether member check-in sequences are truncated at the group test timestamp; if not, the group ground-truth POI leaks into member representations, potentially explaining LLMGPR's gains.","rationale":"The reader's weakest assumption was the heuristic construction of groups from social co-presence. That is a valid external-validity concern, but the more load-bearing issue is internal: the evaluation protocol may allow the ground-truth POI of a test group to appear in the input member sequences. Because group check-ins are defined as simultaneous visits by socially connected users, every group activity is also an individual check-in for each member. If member sequences are not masked at the group test time, the sequencing adapter's representation of a member can directly encode the answer POI, and the aggregation adapter—trained to predict the next group POI from these representations—can learn to exploit that signal. This would inflate LLMGPR's accuracy relative to baselines whose aggregation mechanisms are not trained to copy from member representations. The paper does not describe any temporal masking for member sequences in Algorithm 1 or Section 4.1, so the possibility is real and checkable. This is not an accusation of dishonesty; it is a missing specification in an otherwise plausible framework. The proposed test—a strictly truncated re-evaluation—would settle the issue. If the margins persist, the paper's empirical claim stands and the condition reduces to artifact release. If they collapse, the central claim would need to be revised. Hence the verdict remains CONDITIONAL, but the condition should explicitly include the leakage check, not just code/data release. The framework's components (semantic POI tokens, QLoRA sequencing adapter, aggregation adapter, SSL purpose prediction) are coherent and the ablation patterns are internally consistent; the concern is specifically about the evaluation protocol's temporal hygiene.","tokens_in":18723,"tokens_out":7273,"duration_ms":77837,"concrete_test":"Re-run the group evaluation (Table 2) with a strict temporal split: for each test group, truncate every member's sequence to check-ins strictly before the group's validation timestamp (excluding the held-out test and validation group check-ins and any later solo check-ins), and retrain the aggregation adapter under the same truncation. If LLMGPR's HR@10/NDCG@10 margins over MICL shrink substantially, the reported gains are partly leakage; if margins are stable, the concern is refuted.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"In Algorithm 1 (lines 19-26), for a test group sequence X_g, the model computes individual representations e_u = Theta_s(X_u) from each member's check-in sequence X_u. Section 4.1 states a leave-one-out split for each sequence, but does not specify that X_u is truncated at the group's held-out timestamp. Because group activities are constructed from the same check-ins that appear in members' individual histories (Section 4.1), each member's X_u may include the very POI that is the group's ground-truth next POI. Since the aggregation adapter is trained to predict the group's next POI from these member representations, it can learn to copy an answer already present in its input. The reported 4.4-9.0% improvements over MICL could then reflect this temporal leakage rather than genuine group-preference modeling. This concern is distinct from the proxy-group construct: even if groups are real, the evaluation would still be inflated if member histories are not masked at inference time.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLMGPR, a group POI recommendation framework built on Llama3-8b. It introduces semantic-enhanced POI tokens initialized by prompting the LLM, a QLoRA sequencing adapter that maps check-in sequences to vector representations, an aggregation adapter that combines individual member representations into a group representation, and a self-supervised purpose-prediction task with 11 LLM-generated purpose labels. The framework is evaluated on Foursquare, Weeplace, and Gowalla under a leave-one-out next-POI protocol, reporting improvements over six group baselines and seven individual baselines, plus ablations and cold-start experiments.","tokens_in":18945,"tokens_out":5686,"duration_ms":60605,"significance":"If the reported results survive a corrected evaluation protocol, this would be a worthwhile contribution: it is one of the first LLM-based frameworks for group POI recommendation, the adapter architecture is sensible and clearly separates sequence encoding from preference aggregation, and the authors include ablations that isolate the contribution of each component. I also credit the authors for evaluating against multiple strong group baselines and for checking cold-start performance. The main uncertainties are the unresolved temporal masking of member histories in the group evaluation, the absence of variance or significance reporting, and the co-presence proxy used to define groups; each of these can be addressed in revision without changing the overall architecture.","major_comments":[{"comment":"The evaluation protocol does not specify whether member check-in sequences X_u are truncated at the group's held-out timestamp. Because group activities are constructed from members' individual check-ins (Section 4.1), the member sequences used in lines 20-22 can contain the very POI that serves as the group's ground-truth next POI. The aggregation adapter is then trained and evaluated with the answer present in its input, which alone could explain the 4.4-9.0% improvements over MICL in Table 2. Please state explicitly how member histories are masked at both training and test time; if they are not masked, the experiments must be rerun with a temporal split that removes the group test POI, and any later check-ins, from each member's input sequence before the comparison.","section":"Section 4.1; Algorithm 1 (lines 19-26)"},{"comment":"All reported numbers are single point estimates with no standard deviations, confidence intervals, or significance tests. The margins over the best baseline are small in several cells (e.g., HR@5 0.3182 vs 0.3068 on Foursquare), so the abstract's claim of 'significantly' enhanced accuracy is not supported by the evidence as presented. Please report means and standard deviations over multiple independent runs and, if appropriate, paired significance tests for the headline comparisons.","section":"Section 4.3; Tables 2-5"},{"comment":"The group ground truth in all datasets is defined as sets of socially connected users who visit the same venue at the same time, i.e., co-presence, not validated group decisions. Since this proxy is used for every dataset and every experiment, the reported improvements may characterize co-presence prediction rather than real group outing recommendation. Please add an explicit discussion of this construct-validity limitation and, ideally, a sanity analysis (e.g., filtering to repeated co-visits or a small human-labeled sample of intentional group events) to show the findings transfer to genuine group decisions.","section":"Section 4.1"}],"minor_comments":[{"comment":"The baseline descriptions for GroupIM and CubeRec are identical; the GroupIM entry should describe its mutual-information maximization framework rather than the hypercube-based representation attributed to CubeRec.","section":"Section 4.2"},{"comment":"The training sets D, D_ssl, and D_group are not all formally defined before first use; please add explicit definitions in the notation section or at the start of Section 3.4.","section":"Section 3.4; Algorithm 1"},{"comment":"The variant name LLMGPR-FT is confusing because the caption and text describe it as 'using the LLM directly without fine-tuning'; consider renaming the variants to avoid the apparent contradiction between 'FT' and 'without fine-tuning'.","section":"Section 4.5; Table 4"},{"comment":"The subplot labels and axis captions are missing or too small to read; please provide a clearer figure with labeled axes and a legend.","section":"Figure 3"},{"comment":"Reference [35] attributes Llama to Touvron et al., but the paper uses Llama3-8b; please verify that the citation corresponds to the correct model release.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The decisive issue is the temporal masking of member sequences in the group evaluation. If the authors cannot demonstrate that member histories were truncated at the group's held-out timestamp, the headline group-recommendation results in Tables 2 and 4 are not interpretable. The other issues (variance reporting and construct validity) are important but secondary. I recommend major revision rather than rejection because the protocol can in principle be corrected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimately new combination—first LLM-based group POI recommender with semantic POI tokens, separate sequencing/aggregation adapters, and a purpose SSL task. The empirical work is broad: three datasets, six group baselines, seven individual baselines, ablations, cold-start. If the numbers hold, it's a solid CIKM paper.\n\nThe soft spot is the evaluation. The leave-one-out protocol is described for \"each check-in sequence,\" but Algorithm 1 feeds member check-in sequences X_u into the aggregation adapter without specifying that those sequences are truncated at the group's held-out timestamp. Since groups are constructed from co-occurring check-ins, each member's history contains the same POI as the group's test ground truth. If X_u includes the answer, the aggregation adapter can learn to copy it, and the 4–9% gains over MICL would reflect leakage, not group modeling. The paper doesn't address this. That's not a minor omission; it calls the headline result into question.\n\nAlso, the group definition is a co-visit heuristic, so the task is \"co-presence of social connections,\" not independently validated group outings. That's a validity issue but less fatal than the leakage. Plus no code, no error bars, no significance tests, and purpose labels are LLM-generated with only partial manual check.\n\nThe components themselves are sensible and the ablations show each one helps. I buy the design. The problem is the evaluation protocol, not the architecture.\n\nRecommendation: this deserves a serious referee, but only conditionally. The authors need to state explicitly that member sequences are masked at test time, or fix the protocol, release code/data, and add variance estimates. If the leakage is real, the gains may shrink or vanish; if it's not, the paper is likely fine. Right now I wouldn't accept without seeing the fix.","headline":"A genuinely new LLM-based group POI recommender with consistent gains, but a possible temporal leakage in the evaluation protocol needs to be resolved before those gains are credible.","tokens_in":19490,"tokens_out":2758,"would_cite":false,"duration_ms":29990,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLMGPR uses a fine-tuned large language model to recommend next places for groups, beating specialist group recommenders by 4–9%.","keywords":["group POI recommendation","large language models","QLoRA","self-supervised learning","next POI prediction","preference aggregation","location-based social networks","semantic-enhanced POI tokens"],"falsifier":"Construct or obtain a test set of independently verified group outings (e.g., groups that explicitly visited together, or user-declared sessions), run LLMGPR and MICL on that test set, and check whether the 4–9% HR@k/NDCG@k advantage persists; if the gap vanishes or reverses, the benchmark result is an artifact of the co-location group definition.","tokens_in":18514,"feed_emoji":"📍","tokens_out":2363,"duration_ms":25201,"temperature":0.7,"pith_summary":"This paper argues that large language models (LLMs), properly adapted, can outperform dedicated group point-of-interest (POI) recommenders. The proposed framework, LLMGPR, turns POIs into semantic tokens, fine-tunes an LLM with a QLoRA adapter to encode check-in sequences, and adds a second adapter that aggregates individual members' representations into a group representation. A self-supervised task predicting the purpose of a check-in sequence (business trip, tourism, etc.) supplies extra supervision to counter the extreme sparsity of group check-in data. If the claims hold, LLMGPR improves next-POI ranking accuracy over the best existing group recommender by average margins of 7.09% on Foursquare, 4.39% on Weeplace, and 9.01% on Gowalla, and also beats individual-level LLM recommenders on sequence representation quality.","feed_headline":"LLM-based group POI recommender beats specialists by 4–9%","feed_subtitle":"Semantic POI tokens, QLoRA adapters, and purpose-prediction self-supervision overcome sparse group check-in data.","key_machinery":"The central machinery is the semantic-enhanced POI token, a special token whose embedding is initialized by prompting a pre-trained LLM with a POI's name, category, description, reviews, coordinates, and address, then averaged from the final-layer token embeddings. These POI tokens are combined with word tokens and spatio-temporal difference signals, processed by Llama3-8b with a QLoRA sequencing adapter to produce check-in sequence representations. A second QLoRA aggregation adapter, fed with the set of member representations, produces an aggregated member representation that is added to the group sequence representation with a weight alpha. A purpose-prediction self-supervised task pre-trains the sequencing adapter and POI embeddings using 11 LLM-generated purpose labels for short sequences, giving the model semantic supervision beyond the sparse group check-ins.","core_discovery":"The central claim is that group POI recommendation can be reframed as a sequence-understanding and preference-aggregation problem solvable by a large language model, provided the model is given three components: POI tokens whose embeddings encode semantic text about each place, a QLoRA sequencing adapter that aligns the frozen LLM with check-in sequence representation, and a separate aggregation adapter that combines member-level sequence representations into an enhanced group representation. The paper further claims that a self-supervised purpose-prediction task, trained on LLM-generated labels for short check-in sequences, substantially enriches these representations and mitigates group-level data sparsity. All evidence is empirical: held-out next-POI ranking accuracy on three public LBSN datasets, where LLMGPR outperforms the strongest group baseline MICL and the strongest individual LLM baseline LLM4POI across HR@k and NDCG@k.","pith_inferences":["Because the group labels are constructed by co-location of socially connected users, the benchmark may reward models that predict co-presence patterns rather than genuine group decision-making; an independent validation with explicitly declared group outings would be needed to confirm transfer to real settings.","The semantic POI token initialization from LLM text embeddings is a general technique that could be applied to other ID-based recommenders to relieve cold-start, independent of the group setting.","The purpose-prediction supervision relies on LLM-generated labels, whose 92% agreement with manual labels was checked on only 20% of sequences; extending that manual check to all sequences or to a held-out city would test whether the SSL signal is robust.","The aggregation adapter's treatment of members as an unordered set of representations could be extended to model member roles or influence asymmetries, which the current averaging-plus-residual design does not explicitly capture."],"forward_implications":["If LLMGPR's gains are real, group POI recommenders no longer need to rely only on sparse group check-in frequencies; semantic and contextual signals from individual check-ins can be transferred through the LLM and its adapters.","The purpose-prediction SSL task provides a reusable way to inject latent trip intent into sequence representations, which could improve next-POI accuracy in cold-start and low-data regimes beyond the group setting.","The architecture's separation of a frozen LLM, a sequencing adapter, and an aggregation adapter means the framework can be swapped to any base LLM with sufficient resources, not only Llama3-8b.","The reported cold-start experiments suggest the method degrades less than baselines when groups have fewer than ten check-ins, implying practical value for real deployment where new groups are common.","The approach is the first LLM-based group POI recommender, so it sets a new baseline for future work combining LLMs with group preference aggregation."],"supporting_citations":[{"why":"Supplies the QLoRA method used to fine-tune the frozen LLM with low-rank adapters, the core alignment mechanism for both sequencing and aggregation.","marker":"[6]"},{"why":"Provides Llama3-8b, the base LLM that the framework modifies and fine-tunes.","marker":"[35]"},{"why":"LLM4POI is the strongest individual-level LLM baseline that LLMGPR compares against and outperforms on sequence representation quality.","marker":"[19]"},{"why":"MICL is the strongest group baseline, and the reported 4–9% average improvements over it carry the paper's central empirical claim.","marker":"[18]"},{"why":"CubeRec's group-construction procedure (socially connected users co-visiting the same venue) is adopted to build all three group datasets.","marker":"[3]"},{"why":"LLMMove represents the zero-shot LLM approach for next-POI recommendation, serving as a baseline that motivates the need for fine-tuning adapters.","marker":"[7]"}],"fun_headline_variants":["LLM group POI model tops baselines by 4-9%","Frozen LLM + QLoRA beats group POI specialists","LLMGPR: LLMs with adapters rule group POI","Semantic POI tokens help LLMs pick spots for groups","Purpose-prediction self-supervision powers LLM group picks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a group is correctly defined as socially connected users who happen to visit the same venue at the same time; if co-location is often coincidence rather than a shared plan, the model is trained on a proxy for group behavior and the benchmark gains may not reflect real group outings.","fun_headline_variants_meta":{"raw":{"variants":["LLM group POI model tops baselines by 4-9%","Frozen LLM + QLoRA beats group POI specialists","LLMGPR: LLMs with adapters rule group POI","Semantic POI tokens help LLMs pick spots for groups","Purpose-prediction self-supervision powers LLM group picks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00014,"raw_usage":{"total_tokens":1177,"prompt_tokens":981,"completion_tokens":196,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":104}},"tokens_in":597,"tokens_out":196,"duration_ms":2642,"temperature":1.0,"reasoning_tokens":104,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:26:15.980862+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct or obtain a test set of independently verified group outings (e.g., groups that explicitly visited together, or user-declared sessions), run LLMGPR and MICL on that test set, and check whether the 4–9% HR@k/NDCG@k advantage persists; if the gap vanishes or reverses, the benchmark result is an artifact of the co-location group definition.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MICL is the strongest group baseline, and the reported 4–9% average improvements over it carry the paper's central empirical claim."}],"review_version":1}