pith. sign in

arxiv: 2606.26277 · v1 · pith:EPFXENX7new · submitted 2026-06-24 · 💻 cs.IR · cs.AI· cs.CE· cs.CL· cs.LG

From Clicks to Intent: Cross-Platform Session Embeddings with LLM-Distilled Taxonomy for Financial Services Recommendations

Dianjing Fan , Yao Li , Kyaw Hpone Myint , Dwipam Katariya , Alexandre G.R. Day , Pranab Mohanty , Giri Iyengar This is my paper

Pith reviewed 2026-06-26 00:59 UTC · model grok-4.3

classification 💻 cs.IR cs.AIcs.CEcs.CLcs.LG
keywords session embeddingsintent predictionrecommender systemsfinancial servicesLLM distillationclickstream modelingcross-platform personalizationtransformer embeddings
0
0 comments X

The pith

Self-supervised embeddings from anonymous web clicks improve authenticated financial app recommendations by 1.88 percent recall and 13.38 percent log loss.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows how raw web clickstreams can be turned into compact session embeddings via a self-supervised Transformer while an LLM pipeline distills them into readable intent labels. These two outputs together support both ranking tasks in a logged-in mobile app and human-interpretable explanations, even though no direct link exists between anonymous web visitors and authenticated accounts. The work targets the specific mismatch in financial services where web users browse new products and app users manage existing accounts. Gains appear on production metrics for homepage tile ranking and conversion prediction, with the distilled labels running at low latency. Readers care because the method extracts usable intent from the cheaper, pre-login channel without requiring cross-channel identity matching.

Core claim

A self-supervised Transformer encodes multi-modal web clickstreams into compact session embeddings while a parallel LLM-based taxonomy generation and distillation pipeline yields interpretable intent labels; together these outputs improve macro Recall@1 by 1.88 percent and reduce log loss by 13.38 percent on mobile homepage tile ranking, outperform LLM labels by 4.3 percent micro F1 on conversion prediction, and deliver the labels at ultra-low latency with only a 7 percent performance drop.

What carries the argument

Self-supervised Transformer that encodes multi-modal clickstreams into session embeddings, paired with an LLM taxonomy generation and distillation pipeline that produces interpretable intent labels.

If this is right

  • Session embeddings raise macro Recall@1 by 1.88 percent and cut log loss by 13.38 percent on mobile homepage tile ranking.
  • The same embeddings outperform standalone LLM labels by 4.3 percent micro F1 on user conversion prediction.
  • The distillation layer supplies human-readable intent labels at ultra-low latency while losing only 7 percent of the embedding's performance.
  • Web intent signals become usable for post-authentication personalization without cross-channel entity resolution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same two-output pipeline could be tested on non-financial domains that also separate anonymous browsing from logged-in servicing.
  • Because the method avoids entity resolution it may reduce privacy risk when moving web signals into app models.
  • Further downstream tasks such as churn or upsell prediction could be evaluated to map the full range of transferable signals.
  • Replacing the current LLM step with a smaller distilled model might lower cost while preserving most of the label quality.

Load-bearing premise

Intent signals learned from unauthenticated web clickstreams remain useful for predicting behavior of authenticated app users even when the same people cannot be matched across channels.

What would settle it

Measure whether the session embedding still improves ranking metrics when web sessions are restricted to users who never log in versus users who do log in after the session.

Figures

Figures reproduced from arXiv: 2606.26277 by Alexandre G.R. Day, Dianjing Fan, Dwipam Katariya, Giri Iyengar, Kyaw Hpone Myint, Pranab Mohanty, Yao Li.

Figure 1
Figure 1. Figure 1: System architecture overview. Left (Section 3.2): multi-modal clickstream events are fused and encoded into session [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
read the original abstract

Sequential user behavior modeling is widely adopted in industrial recommender systems; however, significant gaps remain in financial services, where pre-login web interactions and authenticated in-app experiences differ drastically. Specifically, pre-login web users typically explore new products, whereas logged-in app users focus on account servicing. Due to the challenge of cross-channel entity resolution (e.g., matching anonymous web sessions to authenticated mobile accounts), web-based intent signals remain underutilized for post-authentication personalization. Existing methods for capturing web-based intent are often ad-hoc and narrow, lacking the flexibility to support both quantitative downstream recommendations and qualitative understanding at scale. In this work, we propose a scalable and dual-purpose intent prediction framework for web-based interactions and demonstrate its applicability for personalization. Our approach transforms raw web clickstreams into two outputs: a self-supervised Transformer encodes multi-modal clickstreams into a compact session embedding, while an LLM-based taxonomy generation and distillation pipeline produces interpretable intent labels. Our system demonstrates that self-supervised clickstream representations combined with LLM-distilled taxonomies can jointly serve quantitative tasks and qualitative understanding in production: on the mobile homepage tile ranking task, the session embedding improves macro Recall@1 by 1.88% and reduces Log Loss by 13.38% over production baselines. On the user conversion prediction task, the embedding outperforms the LLM labels by 4.3% on micro F1, while the distillation layer delivers interpretable labels at ultra-low latency with only a 7% performance drop.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a dual-purpose framework for financial services recommendations that encodes pre-login web clickstreams into self-supervised Transformer session embeddings and uses an LLM-based taxonomy generation plus distillation pipeline to produce interpretable intent labels. It claims these outputs enable personalization for authenticated mobile app users despite absent cross-channel entity resolution, with reported gains of +1.88% macro Recall@1 and -13.38% Log Loss on homepage tile ranking over production baselines, plus +4.3% micro F1 on conversion prediction (with only 7% drop from full LLM labels at low latency).

Significance. If the transfer results hold under rigorous controls, the work would provide a practical path to leverage anonymous web intent signals for post-auth app experiences in regulated domains like financial services, where entity resolution is often infeasible. The combination of compact embeddings for quantitative ranking and distilled labels for qualitative understanding at production latency is a notable strength for industrial deployment.

major comments (2)
  1. [Abstract] Abstract: The headline performance numbers (macro Recall@1 +1.88%, Log Loss -13.38% on tile ranking; +4.3% micro F1 on conversion) are presented without any description of experimental setup, baseline definitions, data splits, statistical significance tests, or how web embeddings are aligned to mobile-app evaluation users. This is load-bearing for the central empirical claims.
  2. [Evaluation] Evaluation section: The transferability premise—that web-derived session embeddings capture intent signals useful for authenticated app users—is asserted but not directly tested. The manuscript states cross-channel entity resolution is absent, yet provides no account of whether evaluation uses population-level distributional similarity, a matched subset, or controls for selection bias/domain shift between anonymous web explorers and logged-in app users. This leaves the applicability to mobile tasks unverified.
minor comments (2)
  1. [§3.3] The description of the distillation layer latency and the 7% performance drop would benefit from an explicit comparison table against the full LLM and the embedding-only variants.
  2. [§3.1] Notation for the multi-modal clickstream input to the Transformer (e.g., how different event types are tokenized) is introduced without a formal definition or diagram.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the referee's detailed feedback. We address each major comment below and will revise the manuscript to improve clarity on experimental details and evaluation methodology.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The headline performance numbers (macro Recall@1 +1.88%, Log Loss -13.38% on tile ranking; +4.3% micro F1 on conversion) are presented without any description of experimental setup, baseline definitions, data splits, statistical significance tests, or how web embeddings are aligned to mobile-app evaluation users. This is load-bearing for the central empirical claims.

    Authors: We agree that the abstract lacks sufficient context for the reported metrics. In the revised manuscript, we will expand the abstract to include a concise description of the experimental setup, including data characteristics, baseline definitions, evaluation splits, and a high-level note on cross-channel alignment. References to statistical significance testing will be added in the main text. revision: yes

  2. Referee: [Evaluation] Evaluation section: The transferability premise—that web-derived session embeddings capture intent signals useful for authenticated app users—is asserted but not directly tested. The manuscript states cross-channel entity resolution is absent, yet provides no account of whether evaluation uses population-level distributional similarity, a matched subset, or controls for selection bias/domain shift between anonymous web explorers and logged-in app users. This leaves the applicability to mobile tasks unverified.

    Authors: This comment correctly identifies a gap in the current description. The evaluation applies web-derived embeddings to mobile homepage ranking for users with web interaction history, relying on population-level overlap rather than individual resolution. To address selection bias and domain shift, we will add a dedicated subsection in the Evaluation section detailing the data filtering, any distributional matching applied, controls used, and explicit limitations of the transfer setting. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical comparisons to external baselines.

full rationale

The paper reports empirical gains on mobile/app tasks (Recall@1 +1.88%, LogLoss -13.38%, micro F1 +4.3%) from web-derived session embeddings and LLM-distilled labels, framed as comparisons against production baselines. No self-definitional loops, fitted inputs renamed as predictions, or load-bearing self-citations appear in the provided text. The transferability claim rests on distributional similarity rather than a closed mathematical derivation that reduces to its own inputs. The derivation chain is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available; no information on free parameters, axioms, or invented entities can be extracted or assessed.

pith-pipeline@v0.9.1-grok · 5844 in / 1037 out tokens · 25082 ms · 2026-06-26T00:59:21.521595+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 2 canonical work pages

  1. [1]

    Bernhard, Carson K

    Shelby D. Bernhard, Carson K. Leung, Vanessa J. Reimer, and Joshua Westlake

  2. [2]

    InProceedings of the 20th International Database Engineering & Applications Symposium(Montreal, QC, Canada)(IDEAS ’16)

    Clickstream Prediction Using Sequential Stream Mining Techniques with Markov Chains. InProceedings of the 20th International Database Engineering & Applications Symposium(Montreal, QC, Canada)(IDEAS ’16). Association for Computing Machinery, New York, NY, USA, 24–33. doi:10.1145/2938503.2938535

    work page doi:10.1145/2938503.2938535

  3. [3]

    William Black, Alexander Manlove, Jack Pennington, Andrea Marchini, Ercu- ment Ilhan, and Vilda Markeviciute. 2024. TRACE: Transformer-based user Representations from Attributed Clickstream Event sequences

    2024

  4. [4]

    David M Blei, Thomas L Griffiths, and Michael I Jordan. 2010. The nested chinese restaurant process and bayesian nonparametric inference of topic hierarchies. Journal of the ACM (JACM)57, 2 (2010), 1–30

    2010

  5. [5]

    Andrei Broder. 2002. A taxonomy of web search. 36, 2 (2002), 3–10

    2002

  6. [6]

    Qiwei Chen, Huan Zhao, Wei Li, Pipei Huang, and Wenwu Ou. 2019. Behavior sequence transformer for e-commerce recommendation in alibaba. InProceedings of the 1st international workshop on deep learning practice for high-dimensional sparse data. 1–4

    2019

  7. [7]

    Yongjun Chen, Zhiwei Liu, Jia Li, Julian McAuley, and Caiming Xiong. 2022. Intent contrastive learning for sequential recommendation. InProceedings of the ACM web conference 2022. 2172–2182

    2022

  8. [8]

    Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023. ChatGPT outperforms crowd workers for text-annotation tasks.Proceedings of the National Academy of Sciences120, 30 (2023), e2305016120

    2023

  9. [9]

    Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk

  10. [10]

    InPro- ceedings of the 4th International Conference on Learning Representations (ICLR)

    Session-based recommendations with recurrent neural networks. InPro- ceedings of the 4th International Conference on Learning Representations (ICLR)

  11. [11]

    Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. 2023. Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. InFindings of the Association for Computational Linguistics: ACL 2023. 8003–8017

    2023

  12. [12]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recom- mendation. In2018 IEEE international conference on data mining (ICDM). IEEE, 197–206

    2018

  13. [13]

    Dwipam Katariya, Juan Manuel Origgi, Yage Wang, and Thomas Caputo. 2024. Timesync: Temporal intent modelling with synchronized context encodings for financial service applications.arXiv preprint arXiv:2410.12825

    arXiv 2024

  14. [14]

    Yong Soo Kim and Bong-Jin Yum. 2011. Recommender system based on click stream data using association rule mining.Expert Syst. Appl.38, 10 (Sept. 2011), 13320–13327. doi:10.1016/j.eswa.2011.04.154

    work page doi:10.1016/j.eswa.2011.04.154 2011

  15. [15]

    Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Huan Zhao, Pipei Huang, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. 2019. Multi-interest network with dynamic routing for recommendation at Tmall. InProceedings of the 28th ACM international conference on information and knowledge management. 2615–2623

    2019

  16. [16]

    Yueqing Liang, Liangwei Yang, Chen Wang, Xiongxiao Xu, Philip S Yu, and Kai Shu. 2025. Taxonomy-guided zero-shot recommendations with llms. (2025), 1520–1530

    2025

  17. [17]

    Erdi Olmezogullari and Mehmet S. Aktas. 2020. Representation of Click-Stream DataSequences for Learning User Navigational Behavior by Using Embeddings. In2020 IEEE International Conference on Big Data (Big Data). 3173–3179. doi:10. 1109/BigData50022.2020.9378437

    arXiv 2020

  18. [18]

    Chau Minh Pham, Alexander Hoyle, Simeng Sun, Philip Resnik, and Mohit Iyyer

  19. [19]

    InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)

    TopicGPT: A prompt-based topic modeling framework. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2956–2984

    2024

  20. [20]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). 3982–3992

    2019

  21. [21]

    Chirag Shah, Ryen White, Reid Andersen, Georg Buscher, Scott Counts, Sarkar Das, Ali Montazer, Sathish Manivannan, Jennifer Neville, Nagu Rangan, et al

  22. [22]

    Using large language models to generate, validate, and apply user intent taxonomies.ACM Transactions on the Web19, 3, 1–29

  23. [23]

    Alex Stein, Samuel Sharpe, Doron Bergman, Senthil Kumar, C Bayan Bruss, John Dickerson, Tom Goldstein, and Micah Goldblum. 2024. A simple baseline for predicting events with auto-regressive tabular transformers.arXiv preprint arXiv:2410.10648(2024)

    arXiv 2024

  24. [24]

    Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang

  25. [25]

    InProceedings of the 28th ACM international conference on information and knowledge management

    BERT4Rec: Sequential recommendation with bidirectional encoder rep- resentations from transformer. InProceedings of the 28th ACM international conference on information and knowledge management. 1441–1450

  26. [26]

    Mengting Wan, Tara Safavi, Sujay Kumar Jauhar, Yujin Kim, Scott Counts, Jennifer Neville, Siddharth Suri, Chirag Shah, Ryen W White, Longqi Yang, et al. 2024. Tnt- llm: Text mining at scale with large language models. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5836–5847

    2024

  27. [27]

    Gang Wang, Xinyi Zhang, Shiliang Tang, Haitao Zheng, and Ben Y Zhao. 2016. Unsupervised clickstream clustering for user behavior analysis. InProceedings of the 2016 CHI conference on human factors in computing systems. 225–236

    2016

  28. [28]

    Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng

  29. [29]

    InFindings of the Association for Computational Linguistics: EMNLP 2021

    Want to reduce labeling cost? GPT-3 can help. InFindings of the Association for Computational Linguistics: EMNLP 2021. 4195–4205

    2021

  30. [30]

    Zihan Wang, Jingbo Shang, and Ruiqi Zhong. 2023. Goal-driven explainable clustering via language descriptions. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 10626–10649

    2023

  31. [31]

    Alexander Wettig, Kyle Lo, Sewon Min, Hannaneh Hajishirzi, Danqi Chen, and Luca Soldaini. 2025. Organize the web: Constructing domains enhances pre- training data curation.arXiv preprint arXiv:2502.10341

    arXiv 2025

  32. [32]

    Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and An- drew Zhai. 2023. Transact: Transformer-based realtime user action model for recommendation at pinterest. InProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5249–5259

    2023

  33. [33]

    Chao Zhang, Fangbo Tao, Xiusi Chen, Jiaming Shen, Meng Jiang, Brian Sadler, Michelle Vanni, and Jiawei Han. 2018. Taxogen: Unsupervised topic taxonomy construction by adaptive term embedding and clustering. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 2701–2709

    2018

  34. [34]

    Yuwei Zhang, Zihan Wang, and Jingbo Shang. 2023. Clusterllm: Large language models as a guide for text clustering. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 13903–13920

    2023

  35. [35]

    Han Zhu, Xiang Li, Pengye Zhang, Guozheng Li, Jie He, Han Li, and Kun Gai

  36. [36]

    InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining

    Learning tree-based deep model for recommender systems. InProceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1079–1088