Pith. sign in

REVIEW 4 major objections 6 minor 39 references

TADT-CSA: Temporal Advantage Decision Transformer with Contrastive State Abstraction for Generative Recommendation

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TADT-CSA claims that a Decision Transformer for generative recommendation can be made both more accurate and more scalable by conditioning on a temporal advantage signal alongside return-to-go, and by compressing the high-dimensional…

desk verdict Live A/B results are the real story, but Theorem 1's independent-of-|S| bound has a genuine proof gap that needs fixing. read the letter →

arxiv 2507.20327 v2 pith:RP7MYH5L submitted 2025-07-27 cs.IR

Xiang Gao , Tianyuan Liu , Yisha Li , Jingxin Liu , Lexi Gao , Xin Li , Haiyang Lu , Liyin Hong This is my paper · ORCID
classification cs.IR
keywords temporaladvantagedecisiontransformergenerativerecommendationcontrastivestateabstractionvectorquantizationofflinereinforcementlearningtrajectorystitchingreturn-to-go
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a Decision Transformer can be made competitive for large-scale generative recommendation if it is conditioned on a temporal advantage signal as well as return-to-go, and if its state representation is aggressively compressed through a vector-quantized codebook guided by reward and transition prediction. The proposed model, TADT-CSA, claims improvements over existing DT-based, sequential, and TD-learning recommenders in offline evaluation, an online simulation, and a live A/B test. The reason a sympathetic reader should care is practical: industrial recommender state spaces are enormous and noisy, so a method that both shrinks the state space and preserves enough MDP structure to keep near-optimal value would make offline RL policies deployable at scale. The paper also offers a formal guarantee that the value loss from its state abstraction is bounded independently of the original state-space size.

What carries the argument

The load-bearing object is the Contrastive State Abstraction (CSA) module, specifically its TA-conditioned State Vector Quantization (TAC-SVQ): encoder outputs $e_t$ are compared to codebook vectors $c_i$ with similarity $z(e_t,c_i,R^{TA}_t)=\alpha c_i^\top e_t + (1-\alpha)c_i^\top R^{TA}_t$, and Gumbel-Softmax turns the resulting distribution into a differentiable one-hot assignment. The auxiliary Reward Prediction network and Contrastive Transition Prediction network are what justify calling the compression an MDP abstraction: the reward predictor enforces that $c_t,a_t$ can reproduce $r_t$, and the contrastive transition predictor enforces that $(c_t,a_t,c_{t+1})$ is distinguishable from negative transitions. These two networks feed the error terms $\varepsilon_r$ and $\varepsilon_P$ in Theorem 1, which is what makes the bound independent of the original state space size.

What would settle it

Train TADT-CSA on a small MDP with known transition probabilities, then compare the CTP network's predicted distribution over codebook indices, $\hat{P}(c_{t+1}|c_t,a_t)$, to the true aggregated distribution $P(c_{t+1}|c_t,a_t)=\sum_{s'\in f^{-1}(c_{t+1})}P(s'|s_t,a_t)$ on held-out states; a large divergence would show the bound's key assumption fails. Alternatively, measure the actual value gap $V^{\pi^*}(s)-V^{\pi_\Theta}(s)$ and check whether it ever exceeds the right-hand side of Theorem 1.

Watch

Extended reading notes

Core claim

The central claim is that the weaknesses of DT for recommendation—trajectory stitching failures and poor handling of huge, sparse state spaces—can be addressed together. TADT-CSA augments the return-to-go $R^{RTG}_t$ with a temporal advantage score $R^{TA}_t = \sum_{i=2}^t \gamma^{t-i}(R^{RTG}_i - R^{RTG}_{i-1})$, so the conditioning signal carries trend information; adds a quantile-based pairwise ranking loss so training goes beyond behavior cloning; and compresses states via a TA-conditioned vector quantizer with reward-prediction and contrastive transition-prediction auxiliary tasks. Theorem 1 states that the value gap between the optimal policy and the abstracted policy is at most $2/(1-\gamma)^2(\varepsilon_r + \kappa I^{(d+2)/(2d)} |C|^{-1/d} + \gamma \varepsilon_P |C|/(1-\gamma))$, so the error depends on the codebook size, the reward and transition prediction errors, and the concentration of state embeddings, but not on $|S|$.

Load-bearing premise

The bound holds only if the trained transition-prediction network actually approximates the true aggregated transition probabilities in the compressed space (Eq. 33) and if the reward and Q-functions are Lipschitz in the embedding space, neither of which the training procedure guarantees or verifies.

Editorial extensions

If this is right

  • Offline RL recommenders can run with drastically smaller state representations: codebook size, not user or item population, drives the representation cost.
  • The temporal advantage signal gives the policy a gradient that distinguishes trajectories with equal return-to-go but different trends, which should help in stochastic, noisy environments.
  • The pairwise ranking loss makes the DT objective an implicit policy-improvement step, so the model can stitch high-return actions from suboptimal trajectories rather than merely cloning behavior.
  • If the bound holds in practice, the abstraction error can be controlled by choosing the codebook size $|C|$ and by reducing reward and transition prediction errors, providing a design rule for deploying DT policies in production recommenders.
  • The reported online A/B results indicate that the method improves live-stream watch time and average watch time over a TD-learning baseline and over that baseline enhanced with the same CSA module.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the same state-abstraction recipe (reward prediction plus contrastive transition prediction on a vector-quantized codebook) could be lifted from recommendation to other offline RL settings with large state spaces, such as dialogue or robotics, and the same bound would argue for its safety.
  • The proof's reliance on the accuracy of the learned transition predictor (Eq. 33) suggests that the practical gains may come largely from representation learning rather than from the bound itself; a direct diagnostic would be to compare the CTP network's predicted transition distribution to the empirical aggregated distribution on held-out data.
  • The bound suggests an optimal codebook size: increasing $|C|$ reduces the quantization term $|C|^{-1/d}$ but inflates the transition-error term $|C|$. Sweeping codebook size on a fixed dataset could reveal a sweet spot that the current parameter sensitivity analysis does not explore.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes TADT-CSA, a Decision Transformer variant for generative recommendation that combines a Return-to-Go signal with a novel Temporal Advantage (TA) score, a pairwise ranking loss to mitigate behavior cloning, and a Contrastive State Abstraction (CSA) module that vector-quantizes state embeddings under TA conditioning. The CSA module includes reward prediction and contrastive transition prediction auxiliary networks, and the paper provides a theoretical bound (Theorem 1) claiming that the value loss of the abstracted policy is independent of the original state space size. Evaluation is reported on four public datasets, on the VirtualTaobao online simulator, and in a live A/B test on the Kuaishou platform, where TADT-CSA is claimed to outperform DT-based and RL baselines.

Significance. If the theoretical and empirical claims were fully supported, the paper would be a solid systems-oriented contribution: the TA signal and the CSA module are reasonable engineering ideas, and the evaluation spans offline data, a simulator, and a live system, which is unusually comprehensive. The paper also includes a computational-cost comparison and a trajectory-stitching study, both of which strengthen the practical message. However, the central theoretical claim — that CSA preserves MDP structure with an error bound independent of |S| — is not established: the proof of Theorem 1 rests on a definitionally inconsistent transition-error term and an unverified approximation that the CTP network does not train toward. The empirical claims, while suggestive, would be much stronger with variance reporting and significance tests. On balance the work is potentially publishable after substantial revision, but the theoretical and statistical foundations need real work before the paper's headline claims can be accepted.

major comments (4)
  1. [Theoretical Analysis, Eq. (20); Appendix Proof, Eq. (33)] The definition of εP conflates per-state transition probabilities with per-codebook class probabilities. Equation (20) defines εP = max_{s,a,s'} |P(s,a,s') − P_hψ(fθ(s), a, fθ(s'))|, where P(s,a,s') is the probability of the exact next state s', while P_hψ outputs a probability over codebook classes. When two reachable next states s1 and s2 map to the same codebook entry c', a perfect abstract predictor assigns P(s1|s,a) + P(s2|s,a) to that class, so the per-state error for s1 is P(s2|s,a) > 0. Thus εP is bounded away from zero in exactly the stochastic multi-state-to-one-code setting the paper targets, and the term γ εP |C|/(1−γ) in Theorem 1 is not controlled. The claimed independence from |S| is therefore not established by the stated theorem.
  2. [Appendix Proof, Eqs. (31)–(33)] The approximation in Eq. (33), P̂(c_{t+1}|c_t,a_t) ≈ P(c_{t+1}|c_t,a_t), is asserted but not guaranteed by any training objective. The CTP network is trained with the InfoNCE loss in Eq. (16), which estimates density ratios and does not minimize a divergence to the aggregated transition distribution P(c_{t+1}|c_t,a_t). Consequently the bound on term (b2) in Eq. (32) has no foundation, and the theorem's transition-error term is not tied to a quantity the proposed architecture actually optimizes. The Lipschitz constants κr and κQ (Eqs. 27 and 34) are also assumed without estimation or validation, so the quantization term κI^{(d+2)/2d}|C|^{−1/d} is not connected to the trained network's actual behavior.
  3. [Table 1; Implementation Details] The offline evaluation reports a single run of each method per dataset with no standard deviations, confidence intervals, or significance tests. The text claims TADT-CSA 'consistently achieves the highest or near-highest scores,' but the largest gains over CDT4Rec are often small (e.g., MovieLens-20M MRR 0.4264 vs. 0.4201, Netflix NDCG@10 0.5879 vs. 0.5884 for the baseline), so without variance information these differences cannot be interpreted. The data preprocessing is also not described: how trajectories of length 30 are formed, how the 20-dimensional observation vector is derived, and how actions and rewards are defined for each dataset are all unspecified, which prevents replication.
  4. [Online A/B Experiments] The live A/B test reports percentage improvements (e.g., 2.830% and 15.307% over SAC-CSA) but gives no confidence intervals, significance levels, or details about the SAC and SAC-CSA baselines or the evaluation protocol beyond the 20% user split. Since SAC-CSA is introduced only in this section and is not described in the main method or related work, the reader cannot assess whether the comparison is fair or whether the observed differences are within normal day-to-day variation. The online result should be treated as anecdotal evidence rather than a verified claim.
minor comments (6)
  1. [Table 1] The second baseline column is labeled 'BEART4Rec' but the corresponding method in the text and related work is BERT4Rec; this typo appears in the table header.
  2. [Eq. (17)] The loss weights are listed as 'λ3, λ3 and λ3' in the equation; they should be λ3, λ4, and λ5.
  3. [Method, CSA module] The word 'trajactory' should be 'trajectory' in the sentence describing the token representation list.
  4. [Parameter Sensitivity] The sensitivity analysis covers codebook size, δ, α, and β, but not the loss weights λ1–λ5, which are free hyperparameters of the method; the paper should at least state that these were fixed or justify the choice.
  5. [Reproducibility Checklist] The reproducibility checklist is included but left unfilled; the authors should mark the applicable items (e.g., number of runs, preprocessing code, seeds) so that the checklist serves its purpose.
  6. [Online Simulation, Fig. 2] Figure 2 is referenced but its axis labels and the number of seeds or runs used to produce the curves are not described; adding this information would make the simulation result more interpretable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical claims are benchmark-based and the theoretical bound is conditional on network errors rather than self-derived.

full rationale

The paper's central empirical claims are evaluated against external baselines on public datasets (KuaiRand-Pure, MovieLens-20M, Netflix, RetailRocket), the VirtualTaobao simulator, and a deployed A/B test; no fitted parameter is renamed as a prediction. The TA signal is a deterministic linear transform of the RTG values (Eq. 3), but it is used as an auxiliary conditioning feature in the return token [RTG, TA]; this is a design choice, not a derivation whose conclusion is identical to its input. Theorem 1 is a conditional approximate-abstraction bound: it states that the value gap is controlled by epsilon_r, epsilon_P, codebook size |C|, a Lipschitz constant, and a quantization concentration term, citing external results (Abel et al. 2016; Zador 1982; Graf and Luschgy 2000). The bound does not reduce to its own definitions; it is a standard perturbation argument. There are no load-bearing self-citations: the Kuaishou-authored references (Liu et al. 2024a; Gao et al. 2025) are contextual and not used to justify the theoretical or empirical conclusions. The appendix proof has unproven steps (e.g., Eq. 33 assumes P_hat(c_{t+1}|c_t,a_t) is approximately the aggregated true transition, and Eq. 20's epsilon_P compares per-state probabilities to codebook-level predictions), but these are correctness/incompleteness concerns, not circular reductions: the paper does not define its target claim in terms of the CTP network's output and then relabel that output as a prediction. Therefore no circular step meeting the quoted-evidence standard is present.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

Free parameters are the main hyperparameters the central results depend on; none of them is derived from theory, and the paper does not report values for the lambda weights. The axioms are the unproven conditions used in the theoretical bound, most notably the Lipschitz assumptions and the approximate transition consistency. No invented entities are introduced: the temporal advantage score and codebooks are learned or constructed from data, not new physical or conceptual entities.

free parameters (5)
  • alpha
    Balances state embedding and TA signal in codebook similarity (Eq. 11). Selected by hand; sensitivity study in Fig. 4 shows MRR is not sensitive.
  • beta
    Quantile threshold for positive/negative split in pairwise ranking loss (Eq. 8). Parameter sensitivity reported in Fig. 4.
  • delta = 0.3
    Margin in pairwise ranking loss (Eq. 8). The authors report 0.3 gives the best MRR on KuaiRand.
  • lambda_weights
    Loss weights lambda1 through lambda5 in Eqs. 10 and 17. Values are not reported in the paper.
  • codebook_size = 64
    Number of codebook vectors; chosen via sensitivity experiments on KuaiRand.
assumptions (6)
  • ad hoc to paper The embedding distribution p(e) has compact support and p(e) >= rho_min > 0 on that support.
    Used in Lemma 1 to apply Zador's asymptotic quantization formula and the Graf-Luschgy covering radius bound.
  • ad hoc to paper The reward function r(s,a) is kappa_r-Lipschitz in the state embedding space (Eq. 27).
    Required in the proof of Theorem 1 to bound reward difference by embedding distance.
  • ad hoc to paper The optimal Q-function is kappa_Q-Lipschitz in the state embedding space (Eq. 34).
    Required in the proof to bound the transition term (b1).
  • ad hoc to paper The learned transition predictor satisfies P_hat(c_{t+1}|c_t,a_t) approximately equals P(c_{t+1}|c_t,a_t) = sum over s' in f^{-1}(c_{t+1}) of P(s'|s_t,a_t) (Eq. 33).
    Essential for bounding term (b1); the CTP network's output is assumed to match the true aggregated transition probabilities.
  • standard math Zador's asymptotic quantization formula and the covering radius result from Graf and Luschgy.
    Used in Lemma 1 to relate codebook size to covering radius.
  • standard math Lemma 2 from Abel et al. 2016 relating state abstraction errors to policy value loss.
    Used to translate the Q-value difference bound into the final value difference bound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TADT-CSA: Temporal Advantage Decision Transformer with Contrastive State Abstraction for Generative Recommendation." pith.science (2026). https://pith.science/paper/RP7MYH5L

@misc{pith2026250720327,
  author       = {Pith},
  title        = {Pith review of: TADT-CSA: Temporal Advantage Decision Transformer with Contrastive State Abstraction for Generative Recommendation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RP7MYH5L}},
  note         = {Machine review of arXiv:2507.20327}
}
read the original abstract

With the rapid advancement of Transformer-based Large Language Models (LLMs), generative recommendation has shown great potential in enhancing both the accuracy and semantic understanding of modern recommender systems. Compared to LLMs, the Decision Transformer (DT) is a lightweight generative model applied to sequential recommendation tasks. However, DT faces challenges in trajectory stitching, often producing suboptimal trajectories. Moreover, due to the high dimensionality of user states and the vast state space inherent in recommendation scenarios, DT can incur significant computational costs and struggle to learn effective state representations. To overcome these issues, we propose a novel Temporal Advantage Decision Transformer with Contrastive State Abstraction (TADT-CSA) model. Specifically, we combine the conventional Return-To-Go (RTG) signal with a novel temporal advantage (TA) signal that encourages the model to capture both long-term returns and their sequential trend. Furthermore, we integrate a contrastive state abstraction module into the DT framework to learn more effective and expressive state representations. Within this module, we introduce a TA-conditioned State Vector Quantization (TAC-SVQ) strategy, where the TA score guides the state codebooks to incorporate contextual token information. Additionally, a reward prediction network and a contrastive transition prediction (CTP) network are employed to ensure the state codebook preserves both the reward information of the current state and the transition information between adjacent states. Empirical results on both public datasets and an online recommendation system demonstrate the effectiveness of the TADT-CSA model and its superiority over baseline methods.

Figures

Figures reproduced from arXiv: 2507.20327 by the authors.

Figure 1
Figure 1. Overall framework of the TADT-CSA model. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Online Simulation result of all compared methods. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Parameter sensitivity evaluation of TADT-CSA. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Parameter sensitivity evaluation on α and β [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Trajectory stitching evaluation of different DT-based recommendation methods. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 14 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Abel, D.; Hershkowitz, D.; and Littman, M. 2016. Near optimal behavior via approximate state abstraction. In International Conference on Machine Learning, 2915--2923. PMLR

    work page 2016

  4. [4]

    A.; and Terry, M

    Bradley, R. A.; and Terry, M. E. 1952. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika, 39(3/4): 324--345

    1952

  5. [5]

    Brandfonbrener, D.; Bietti, A.; Buckman, J.; Laroche, R.; and Bruna, J. 2022. When does return-conditioned supervised learning work for offline reinforcement learning? Advances in Neural Information Processing Systems, 35: 1542--1553

    2022

  6. [6]

    Cai, Q.; Liu, S.; Wang, X.; Zuo, T.; Xie, W.; Yang, B.; Zheng, D.; Jiang, P.; and Gai, K. 2023. Reinforcing user retention in a billion scale short video recommender system. In Companion Proceedings of the ACM Web Conference 2023, 421--426

    2023

  7. [7]

    Chen, L.; Lu, K.; Rajeswaran, A.; Lee, K.; Grover, A.; Laskin, M.; Abbeel, P.; Srinivas, A.; and Mordatch, I. 2021. Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems, 34: 15084--15097

    2021

  8. [8]

    Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning, 1597--1607. PmLR

    2020

Show all 39 references
  1. [9]

    Chen, X.; Wang, S.; and Yao, L. 2024. Maximum-Entropy Regularized Decision Transformer with Reward Relabelling for Dynamic Recommendation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 376--384

    2024

  2. [10]

    F.; Leike, J.; Brown, T.; Martic, M.; Legg, S.; and Amodei, D

    Christiano, P. F.; Leike, J.; Brown, T.; Martic, M.; Legg, S.; and Amodei, D. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30

    2017

  3. [11]

    W.; Radford, A.; and Sutskever, I

    Dhariwal, P.; Jun, H.; Payne, C.; Kim, J. W.; Radford, A.; and Sutskever, I. 2020. Jukebox: A generative model for music. arXiv preprint arXiv:2005.00341

    2020 arXiv

  4. [12]

    Gao, C.; Li, S.; Zhang, Y.; Chen, J.; Li, B.; Lei, W.; Jiang, P.; and He, X. 2022. Kuairand: An unbiased sequential recommendation dataset with randomly exposed videos. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 3953--3957

    2022

  5. [13]

    Gao, C.-X.; Wu, C.; Cao, M.; Kong, R.; Zhang, Z.; and Yu, Y. 2024. ACT: empowering decision transformer with dynamic programming via advantage conditioning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 12127--12135

    2024

  6. [14]

    Gao, J.; Li, Y.; Mao, S.; Jiang, P.; Jiang, N.; Wang, Y.; Cai, Q.; Pan, F.; Gai, K.; An, B.; et al. 2025. Generative Auto-Bidding with Value-Guided Explorations. arXiv preprint arXiv:2504.14587

    2025

  7. [15]

    Graf, S.; and Luschgy, H. 2000. Foundations of quantization for probability distributions. Springer Science & Business Media

    2000

  8. [16]

    Haarnoja, T.; Zhou, A.; Abbeel, P.; and Levine, S. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, 1861--1870. Pmlr

    2018

  9. [17]

    Hu, S.; Fan, Z.; Huang, C.; Shen, L.; Zhang, Y.; Wang, Y.; and Tao, D. 2024. Q-value regularized transformer for offline reinforcement learning. arXiv preprint arXiv:2405.17098

    2024 arXiv

  10. [18]

    Jang, E.; Gu, S.; and Poole, B. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144

    2016 arXiv

  11. [19]

    Ji, J.; Li, Z.; Xu, S.; Hua, W.; Ge, Y.; Tan, J.; and Zhang, Y. 2024. Genrec: Large language model for generative recommendation. In European Conference on Information Retrieval, 494--502. Springer

    2024

  12. [20]

    Kang, W.-C.; and McAuley, J. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), 197--206. IEEE

    2018

  13. [21]

    Kostrikov, I.; Nair, A.; and Levine, S. 2021. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169

    2021 arXiv

  14. [22]

    Kumar, A.; Zhou, A.; Tucker, G.; and Levine, S. 2020. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 33: 1179--1191

    2020

  15. [23]

    Liu, J.; Gao, X.; Li, Y.; Li, X.; Lu, H.; and Wang, B. 2024 a . Supervised Learning-enhanced Multi-Group Actor Critic for Live Stream Allocation in Feed. arXiv preprint arXiv:2412.10381

    2024 arXiv

  16. [24]

    Liu, Z.; Liu, S.; Zhang, Z.; Cai, Q.; Zhao, X.; Zhao, K.; Hu, L.; Jiang, P.; and Gai, K. 2024 b . Sequential recommendation for optimizing both immediate feedback and long-term retention. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development...

    2024

  17. [25]

    Ma, J.; Zhou, C.; Cui, P.; Yang, H.; and Zhu, W. 2019. Learning disentangled representations for recommendation. Advances in neural information processing systems, 32

    2019

  18. [26]

    Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I.; et al. 2018. Improving language understanding by generative pre-training

    2018

  19. [27]

    Shi, J.-C.; Yu, Y.; Da, Q.; Chen, S.-Y.; and Zeng, A.-X. 2019. Virtual-taobao: Virtualizing real-world online retail environment for reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, 4902--4909

    2019

  20. [28]

    Sun, F.; Liu, J.; Wu, J.; Pei, C.; Lin, X.; Ou, W.; and Jiang, P. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management, 1441--1450

    2019

  21. [29]

    Van Den Oord, A.; Vinyals, O.; et al. 2017. Neural discrete representation learning. Advances in neural information processing systems, 30

    2017

  22. [30]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  23. [31]

    Wang, S.; Chen, X.; Jannach, D.; and Yao, L. 2023 a . Causal decision transformer for recommender systems via offline reinforcement learning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 1599--1608

    2023

  24. [32]

    Wang, S.; Chen, X.; and Yao, L. 2024. Retentive decision transformer with adaptive masking for reinforcement learning based recommendation systems. arXiv preprint arXiv:2403.17634

    2024 arXiv

  25. [33]

    Wang, W.; Lin, X.; Feng, F.; He, X.; and Chua, T.-S. 2023 b . Generative recommendation: Towards next-generation recommender paradigm. arXiv preprint arXiv:2304.03516

    2023 arXiv

  26. [34]

    Xie, J.; Girshick, R.; and Farhadi, A. 2016. Unsupervised deep embedding for clustering analysis. In International conference on machine learning, 478--487. PMLR

    2016

  27. [35]

    Yamagata, T.; Khalil, A.; and Santos-Rodriguez, R. 2023. Q-learning decision transformer: Leveraging dynamic programming for conditional sequence modelling in offline rl. In International Conference on Machine Learning, 38989--39007. PMLR

    2023

  28. [36]

    Zador, P. 1982. Asymptotic quantization error of continuous signals and the quantization dimension. IEEE Transactions on Information Theory, 28(2): 139--149

    1982

  29. [37]

    Zhang, G.; Wang, Y.; Chen, X.; Qian, H.; Zhan, K.; and Wang, B. 2024. UNEX-RL: reinforcing long-term rewards in multi-stage recommender systems with unidirectional execution. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 9305--9313

    2024

  30. [38]

    Zhang, L.; Lieffers, J.; and Pyarelal, A. 2022. Deep reinforcement learning with vector quantized encoding. arXiv preprint arXiv:2211.06733

    2022 arXiv

  31. [39]

    Zhao, K.; Zou, L.; Zhao, X.; Wang, M.; and Yin, D. 2023. User retention-oriented recommendation with decision transformer. In Proceedings of the ACM Web Conference 2023, 1141--1149

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.