Pith. sign in

REVIEW 9 cited by

BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1904.06690 v2 pith:VFCZSJVM submitted 2019-04-14 cs.IR cs.LG

Fei Sun , Jun Liu , Jian Wu , Changhua Pei , Xiao Lin , Wenwu Ou , Peng Jiang This is my paper · ORCID
classification cs.IRcs.LG
keywords bidirectionalmodelrepresentationssequencesequentialhistoricalitemleft
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Modeling users' dynamic and evolving preferences from their historical behaviors is challenging and crucial for recommendation systems. Previous methods employ sequential neural networks (e.g., Recurrent Neural Network) to encode users' historical interactions from left to right into hidden representations for making recommendations. Although these methods achieve satisfactory results, they often assume a rigidly ordered sequence which is not always practical. We argue that such left-to-right unidirectional architectures restrict the power of the historical sequence representations. For this purpose, we introduce a Bidirectional Encoder Representations from Transformers for sequential Recommendation (BERT4Rec). However, jointly conditioning on both left and right context in deep bidirectional model would make the training become trivial since each item can indirectly "see the target item". To address this problem, we train the bidirectional model using the Cloze task, predicting the masked items in the sequence by jointly conditioning on their left and right context. Comparing with predicting the next item at each position in a sequence, the Cloze task can produce more samples to train a more powerful bidirectional model. Extensive experiments on four benchmark datasets show that our model outperforms various state-of-the-art sequential models consistently.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. GARDRec: Decision-Level Graph Grounding for Large Language Model Recommendation

    cs.IR 2026-08 conditional novelty 6.0 of 10

    GARDRec improves LLM-based next-item ranking by grounding decisions in knowledge-graph embeddings, personalized graph contexts, and late-stage scoring rather than prompt text.

  2. LLM-Based User Personas for Recommendations at Scale

    cs.IR 2026-06 unverdicted novelty 6.0 of 10

    A framework for real-time LLM-based user interest personas in large-scale video recommendations, using distillation, async inference, and video clustering to balance interests with novel topics and improve viewer valu...

  3. Scaling Generative Recommendations with Context Parallelism on Hierarchical Sequential Transducers

    cs.IR 2025-07 conditional novelty 6.0 of 10

    Context parallelism with AllToAll and load-balanced mini-chunks lets HSTU recommendation models support 5.3x longer interaction sequences at 1.55x throughput scaling.

  4. MARS: Modality-Aligned Retrieval for Sequence Augmented CTR Prediction

    cs.IR 2025-09 conditional novelty 5.0 of 10

    MARS aligns text and image features of items, then augments sparse user sequences with retrieved sequences from similar heavy users, improving CTR prediction offline and online.

  5. TransAct V2: Lifelong User Action Sequence Modeling on Pinterest Recommendation

    cs.IR 2025-06 conditional novelty 5.0 of 10

    TransAct V2 adds lifelong user sequences and a next-action loss to Pinterest's CTR model, reporting online gains of +6.35% repin volume and -12.80% hide volume.

  6. An Industrial-Scale Sequential Recommender for LinkedIn Feed Ranking

    cs.IR 2026-02 conditional novelty 4.0 of 10

    A transformer-based sequential recommender, Feed SR, improved LinkedIn Feed time spent by 2.10% in an online A/B test and now serves the majority of Feed traffic.

  7. Coarse-to-Fine Long-term Interest Modeling for Generative Recommendation

    cs.IR 2026-02 conditional novelty 4.0 of 10

    GLASS extends generative retrieval with a tiered long-term interest vector and a first-SID-keyed search of long histories, reporting consistent gains over Tiger and DualGR on two public datasets.

  8. Agentic Web: Weaving the Next Web with AI Agents

    cs.AI 2025-07 conditional novelty 3.0 of 10

    A position paper defines the Agentic Web as the next web era and proposes a three-dimensional conceptual framework for understanding and building it.

  9. When Transformers Meet Recommenders: Integrating Self-Attentive Sequential Recommendation with Fine-Tuned LLMs

    cs.IR 2025-07 reject novelty 3.0 of 10

    SASRecLLM, a hybrid of SASRec and a LoRA-fine-tuned LLM, reports modest gains over TALLRec on MovieLens but shows inconsistent and contradictory results on Amazon and in warm-start settings.

Pith tools