{"id":"8ca209b0-0301-4e89-b025-2354b64f7796","arxiv_id":"2501.09354","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Adding VGG-19 style embeddings and shopping cart training sessions to a transformer recommender raised HR@5 from 0.681 to 0.735 on a proprietary e-commerce dataset.","lead":"Style4Rec adds image-style features and shopping cart sessions to a transformer recommender and reports higher HR@5, NDCG@5, and MRR@5 than BERT4Rec and SASRec on a private household-goods dataset. The gains are real in its tables, but the headline comparison gives Style4Rec extra training data and a wider model, so the advantage is not cleanly attributed to the new signals.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline gains over BERT4Rec/SASRec are confounded: Style4Rec is trained on roughly twice the sessions (cart + purchase) and with a wider model (8 heads, 1024-d embeddings) than the baselines, so the reported HR/NDCG/MRR improvements cannot be attributed to style and cart signals.","rationale":"I read the paper in good faith. The proposed method combines a transformer encoder with style embeddings extracted via neural style transfer and a training strategy that adds shopping-cart sessions during training/validation but not testing. The intended contribution is to show that these two signals improve sequential recommendation over strong transformer baselines. That claim requires a comparison where the only systematic difference between Style4Rec and the baselines is the proposed signals. The paper does not provide such a comparison: the final Style4Rec configuration is explicitly wider (8 heads, 1024-d embeddings) and is trained on roughly twice the data (cart plus purchase), while the baselines use only purchase sessions. The reader's weakest assumption identifies exactly this confound, and the paper's own prose confirms it. The controlled ablations in Table 3 are also internally weakened because the full model is the widened one. Without matched capacity, matched data, error bars, or significance tests, the strongest claimed numbers cannot be attributed to the modules under study. This is not a question of disagreement with a prevailing consensus; it is a load-bearing flaw in the experiment that determines the headline result. For these reasons, the reader's REJECT verdict remains appropriate.","tokens_in":10766,"tokens_out":3162,"duration_ms":80413,"concrete_test":"Retrain the strongest baseline, SASRec, under the same data and capacity conditions as the final Style4Rec: use purchase plus cart sessions in training/validation, 8 heads, 1024-d product embeddings, and the same negative-sampling protocol. If SASRec's HR@5, NDCG@5, and MRR@5 approach or match Style4Rec's values (0.735, 0.674, 0.654), then the reported improvements are explained by extra data and model width rather than by the style/cart-specific architecture. Additionally, in Table 3, rerun Style4Rec in all four configurations with identical architecture and data counts to verify that the style and cart ablations are not confounded by width.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparative claim in the abstract and Table 2 is not supported by the experimental design. Training Procedures states that the number of transformer blocks and heads was set to 2 'for fair comparison with other benchmarks (BERT4Rec, SASRec),' but the Results section then says that for the final Style4Rec configuration the authors 'increased the number of heads to 8 and the dimension of the learnable product embeddings to 1024 to make the model wider.' Table 2 compares this full Style4Rec, trained on purchase plus cart sessions plus style embeddings, against BERT4Rec and SASRec, which by the paper's own description use only purchase sessions and their own hyperparameter settings. Thus the reported HR@5 gain from 0.681 to 0.735 is confounded by three simultaneous changes: more training data, a much larger model, and the proposed style/cart modules. The paper's own controlled ablation (Table 3) shows that adding cart data alone (Style4Rec3) accounts for most of the gain (HR@5 0.670 to 0.718), while style embeddings alone improve only 6 of 9 metrics with no significance testing; and even that ablation compares the full Style4Rec against narrower configurations. Therefore the headline claim that style and cart information 'enhance' existing transformers is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Style4Rec, a Transformer-based sequential recommender that augments purchase sessions with two additional signals: style embeddings extracted from product images via VGG-19 gram matrices, and shopping cart sessions used during training and validation (but not testing). The model is evaluated on a proprietary e-commerce dataset against BERT4Rec and SASRec, with reported gains in HR@5, NDCG@5, and MRR@5. The paper also presents ablations of the style and cart components, an analysis of negative sampling, and a study of varying maximum session length.","tokens_in":11053,"tokens_out":5931,"duration_ms":57008,"significance":"The practical motivation is clear: shopping cart actions reflect user interest even when no purchase occurs, and visual style is an underused signal in sequential recommendation. The ablation design in Table 3 and the negative-sampling analysis in Table 4 are useful steps toward understanding what matters. If the headline gains were established under controlled comparisons, this would be a valuable applied contribution for e-commerce recommendation. However, the current empirical evidence is confounded: the main comparison varies data, model width, and the proposed modules simultaneously, and no statistical testing is provided. The contribution is therefore not yet substantiated, although it is potentially salvageable with additional experiments.","major_comments":[{"comment":"The headline comparison in Table 2 is confounded. Style4Rec is trained on both purchase and shopping cart sessions (38,117 sessions total, per Table 1) with the final configuration using 8 attention heads and 1024-dimensional learnable product embeddings, whereas BERT4Rec and SASRec are trained on purchase sessions only (19,463 sessions) and, according to the paper's own description, use the recommended hyperparameters from their respective papers. The improvements in HR@5, NDCG@5, and MRR@5 reported in the abstract therefore conflate the proposed style and cart signals with additional training data and substantially increased model capacity. To support the claim that style and cart information enhance existing transformers, the authors must report baselines trained on the same cart-augmented data and at a matched model width, and an ablation that isolates data and capacity.","section":"Training Procedures and Results, Table 2"},{"comment":"The ablation in Table 3 does not isolate the contribution of the proposed components in the final row. The text states that for the full Style4Rec configuration the number of heads was increased to 8 and the dimension of the learnable product embeddings to 1024 to make the model wider, while the other configurations in Table 3 appear to use the narrower default (2 heads and 128-dimensional embeddings described in the Methodology). The 'Improvement' row, which drives the conclusion that the full combination is best, therefore reflects a simultaneous change in model width plus the addition of style and cart data. All four configurations should be evaluated at a fixed width (or at both widths) with a controlled data budget.","section":"Results, Table 3"},{"comment":"No statistical significance analysis is provided. Every table reports a single point estimate per metric without standard deviations, number of seeds, or significance tests. This is especially problematic because the style-only ablation (Style4Rec1 vs. Style4Rec2 in Table 3) improves only 6 of 9 metrics and ties on MRR@10, so the claimed gains may be within noise. The authors should report means and standard deviations over multiple random seeds and perform paired significance tests (e.g., paired t-test or Wilcoxon signed-rank test) on the primary metrics.","section":"Training Procedures and Results"},{"comment":"The preprocessing choices of removing overlapped sessions that contain both purchase and cart products and removing repeated final products are asserted to improve real-world generalization, but no supporting experiment is provided. These choices change the composition of the training data and could interact with the comparison to baselines. Their effect should be quantified, for example by reporting results on a version of the data that keeps overlapped sessions or repeated final items.","section":"Preprocessing"}],"minor_comments":[{"comment":"The column header 'MMR@20' should be 'MRR@20'.","section":"Tables 2 and 3"},{"comment":"The captions appear inconsistent with the text: Figure 1 is captioned as updating the content image, while Figure 2 is captioned 'First method of neural style transfer algorithm' but describes updating a noisy input image; the second method is the one that starts from Gaussian noise. Please align the captions with the description in Section 'Style Embeddings'.","section":"Figures 1 and 2"},{"comment":"The capitalization of the proposed model is inconsistent ('Style 4Rec', 'STYLE4Rec', 'Style4Rec') and the baselines are written as both 'Bert4Rec' and 'BERT4Rec'; please standardize the notation.","section":"Throughout"},{"comment":"The row labeled 'Style 4Recneg' has exactly the same metric values as Style4Rec1 in Table 3; please clarify whether the negative-sampling study was performed only on the purchase-only configuration rather than on the full model, and if so, state that explicitly.","section":"Table 4"},{"comment":"The statement that the dimension of the gram matrices is 'independent of the dimension of the feature maps' is confusing; the gram matrix dimension depends on the number of feature maps, not their spatial dimensions, so please rephrase.","section":"Style Embeddings"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is valid and lands: the central comparison in Table 2 is confounded by data volume and model width, and the final row of Table 3 is similarly confounded. I recommend major revision rather than rejection because the issues are addressable with additional controlled experiments and significance testing. If the authors cannot provide matched-capacity baselines and multi-seed error bars, the current claims should not be accepted. The paper would also benefit from a public data release or code, given that the dataset is proprietary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is an incremental but plausible idea wrapped in an experiment that doesn't support its own headline. The style embedding via VGG-19 gram matrices and the use of cart sessions during training are legitimate, and the paper is transparent about its design choices. But the reported gains over BERT4Rec and SASRec are not attributable to the proposed components: the full model trains on roughly twice the sessions (purchase plus cart) and uses a wider architecture (8 heads, 1024-d embeddings) than the baselines. The paper itself says it widened the model for the final configuration, so this isn't hidden, but it does break the comparison.\n\nWhat's actually new: the specific combination of style embeddings from gram matrices and cart-session training hasn't been done in prior transformer sequential recommenders, as far as the cited work suggests. The component ablations in Table 3 are informative and show that cart data alone drives most of the gain; style embeddings give a small, inconsistent boost. The dynamic recommendation and negative-sampling experiments are sensible extras.\n\nThe soft spots are real. No code, no data, no error bars, no significance tests. Evaluation uses 100 negative samples per target, making the numbers tricky to compare with full-corpus ranking. And the style-versus-cart comparison in Table 3 is itself confounded because the full Style4Rec is wider than Style4Rec3. The paper's own numbers put the style contribution at only 6 of 9 metrics improving, so the \"style enhances\" claim is weak even on its own terms.\n\nIs the paper serious? Yes. It is clearly written and honest about limitations. But the central comparative claim is not established. I would still send it to a serious referee—the idea is worth testing properly, and the authors have the kind of dataset that could settle it if they run controlled baselines with matched data and capacity. As is, it is not a publishable result.\n\nRecommendation: engage with it, but expect a major revision that fixes the confound and adds error bars.","headline":"Incremental but plausibly useful idea whose headline comparison is confounded by more data and a wider model for the full system.","tokens_in":11586,"tokens_out":1996,"would_cite":false,"duration_ms":20922,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Style4Rec claims that adding product-image style embeddings and shopping cart sessions to a Transformer recommender improves next-item prediction over BERT4Rec and SASRec on a large e-commerce dataset.","keywords":["sequential recommendation","transformer","style embeddings","neural style transfer","shopping cart","e-commerce","self-attention","next-item prediction"],"falsifier":"Train SASRec and BERT4Rec on the same union of purchase and cart sessions with the same 8-head, 1024-dimension configuration used for Style4Rec, and compare HR@5, NDCG@5, and MRR@5 on the identical held-out purchase sessions; if the gap between Style4Rec and the equally-equipped baselines shrinks to near zero, the claim that style and cart signals cause the improvement is falsified.","tokens_in":10528,"feed_emoji":"🛒","tokens_out":2291,"duration_ms":22678,"temperature":0.7,"pith_summary":"The paper proposes Style4Rec, a Transformer-based sequential recommender that augments purchase history with two underused signals: visual style extracted from product images and shopping cart sessions that indicate interest without a purchase. The authors aim to show that these signals let a multi-layer Transformer encoder beat established baselines on real company data. They report HR@5 rising from 0.681 to 0.735, NDCG@5 from 0.594 to 0.674, and MRR@5 from 0.559 to 0.654. The contribution matters because existing Transformer recommenders rely on purchase sequences alone, leaving style and cart information on the table.","feed_headline":"Style and cart signals lift Transformer recommender","feed_subtitle":"Adding product-image style embeddings and cart sessions raised HR@5 from 0.681 to 0.735 over BERT4Rec and SASRec.","key_machinery":"The key machinery has three parts: style embeddings, cart-session training, and a dual product-vector design. Style embeddings come from the Gram matrices of the first two VGG-19 layers, max-pooled to 2×16×16 dimensions, following the neural style transfer algorithm of Gatys, Ecker, and Bethge. Shopping cart sessions are added to training and validation but excluded from testing, letting the model learn from interest signals without inflating test performance. The model separates a product vector of historical behavior (Transformer encoder output) from learnable product embeddings, then scores candidates by cosine similarity, which also lets new products be compared without retraining.","core_discovery":"On the paper's own terms, the central discovery is that a Transformer-based sequential product recommender can be meaningfully improved by injecting style embeddings computed from product images via the neural style transfer algorithm's Gram matrices, and by training on shopping cart sessions in addition to purchase sessions. The model compares a product vector of historical behavior against learnable product vectors using cosine similarity and binary cross-entropy loss with negative sampling. In experiments on a 1.5-year e-commerce dataset with 38,117 sessions, Style4Rec outperformed BERT4Rec and SASRec across HR, NDCG, and MRR at list lengths 5, 10, and 20, with the largest gains at shorter list lengths.","pith_inferences":["The style embedding mechanism may transfer most to categories where visual appearance drives choice, such as apparel or home decor, and less to utilitarian goods where function dominates.","Cart sessions may serve as a weak form of purchase-intent labeling; a testable extension is whether treating cart adds as a separate positive class with its own label improves performance beyond simply pooling them with purchases.","The reported gains likely depend on the wider final configuration (8 attention heads, 1024-dim embeddings) as much as on the new signals; a clean ablation would hold architecture constant while toggling style and cart inputs.","The method's reliance on Gram-matrix style could generalize to other visual attributes, such as texture or color distribution, suggesting a family of image-derived embeddings for sequential recommenders."],"forward_implications":["If Style4Rec's reported gains hold, e-commerce recommenders can harvest visual style and cart activity as free additional training signals without changing the core Transformer architecture.","The dual product-vector design implies that newly added products can be recommended immediately by comparing their learnable embeddings with the historical-behavior vector, avoiding full retraining.","The finding that cart sessions help only during training suggests that implicit interest signals can improve purchase prediction even when they are absent at inference time.","The improvement pattern, larger at HR@5 than HR@20, indicates that style and cart information sharpen the top of the recommendation list, which is the part users actually see.","The negative-sampling result quantifies how much evaluation scope matters: restricting prediction to 101 candidates overstates performance relative to ranking all 2,991 products, a caveat for comparing reported numbers."],"supporting_citations":[{"why":"Supplies the neural style transfer algorithm and the Gram-matrix style representation used to build style embeddings from product images.","marker":"Gatys, Ecker, and Bethge 2015"},{"why":"Provides the VGG-19 convolutional network whose early-layer feature maps yield the Gram matrices for style extraction.","marker":"Simonyan and Zisserman 2015"},{"why":"Supplies the BERT4Rec baseline, a bidirectional Transformer recommender that Style4Rec claims to outperform.","marker":"Sun et al. 2019"},{"why":"Supplies the SASRec baseline, a unidirectional Transformer recommender that serves as the primary comparison and second-best model.","marker":"Kang and McAuley 2018"},{"why":"Defines the multi-head self-attention and point-wise feed-forward blocks that form the deep Transformer encoder at the core of Style4Rec.","marker":"Vaswani et al. 2017"}],"fun_headline_variants":["Style and cart signals boost Transformer recommendations","Style4Rec: Using style and cart to outperform benchmarks","Image style plus cart data lifts recommender performance","Cart and style context improves Transformer recommender","Style and cart features elevate e-commerce ranking metrics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the performance gains come from the style and cart signals themselves, but its final Style4Rec model is trained on roughly twice the sessions (purchase plus cart) and uses a wider architecture (8 heads, 1024-dimensional embeddings) than the SASRec and BERT4Rec baselines, so the headline gap could partly reflect data and capacity differences rather than the proposed signals.","fun_headline_variants_meta":{"raw":{"variants":["Style and cart signals boost Transformer recommendations","Style4Rec: Using style and cart to outperform benchmarks","Image style plus cart data lifts recommender performance","Cart and style context improves Transformer recommender","Style and cart features elevate e-commerce ranking metrics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000309,"raw_usage":{"total_tokens":1743,"prompt_tokens":904,"completion_tokens":839,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":769}},"tokens_in":520,"tokens_out":839,"duration_ms":9532,"temperature":1.0,"reasoning_tokens":769,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:06:10.044543+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SASRec and BERT4Rec on the same union of purchase and cart sessions with the same 8-head, 1024-dimension configuration used for Style4Rec, and compare HR@5, NDCG@5, and MRR@5 on the identical held-out purchase sessions; if the gap between Style4Rec and the equally-equipped baselines shrinks to near zero, the claim that style and cart signals cause the improvement is falsified.","supporting_citations":[],"review_version":1}