{"id":"b09df72c-5e2e-424a-90c9-24fca31d18e5","arxiv_id":"2504.14214","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GUIDER combines modality-similarity recalibration with teacher-student distillation based on optimal transport to remove noisy interactions in multi-modal recommender systems, improving ranking metrics across four datasets and five models.","lead":"This paper proposes GUIDER, a plug-in framework that cleans noisy user-item interactions in multi-modal recommender systems by reclassifying suspicious clicks using image and text similarity, then distilling a cleaned ID-based teacher into the multi-modal model. It reports consistent gains across four datasets and five base recommender models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (3) is one-directional: initial low-loss items are never demoted, so the §3.2 unreliable loss partition contaminates the DBPR teacher; no oracle-label validation is provided.","rationale":"The reader's conditional verdict is appropriate, and my stress-test pass found a sharper form of the same concern. The paper's central claim is that AMSC identifies noisy interactions and DBPR/KD then removes them. But Eq. (3) is one-directional: I_true is I_rel plus some reclassified spurious items, and no item in I_rel can ever be demoted. Since Eq. (2) uses loss, and Section 3.2 demonstrates that in MMRecs loss-based similarity scores of clean and noisy interactions overlap, I_rel is expected to contain noise. That noise is then frozen into the teacher's positive set and the student's KD target. The paper does not validate the final labels against the oracle noise it already knows how to inject, so the denoising effect is not directly evidenced. A possible response is that empirical gains still hold regardless of mechanism, but the paper's framing specifically claims denoising; an oracle-label study would settle whether the labels are the active ingredient. The w/o AMSC ablation shows the component matters, but not that it labels correctly. I therefore keep the CONDITIONAL verdict: the empirical tables are encouraging, but the mechanism's core assumption is unverified and has an internal asymmetry. The proposed test is straightforward with the released code.","tokens_in":18639,"tokens_out":6538,"duration_ms":63623,"concrete_test":"Reproduce the Section 3.2 oracle-noise setup on Baby and MicroLens with BM3 as the base model: inject 10% random interactions, run GUIDER's AMSC, and compare the final I_true/I_false labels against the oracle noise. Report precision, recall, and the fraction of oracle-noisy items that remain in I_true because they were in the initial low-loss set (Eq. 2). If recall of noise in I_true is high, train the DBPR teacher on (a) AMSC labels and (b) oracle labels, and compare downstream Recall@20/NDCG@20. If oracle labels clearly outperform AMSC labels, the denoising mechanism is not being driven by correct labels; if they perform similarly, the concern is answered.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing link in GUIDER is AMSC (Eqs. 2–6): it converts a loss-based partition into 'clean' I_true and 'noisy' I_false, which are then used to train the DBPR teacher whose logits are distilled into the student. The paper's own Section 3.2 shows that in MMRecs the similarity/loss signal does not separate clean from noisy interactions (Figure 4: distributions 'almost overlap'). Eq. (2) nevertheless uses exactly that signal as the initial partition. Eq. (3) then defines I_true as I_rel plus spurious items that are modally similar to a reliable item; no rule moves any item from I_rel to I_false. Thus every interaction in the initial low-loss set is permanently treated as clean, regardless of modality evidence. If the initial I_rel contains noisy interactions—which the Section 3.2 overlap result implies it will—those errors are frozen into the teacher's positive set and the student's KD target. AMSC can only add items to I_true, not remove false positives from it. The paper reports no precision/recall validation of the final I_true against oracle noise, even though it already uses injected oracle noise in Section 3.2. The w/o AMSC ablation shows the component changes performance, but it does not show the labels are correct. Because DBPR and the KD signal are downstream of these labels, the central 'denoising' claim rests on an unvalidated and internally asymmetric label construction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GUIDER, a universal denoising framework for multi-modal recommender systems (MMRecs). The method first partitions user-item interactions into reliable and spurious sets using the recommendation loss (Eq. 2), then recalibrates these sets via Adaptive Modality Similarity Calibration (AMSC) using intra-modal text/vision similarity and a cross-modal hashing confidence score (Eqs. 3-6). A teacher IDRecommender is trained with a Denoising Bayesian Personalized Ranking (DBPR) loss on the calibrated clean/noisy labels (Eq. 7), and its ranking logits are distilled into the student MMRec through an optimal-transport-based knowledge distillation objective (Eqs. 9-15). Experiments on Baby, Sports, Clothing, and MicroLens with five base models (VBPR, MMGCN, BM3, DRAGON, FREEDOM) report consistent improvements over T-CE, R-CE, WBPR, and BOD, with ablations attributing gains to each component.","tokens_in":18928,"tokens_out":5962,"duration_ms":52461,"significance":"If the empirical claim holds, GUIDER would provide a reusable, model-agnostic denoising layer for MMRecs, addressing a gap that the paper convincingly identifies in IDRec-oriented denoising methods. The experimental coverage is broad (5 base models × 4 datasets), the paper includes a component ablation, hyperparameter sensitivity analysis, robustness under injected noise, and statistical significance markers, and the source code is released. These are substantive strengths. However, the central label-construction mechanism is not validated against oracle noise, and the paper's own Section 3.2 findings create an internal tension with the loss-based partition that AMSC relies on. The OT-KD formulation also has technical ambiguities that affect reproducibility. The core idea is worth pursuing, but the current evidence for the correctness of the denoising labels is incomplete.","major_comments":[{"comment":"The calibration rule in Eq. (3) is one-directional: any interaction initially placed in I_rel is permanently in I_true, and I_false is defined only as the portion of I_spr not rescued by modality similarity. No rule moves a low-loss item into the noisy set. This matters because §3.2 reports that in MMRecs the similarity distributions of clean and noisy interactions 'almost overlap' (Figure 4), which directly implies that the initial loss-based partition in Eq. (2) contains false positives. Those false positives are then frozen into the teacher's positive set for DBPR (Eq. 7) and into the KD signal. The w/o AMSC ablation (Table 3) only shows that the component changes performance; it does not show that the resulting labels are correct. Since the paper already injects oracle noise in §3.2, the authors should report precision/recall (or similar) of I_true and I_false against these oracle labels, and if the labels are found to be corrupted, the calibration rule should be revised to allow demotion of unreliable items.","section":"§4.1, Eqs. (2)-(6)"},{"comment":"The OT-KD objective as written is not well-defined. In Eq. (10), z is defined as a vector of log-sigmoid pairwise scores {log(σ(s_ui − s_uj))}, which are negative scalars and not a probability distribution. The optimal transport formulation in Eq. (12) imposes the marginal constraints P1_d = z_s and P^T 1_d = z_t, which require z_t and z_s to be probability histograms; the paper does not state how these logits are normalized or how the teacher and student pair sets are aligned into a common cost matrix. Independent of this, Eq. (13) contains an apparent typo: the second Sinkhorn update also uses z_s, whereas it should use z_t to enforce the teacher marginal. These issues make the central KD loss ambiguous and unreproducible, and they should be corrected and clarified.","section":"§4.3, Eqs. (10)-(13)"},{"comment":"The source of the loss values used to form I_rel and I_spr is not specified. In Algorithm 1, the teacher is updated before the partition is computed, and both the teacher IDRec and the student MMRec have different representation spaces; the partition could differ substantially depending on which model's L_Rec is used. Since this partition, the DBPR labels, and the entire downstream KD signal all depend on this choice, the ambiguity is load-bearing for reproducibility. The paper should state explicitly which model's loss is used in Eq. (2) at each iteration.","section":"Algorithm 1 and Eq. (2)"}],"minor_comments":[{"comment":"The row label 'MGCN' should be 'MMGCN' to match the model name in Section 5.1.3.","section":"Table 2"},{"comment":"The R@20 value of 0.579 is implausible (R@20 cannot exceed 1 and all other values in the table are below 0.12); this appears to be a typo, likely 0.0579, and should be corrected.","section":"Table 2, VBPR + R-CE on Clothing"},{"comment":"The default value of S_thres used in the main experiments (Table 2) is not reported; the sensitivity analysis in Figure 7 shows the effect of different thresholds but does not identify the operating point used elsewhere.","section":"§5.4.2"},{"comment":"The paper does not specify which base models, training epochs, or checkpoints were used to produce the similarity-score distributions in Figure 4, which limits the reader's ability to judge how representative these curves are of the actual MMRecs used in the main experiments.","section":"§3.2, Figure 4"},{"comment":"Reference [41] contains a formatting error: the names 'Beibei Kong, Zhijin Wang, Bo Hu, and Zang Li' appear inserted into the citation text before 'TransRec', and the entry is incomplete.","section":"References"},{"comment":"The abstract contains a typo ('multi-modalrecommndation') and the contributions section has an ungrammatical phrase ('with via fine-grained re-calibration'); these should be fixed in revision.","section":"Abstract and Contributions"}],"recommendation":"major_revision","confidential_remarks":"The main reason for major revision is the one-directional AMSC issue combined with the paper's own Section 3.2 finding that loss-based similarity is not a reliable noise indicator in MMRecs. I would ask the authors to validate I_true/I_false against oracle noise (they already have the infrastructure from Section 3.2) and, if the validation reveals contamination, to revise the calibration rule so that items can be demoted as well as promoted. The OT-KD formulation also needs correction and clarification before the paper is reproducible. The experimental breadth, released code, and ablation structure are strengths, and the core idea is worth pursuing; I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing worth taking from this paper is the negative result: similarity- and loss-based denoising signals that work for ID-based recommenders largely collapse in multi-modal recommenders. Figure 4, where clean and noisy interaction scores almost overlap for MMRecs but separate for IDRecs, is a genuine finding and likely the most citable part of the work. The second thing is that GUIDER, as a plug-in layer, consistently improves ranking metrics over T-CE, R-CE, WBPR, and BOD across five base models and four datasets. The ablations support the contribution of each component, and the code is public. That is solid applied work.\n\nThe main soft spot is the AMSC label construction. The paper's own Section 3.2 says loss and similarity are unreliable in MMRecs, yet Eq. (2) uses exactly that signal for the initial partition, and Eq. (3) only moves items from the spurious set into the clean set. No item from the initial reliable set is ever demoted. So any noisy interaction that happens to have low loss is frozen into the positive set for DBPR and into the teacher's KD target. The stress-test note is right that this asymmetry makes the 'denoising' claim depend on an unvalidated label set. They already inject oracle noise in Section 3.2, so reporting precision/recall of the final I_true against those known labels would be straightforward and would answer this directly. Without it, the mechanism is plausible but unproven.\n\nOther issues are minor by comparison: SGDL and DeCA are cited but missing from the baselines, the CMH projection in Eq. (6) is under-specified, and the p<0.05 claim appears without error bars or test details. There are also a few typos in the tables (e.g., 'MGCN' and a stray 0.579).\n\nWho should read this: anyone working on multimodal recommendation or denoising. The negative result is worth citing, and the framework is a reasonable starting point even if the label quality question remains open. I would not desk-reject it; I would send it to a referee who is willing to ask for a direct validation of the denoising labels, and I would treat the current empirical gains as provisional until that validation exists.","headline":"A useful negative result about IDRec denoising in multimodal recommenders, wrapped in a plug-and-play framework whose empirical gains are consistent but whose label construction needs direct validation.","tokens_in":19498,"tokens_out":3255,"would_cite":true,"duration_ms":32039,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GUIDER is a plug-and-play denoising layer for multi-modal recommender systems: it relabels noisy feedback using text and image similarity, then distills clean ranking knowledge into the model via optimal transport.","keywords":["multi-modal recommender systems","recommendation denoising","knowledge distillation","optimal transport","Bayesian personalized ranking","modality similarity calibration","noisy implicit feedback","plug-and-play framework"],"falsifier":"Inject known random noisy interactions at controlled rates into the training data, run AMSC, and compare its reclassified labels against the injected labels; if AMSC's precision and recall at recovering injected noise are no better than a loss-based threshold or random assignment, the calibration claim is not supported.","tokens_in":18419,"feed_emoji":"🧹","tokens_out":8348,"duration_ms":67924,"temperature":0.7,"pith_summary":"The paper argues that denoising methods built for ID-based recommenders break down in multi-modal recommenders, because the interaction-similarity scores they rely on no longer separate clean from noisy feedback in the multi-modal representation space. It proposes GUIDER, a plug-in training layer that reclassifies each user-item interaction using text and image similarity to items already judged clean, trains a denoised ID-based teacher with a ranking loss, and distills that teacher into the multi-modal student using an optimal-transport distance. Across four public datasets and five base recommenders, adding GUIDER improves Recall and NDCG at cutoffs 5 and 20, with the reported gains marked statistically significant, while the IDRec denoising baselines often leave the base model worse off. The payoff, if the claim holds, is a reusable denoising component that any multi-modal recommender can adopt without changing its architecture.","feed_headline":"Denoising layer lifts five multi-modal recommenders on four datasets","feed_subtitle":"GUIDER relabels noisy feedback by image and text similarity, then distills clean rankings into the model.","key_machinery":"Three components carry the argument. Adaptive Modality Similarity Calibration (AMSC) is the relabeling rule: a spurious interaction becomes clean when the maximum of its text-similarity and image-similarity to a reliable item exceeds a threshold, scaled by a cross-modal confidence score from hashing. Denoising Bayesian Personalized Ranking (DBPR) is a pairwise ranking loss that treats the calibrated clean set as positives and the calibrated noisy set as negatives. The OT-based knowledge distillation objective is an entropy-regularized optimal-transport distance between teacher and student ranking logits, solved by iterative matrix scaling; it is what lets a denoised ID teacher guide the multi-modal student without applying the denoising loss inside the multi-modal space.","core_discovery":"The central claim is that noisy feedback in multi-modal recommenders should not be cleaned by applying IDRec denoising losses directly to the multi-modal model. GUIDER instead starts from a loss-based partition of each user's interactions, then repairs that partition: an item initially labelled spurious is reclassified as clean if the maximum of its text-similarity and image-similarity to an already-clean item exceeds a threshold, with a cross-modal hashing confidence score weighting the decision. The repaired labels feed a Denoising Bayesian Personalized Ranking objective that trains an ID-based teacher, and the teacher's ranking logits are transferred to the multi-modal student through an entropy-regularized optimal-transport distance computed by the standard iterative entropic optimal-transport solver. The paper's empirical claim is that the full combination, not knowledge distillation alone and not direct application of AMSC and DBPR to the student, yields consistent gains over T-CE, R-CE, WBPR, and BOD across VBPR, MMGCN, BM3, DRAGON, and FREEDOM on Baby, Sports, Clothing, and MicroLens.","pith_inferences":["The paper's own finding that clean and noisy interactions have overlapping similarity scores leaves open whether AMSC's value comes from accurate noise identification or from acting as a useful reweighting prior; a direct label-recovery test would separate those two explanations.","The OT distillation may contribute regularization beyond denoising, since it aligns the geometry of teacher and student logits; gains could therefore partly reflect better knowledge transfer even on already-clean data.","A natural extension is to use AMSC's relabeling as a general noise-cleaning step for multimodal representation learning beyond recommendation, such as filtering image-text pairs before contrastive pretraining."],"forward_implications":["Each of the five base recommenders improves over its own no-denoising baseline when GUIDER is added, and the gains are largest on MicroLens, the dataset the paper identifies as noisiest.","Removing any one of AMSC, DBPR, or knowledge distillation lowers performance, and replacing the OT distance with KL divergence also lowers performance, so the three-component design is load-bearing.","As injected noise rises from 5% to 20%, GUIDER keeps performance from decaying as fast as the T-CE, R-CE, WBPR, and BOD baselines.","Because GUIDER is a training-time plugin, the same layer can be attached to future multi-modal recommenders without changing their architecture."],"supporting_citations":[{"why":"Supplies the T-CE and R-CE denoising baselines and the loss-based weighting idea that AMSC starts from.","marker":"[42]"},{"why":"Supplies WBPR, the popularity-based negative-sampling baseline that GUIDER must beat.","marker":"[11]"},{"why":"Supplies BOD, the bi-level optimization denoising baseline used for comparison.","marker":"[44]"},{"why":"Provides LightGCN, the default ID-rec teacher model trained with DBPR and distilled into the student.","marker":"[17]"},{"why":"Provides the entropic optimal-transport distance and its iterative solver used for the KD objective.","marker":"[7]"},{"why":"Establishes knowledge distillation, the teacher-student mechanism that GUIDER adapts to recommenders.","marker":"[21]"},{"why":"Defines BPR, the pairwise ranking objective that DBPR modifies to use calibrated positives and negatives.","marker":"[34]"},{"why":"VBPR, one of the base multi-modal recommenders used to test GUIDER's plug-and-play effect.","marker":"[16]"},{"why":"BM3, the base model used in the ablation study and a representative self-supervised multi-modal recommender.","marker":"[58]"},{"why":"Provides cross-modal hashing, used to compute the confidence score in AMSC's relabeling rule.","marker":"[54]"}],"fun_headline_variants":["Universal denoising for multi-modal recommenders via calibration","GUIDER: guided calibration denoises multi-modal recommenders","Plug-and-play denoising lifts multi-modal recommenders","Cross-modal similarity cleans noisy feedback in recommenders","Denoising multi-modal recommenders via optimal transport distillation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an item whose image or text looks like an item already labelled clean for a user is itself a clean interaction for that user, even though the paper also shows visually similar items can belong to different categories.","fun_headline_variants_meta":{"raw":{"variants":["Universal denoising for multi-modal recommenders via calibration","GUIDER: guided calibration denoises multi-modal recommenders","Plug-and-play denoising lifts multi-modal recommenders","Cross-modal similarity cleans noisy feedback in recommenders","Denoising multi-modal recommenders via optimal transport distillation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000351,"raw_usage":{"total_tokens":1939,"prompt_tokens":994,"completion_tokens":945,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":865}},"tokens_in":610,"tokens_out":945,"duration_ms":8097,"temperature":1.0,"reasoning_tokens":865,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:54:03.153247+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inject known random noisy interactions at controlled rates into the training data, run AMSC, and compare its reclassified labels against the injected labels; if AMSC's precision and recall at recovering injected noise are no better than a loss-based threshold or random assignment, the calibration claim is not supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies WBPR, the popularity-based negative-sampling baseline that GUIDER must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides LightGCN, the default ID-rec teacher model trained with DBPR and distilled into the student."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides cross-modal hashing, used to compute the confidence score in AMSC's relabeling rule."}],"review_version":1}