{"id":"56de7a54-41a4-4abb-a8c7-9e6e6ab27cae","arxiv_id":"2607.11030","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A shared MLLM backbone with task-specific tokens learns four collaborative signals simultaneously and feeds multiplex embeddings into multitask search ranking, improving GAUC and online metrics at JD.","lead":"MMRM fine-tunes one multimodal LLM with four e-commerce collaborative signals at once, producing four disentangled item embeddings in a single pass, then uses them for task-specific user modeling in ranking. The system is live at JD and reports small but revenue-relevant lifts in click, cart, and conversion rates.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The multiplex advantage may be driven more by extra embedding capacity and soft-search than by true signal disentanglement via task tokens.","rationale":"The paper is a clean industrial contribution with consistent offline tables and a real online deployment; I agree with the reader’s overall CONDITIONAL stance and moderate confidence. The reader correctly flags that the four graph-derived signals must be sufficiently heterogeneous for the shared-backbone design to work. My concern is one step downstream and more decisive for the strongest claim: even granting clean heterogeneous signals, the ranking ablations never isolate whether task-specific tokens produce representations whose differences matter, versus simply giving the ranking model four soft-search tables and extra capacity. Table 2 supports the representation side, but Table 4’s jump from any single MMRM[t] to ALL is modest and un-ablated against capacity-matched controls. That is the softest link in the causal chain from “task tokens + shared backbone” to “online lifts.” The concrete test above would settle it with one additional ranking run; until then the claim remains plausible but not fully isolated. No change to the reader’s verdict is required—only a sharper condition.","tokens_in":11769,"tokens_out":632,"duration_ms":5866,"concrete_test":"Retrain the ranking model of §2.3 under three controlled variants on the same 7-day logs: (1) four independently trained Single embeddings used as the four soft-search tables; (2) four identical copies of the MMRM SEARCH embedding; (3) the original MMRM[ALL]. If GAUC of (1) or (2) matches or exceeds MMRM[ALL] on CTR/ACR/CVR, the disentanglement claim is not load-bearing for the reported gains.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that the four task-specific tokens + projection heads produce genuinely disentangled representations that improve multitask ranking beyond what a single strong multimodal embedding (or simply more tables) would achieve. Table 2 shows MMRM beating Single and Vanilla-Multi on retrieval, which is consistent with disentanglement, but the ranking results in Table 4 leave a gap: SIMsoft(item_MMRM[t]) already lifts GAUC substantially over the e2e baseline for every individual t, and SIMsoft(item_MMRM[ALL]) adds a further ~0.01 GAUC. Without an ablation that (a) freezes the backbone and trains only the four heads, (b) replaces the four specialized tables with four independently trained Single models, or (c) uses four copies of the same SEARCH embedding under the ALL regime, it remains possible that the online lifts (UCTR +0.42 % etc.) are largely capacity / multi-table soft-search effects rather than the claimed multiplex alignment. The reader’s weakest assumption (signal heterogeneity + sampling) is related but secondary; even if the graphs are clean, the ranking experiments do not isolate whether the shared-backbone + task-token design is doing the work.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes MMRM, a shared-backbone MLLM (initialized from Qwen3-VL-4B) that is jointly aligned to four heterogeneous collaborative signals (q2i_click, i2i_click, i2i_cart, i2i_order) via task-specific special tokens and projection MLPs. It produces four disentangled item embeddings in a single forward pass. These embeddings are then used inside a multitask ranking model for task-specific soft-search over user behavior sequences and multi-head target attention, feeding an MMoE + task towers. Offline retrieval (F1@5, NDCG@5) and ranking (GAUC) results, plus a week-long online A/B test on JD search (UCTR +0.42 %, UACR +0.37 %, UCVR +0.35 %), are reported as evidence of superiority over single-task, vanilla-multi, and prior soft-search baselines.","tokens_in":12136,"tokens_out":730,"duration_ms":6649,"significance":"If the claimed multiplex advantage is real, the work is practically important: it shows how to amortize a large MLLM across the heterogeneous signals that matter for CTR/ACR/CVR ranking, yields measurable online lifts on a major e-commerce platform, and has already been fully deployed. The architectural idea (shared backbone + task tokens + dedicated heads) is simple, efficient at inference, and directly addresses two documented limitations of prior multimodal ranking pipelines. The industrial scale of the datasets and the online A/B results give the contribution concrete weight for the IR community.","major_comments":[{"comment":"Table 4 leaves a critical causal gap for the central claim. SIMsoft(item_MMRM[t]) already improves GAUC substantially over the e2e baseline for every individual t; SIMsoft(item_MMRM[ALL]) adds a further ~0.01. Without ablations that (a) replace the four specialized tables by four independently trained Single models, (b) use four copies of the same SEARCH embedding under the ALL regime, or (c) freeze the backbone and train only the four heads, it remains possible that the online lifts are largely multi-table capacity / soft-search effects rather than true signal disentanglement produced by the shared-backbone + task-token design. This isolation is load-bearing for the paper’s strongest claim.","section":null},{"comment":"§2.1 and Table 1: the four graph-derived signals are constructed with markedly different time windows, neighbor caps, and hard-negative rules. The paper asserts that these signals are sufficiently heterogeneous for a shared backbone to learn non-interfering representations, yet provides no quantitative analysis of signal overlap, neighbor-set Jaccard, or destructive interference under joint training. If the graphs largely capture the same co-occurrence structure, the multiplex advantage collapses; a short diagnostic (e.g., cross-task retrieval or embedding cosine heatmaps) is needed to support the weakest assumption.","section":null},{"comment":"§3.1–3.4 and Tables 2–4 report only point estimates. No confidence intervals, bootstrap standard errors, or statistical significance tests accompany the F1/NDCG/GAUC numbers or the online relative lifts. Given the massive sample sizes, even small absolute differences can be noise; without uncertainty quantification the claim of “significant performance gains” cannot be rigorously assessed.","section":null}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean applied IR paper that ships a working multi-signal fine-tune of Qwen3-VL-4B and a matching ranking stack, already deployed with real online lifts (UCTR +0.42 %, UACR +0.37 %, UCVR +0.35 %). That is the headline.\n\nWhat is new is the simultaneous four-signal training (q2i_click + three i2i graphs) on a shared backbone with dedicated [SEARCH]/[CLICK]/[CART]/[ORDER] tokens and projection heads, plus the downstream use of those four tables for task-specific soft-search + MHTA user modeling inside an MMoE ranking model. Table 2 shows MMRM beating both Single-task models and a Vanilla-Multi (shared [EMB] + prompts) baseline on F1/NDCG across all four retrieval tasks; Table 4 shows progressive GAUC gains when the multiplex tables are plugged into SIM-style ranking. The graph construction details (time windows, neighbor caps, hard-negative sampling) and GradCache scaling are practical and reproducible in spirit even if the data are closed. Citations are appropriate; no circularity.\n\nThe soft spot is real but secondary: the ranking ablations do not fully isolate whether the gains come from true signal disentanglement versus simply having four specialized embedding tables and better soft-search. SIMsoft(item_MMRM[t]) already lifts every task over the e2e baseline; ALL adds another ~0.01 GAUC. Without a frozen-backbone four-head ablation or a four-copy-of-SEARCH control, the stress-test concern stands—capacity and multi-table soft-search could be doing a lot of the work. Online numbers also lack error bars. These are the usual industrial gaps, not load-bearing flaws; the central engineering claim still holds for a SIGIR-style paper.\n\nThis is for people building large-scale multimodal ranking systems who need a concrete multi-signal recipe that actually ships. It is not a theoretical advance, but it is honest, well-executed, and already live. I would send it to peer review; the community benefits from seeing the full pipeline and the online numbers.","headline":"Solid industrial multi-signal MLLM recipe already live at JD; the multiplex claim is useful engineering but not fully isolated from capacity/soft-search effects.","tokens_in":12720,"tokens_out":533,"would_cite":true,"duration_ms":4694,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A shared multimodal model with task-specific tokens produces four disentangled product embeddings in one pass and lifts multitask search ranking.","keywords":["Multimodal Representation","Contrastive Learning","Multitask Learning","E-commerce Search System","MLLM fine-tuning","user behavior modeling","soft search"],"falsifier":"Train the identical architecture on deliberately mixed or randomly relabeled versions of the four signals; if the four embeddings become nearly identical and the multitask ranking lift disappears, the claim that the signals are usefully heterogeneous fails.","tokens_in":12687,"feed_emoji":"🛒","tokens_out":616,"duration_ms":5273,"temperature":0.7,"pith_summary":"E-commerce ranking needs to optimize several goals at once—clicks, carts, and purchases—and product images and titles contain the semantics that pure ID models miss. Prior work fine-tunes a multimodal language model on only one kind of user signal and then plugs the resulting embedding into the ranker as just another item feature. This paper argues that four different collaborative signals (query-to-item clicks plus item-to-item click, cart, and order sequences) are genuinely heterogeneous, and that a single shared backbone equipped with four special tokens and four projection heads can learn four complementary embeddings without destructive interference. Those multiplex embeddings are then used not merely as features but as the basis for task-specific soft-search over a user’s history, producing a dedicated user vector for each ranking tower. Offline and online results on JD traffic show consistent gains in retrieval quality and multitask GAUC, together with measurable lifts in live click-through, add-to-cart, and conversion rates.","feed_headline":"One model, four product embeddings, better search ranking","feed_subtitle":"Task tokens let a shared multimodal backbone serve clicks, carts, and orders at once—and lift live metrics on JD.","key_machinery":"MMRM: a shared MLLM backbone that appends one of four special tokens ([SEARCH], [CLICK], [CART], [ORDER]) and routes the final hidden state of that token through a dedicated MLP, trained with a masked multi-task contrastive loss so each token specializes without requiring separate models.","core_discovery":"Aligning one multimodal backbone to four heterogeneous collaborative signals through task-specific tokens and projection layers yields four disentangled item representations in a single forward pass; feeding those representations into task-specific soft-search user modeling measurably improves multitask ranking both offline and in production.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["One multimodal backbone, four task tokens, four ranking-ready product embeddings","Task-specific tokens turn a shared MLLM into multiplex item reps for search","Single pass yields disentangled click-cart-order embeddings that lift multitask ranking","Multiplex item views from heterogeneous signals improve user behavior modeling in search","Shared backbone plus projection layers deliver four collaborative product representations"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The four graph-derived signals (with their particular time windows, neighbor caps, and hard-negative rules) are different enough, and clean enough, that a shared backbone plus task tokens can learn truly separate representations rather than collapsing into one entangled embedding.","fun_headline_variants_meta":{"raw":{"variants":["One multimodal backbone, four task tokens, four ranking-ready product embeddings","Task-specific tokens turn a shared MLLM into multiplex item reps for search","Single pass yields disentangled click-cart-order embeddings that lift multitask ranking","Multiplex item views from heterogeneous signals improve user behavior modeling in search","Shared backbone plus projection layers deliver four collaborative product representations"]},"model":"grok-4.5","effort":"low","cost_usd":0.006672,"raw_usage":{"total_tokens":1682,"prompt_tokens":759,"num_sources_used":0,"completion_tokens":96,"cost_in_usd_ticks":66720000,"prompt_tokens_details":{"text_tokens":759,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":827,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":759,"tokens_out":96,"duration_ms":8317,"temperature":1.0,"reasoning_tokens":827,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T07:31:31.046556+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train the identical architecture on deliberately mixed or randomly relabeled versions of the four signals; if the four embeddings become nearly identical and the multitask ranking lift disappears, the claim that the signals are usefully heterogeneous fails.","supporting_citations":[],"review_version":1}