{"id":"230a1bf8-814c-4414-b7d9-7ca36012f76d","arxiv_id":"2412.08198","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adaptive^2 learns latent domain assignments with VQ-VAE and uses them to route inputs through shared and domain-specific networks, outperforming hand-crafted domain baselines in ad CTR prediction.","lead":"This paper proposes Adaptive^2, a system that automatically discovers hidden data domains in online advertising using a self-supervised VQ-VAE, then routes each ad request through shared and domain-specific networks. The method reports gains over hand-crafted domain approaches on two public ad benchmarks and a 2.3% revenue lift in Kuaishou's live-streaming ad system.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VQ-VAE routing is trained only on reconstruction (Eq. 10), with no task-gradient path into the codebook; unless reconstruction clusters are label-aligned, the reported gains may be from extra capacity, not from mined domains. A randomized-routing control would settle it.","rationale":"I read the paper as claiming that automatically mined latent domains, not hand-crafted ones, drive the gains. The whole architecture is a shared/specific network whose only input-dependent routing signal is the VQ-VAE codebook id. If that id is not task-aligned, the specific networks are just extra capacity applied to arbitrary feature-space slices, and the central claims about 'fine-grained domain adaptation' and 'significance of domain mining' collapse. The reader's weakest assumption identifies exactly this, and the paper's objectives (Eq. 10, Eq. 18) do not contain any term that would enforce or even encourage task alignment. This is an internal gap, not a disagreement with the field outside consensus: VQ-VAE provably clusters by reconstruction distance, not by label shift, absent an additional loss. The proposed randomized-routing control would settle it: with identical capacity and only the content dependence of routing removed, if performance holds, the mechanism is not doing the claimed work. I therefore keep the reader's conditional verdict: the paper is plausible and the online numbers are suggestive, but the central mechanism is unverified. I also note a secondary internal inconsistency: §5.4 claims Adaptive2 surpasses an MLP with twice the FLOPs on Criteo, but Table 4 shows MLP at 2.12G and Adaptive2 at 2.17G, and PEPNet has fewer FLOPs (2.02G) than Adaptive2, so the 'fair FLOPs' narrative needs a corrected table or a clarified comparison. This reinforces the need for independent verification but is not the core concern. The paper deserves credit for a deployed industrial system, but deployment without a mechanism control does not establish that the domain-mining premise holds.","tokens_in":15195,"tokens_out":6771,"duration_ms":72877,"concrete_test":"Retrain Adaptive2 from scratch with the same architecture, codebook size K, and training budget, but replace the codebook index used in Eq. (13) with a random permutation of the mined indices that preserves each index's marginal frequency; keep the VQ-VAE reconstruction loss running so the comparison differs only in whether the assignment is content-dependent. Run 5 seeds and report AUC/LogLoss with standard errors for (i) Adaptive2, (ii) random-routing control, (iii) MLP. If (ii) matches (i) within noise, the central claim that mined domain structure improves CTR fails. If (ii) is significantly worse, the specific assignments carry task-relevant signal. As a secondary check, report the KL divergence between P(Y|k) and P(Y) to quantify label alignment of mined domains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is stated in §4.1: VQ-VAE self-supervision 'ensures that data points assigned to different codebooks come from distinct data distributions...'. This premise is not entailed by the objective. In Eq. (10), the domain-mining loss L_d = ||z - z_hat||^2 + ... is a reconstruction loss on the input embedding projection z = FFN(concat(x_u|x_v)) (Eq. 5). The task loss (Eq. 17) is computed on the shared/specific network outputs after routing, but because the VQ-VAE receives a stop-gradient on its input and the codebook selection in Eq. (8) is non-differentiable, no gradient from L_task flows into the encoder, decoder, or codebook. Hence the partition k is optimized purely to reconstruct the raw feature mix, not to separate click/conversion distributions. Reconstruction-similar clusters (e.g., by ad ID or user behavior sparsity) may be statistically orthogonal to the label shift that matters for CTR, in which case routing to W_k is task-arbitrary and any gain over MLP could come from the added domain-specific parameters or from non-linear capacity rather than from mining 'fine-grained domains.' The paper's own Appendix D concedes VQ-VAE is an empirical choice with 'insufficient theoretical support.' The ablation (ADMM+HD vs ADMM+DMM) shows mined domains beat two hand-picked features, but it never includes a random-routing control, so it cannot rule out the capacity/partitioning explanation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"Adaptive^2 proposes to replace hand-crafted domain definitions in multi-domain CTR prediction with an automatically mined partition, using a VQ-VAE trained by reconstruction as a domain mining module and a shared-and-specific network that routes samples to domain-specific layers based on the mined domain index. The paper reports offline improvements over MLP, MMoE, PLE, ADIN, and PEPNet on Avazu and Criteo, shows that several baselines underperform a tuned MLP under approximately matched FLOPs, and reports an online deployment at Kuaishou with +2.376% revenue and +2.522% conversions. The central conceptual claim is that self-supervised reconstruction clustering discovers latent domain structure that hand-crafted domain features miss, and that this structure is responsible for the gains.","tokens_in":2223,"tokens_out":1825,"duration_ms":50740,"significance":"If the central claim were established, the paper would make a useful practical contribution by showing that domain definitions for multi-domain advertising models can be learned automatically rather than hand-crafted, and its computational-fairness comparison is a valuable reminder that model comparisons in this area should control for resource costs. The paper is also refreshingly explicit about its limitations in Appendix D, conceding that the VQ-VAE choice is empirical and lacks theoretical support. However, the load-bearing premise that reconstruction-based VQ-VAE routing identifies task-relevant domains is not demonstrated; the evidence is thin, with small AUC gaps and no reported uncertainty for the main results or the online deployment. The contribution is therefore promising but needs substantially stronger evidence before the claims can be accepted.","major_comments":[{"comment":"The load-bearing premise that VQ-VAE self-supervision 'ensures that data points assigned to different codebooks come from distinct data distributions' is not entailed by the objective. In Eq. (10), the domain-mining loss is purely a reconstruction loss on z = FFN(concat(x_u|x_v)), and Section 4.3 states that a stop gradient is applied to the VQ-VAE input, so no gradient from the task loss L_task (Eq. 17) flows into the encoder, decoder, or codebook selection in Eq. (8). The cluster assignment k is thus optimized to reconstruct the raw feature mix, not to separate click/conversion distributions. The ablation in Table 3 compares mined domains against two hand-picked features but never includes a randomized-routing control (e.g., random codebook assignment, or routing by a fixed random partition with the same architecture). Without such a control, the reported gains could be attributed to the extra capacity of the domain-specific parameters or to more flexible routing rather than to mined domain structure. Please add a random-routing control, and ideally a variant where the routing is trained with a task-gradient path (e.g., Gumbel-softmax), to test the mechanism.","section":"§4.1, Eq. (10)"},{"comment":"The main offline results show AUC improvements over the best baseline of about 0.002-0.003 (Avazu: 0.7850 vs 0.7823 for ADIN; Criteo: 0.8097 vs 0.8068 for PLE). The manuscript gives no error bars, no number of seeds, and no significance test for these differences. In Table 4 the italicized items are claimed to be statistically significant over MLP with P-value<0.01, but the testing procedure is not described (number of runs, paired or unpaired, which model's variance). Please report mean and standard deviation over multiple independent training runs (at least 3-5 seeds) and a proper significance test, for both the main results and the FLOPs-fair comparison. Otherwise the headline improvements are within the range of typical run-to-run noise for CTR models.","section":"Tables 2 and 4"},{"comment":"The online experiment reports single raw deltas of +2.376% revenue and +2.522% conversions over 'Previous SOTA', with no confidence intervals, no number of experimental units, and no statistical test. The comparison against ADMM+HD(PageID) (-0.112% revenue, -0.256% conversions) is likewise presented without uncertainty. Given that the experiment ran for 15 days on 10% traffic, daily or bucket-level variance is likely material. Please provide uncertainty estimates (e.g., bootstrap confidence intervals over days or traffic splits) and state explicitly whether the positive and negative deltas are statistically distinguishable from zero and from each other.","section":"Table 5 (online)"},{"comment":"The number of latent domains (codebook size m) is selected by increasing it until validation performance stops improving (§5.1.4). This means the clusters are tuned on the downstream objective, and the subsequent interpretation of those clusters as 'domains' with real-world meaning (Appendix B) is therefore circular to some degree: they are by construction clusters that improve validation AUC, not clusters validated against an external notion of domain. The paper should report sensitivity of results to the codebook size m, and, if possible, compare the mined partition against an external domain signal (e.g., ad placement, advertiser sector) to show whether the mined domains correspond to interpretable business segments.","section":"§5.1.4 and Appendix B"}],"minor_comments":[{"comment":"The text refers to 'MES' when discussing the mean-square-error codebook assignment; should be 'MSE' for consistency.","section":"Appendix C"},{"comment":"The sentence 'This process ensures that data points assigned to different codebooks come from distinct data distributions' is too strong as a description of what a reconstruction loss guarantees; rephrase to 'aims to encourage' or 'is intended to separate'.","section":"§4.1"},{"comment":"Figure 3 and Figures 4-5 are referenced in the text but do not appear in the manuscript body; ensure that all figures are actually included in the submitted version.","section":"Figures"},{"comment":"The claim to be 'the first approach to automatically learn both domain identification and adaptation in online advertising' should be qualified in light of existing work on automatic domain division (e.g., ref [16], D3) and self-supervised clustering methods that have been applied to recommendation; please clarify the precise novelty claim.","section":"Introduction / Related Work"},{"comment":"The online experiment uses 'ARF loss' but the acronym is never defined in the text; please define it or cite it more prominently.","section":"§5.5"},{"comment":"For the t-SNE visualization, the citation should refer to van der Maaten and Hinton (2008), not to ref [22] which is about autoregressive image generation.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The central mechanism of the paper — that reconstruction-based VQ-VAE routing discovers task-relevant domains — is plausible but not tested against a random-routing control, and the reported gains are small and lack uncertainty quantification. Given the industrial setting, the online metrics should be scrutinized for possible selection effects and for whether the +2.3%/+2.5% figures are stable across time. The paper would be substantially strengthened by adding a random-routing control, multi-seed offline results with significance tests, and confidence intervals for the online deployment. The authors' own Appendix D admits the VQ-VAE choice is empirical; this is honest but means the burden of proof is higher for the domain-mining claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you work on multi-domain CTR or recommenders. The core idea is simple and appealing: instead of hand-coding domains like ad placement, train a VQ-VAE on the raw feature mix to assign each sample a latent domain id, then route through per-domain networks alongside a shared network. On Avazu and Criteo it beats MLP, MMoE, PLE, ADIN, and PEPNet, and a Kuaishou deployment shows a 2.3% revenue lift and 2.5% conversion lift. Those are real numbers in that world.\n\nThe FLOPs-fair comparison is also a useful contribution. It shows some domain-adaptation baselines only win because they are larger, and a well-tuned MLP is a stronger baseline than many papers use. That is worth saying in public.\n\nThe soft spot is the one the stress-test note points to. The VQ-VAE is trained only on reconstruction loss, with stop-gradient, so no task gradient reaches the encoder, decoder, or codebook. The claim that different codebooks capture 'distinct data distributions' is not entailed by the objective. Clusters that reconstruct the feature mix well may be orthogonal to click behavior. The ablation compares mined domains to two hand-picked features, but there is no randomized-routing control. That control would have separated 'the partition helps' from 'any partition plus extra parameters helps.' Its absence leaves the central mechanism under-supported.\n\nTwo smaller issues: the text says the model surpasses an MLP with twice the FLOPs, but Table 4 shows roughly equal FLOPs; and D3, the closest prior work on automatic domain division, is cited but never compared. The online experiment is a single deployment with raw deltas, no confidence intervals, though in industry that is often how it is reported.\n\nThe paper's Appendix D is candid: VQ-VAE is an empirical choice and the mined domains lack interpretability and theoretical support. That honesty counts for something. The method is plausible, the results are consistent, and the industrial evidence is suggestive. But the central claim that the VQ-VAE mines task-relevant domains needs one clean control before I would trust the mechanism rather than the capacity.\n\nVerdict: send it to peer review, but ask for the randomized routing ablation, a direct D3 comparison, and a corrected FLOPs statement. If the mechanism holds up, this is a practical step forward for automatic domain discovery in ads.","headline":"VQ-VAE domain mining is a promising industrial idea, but the missing randomized-routing control leaves the central claim unproven.","tokens_in":16091,"tokens_out":3906,"would_cite":true,"duration_ms":40457,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Latent domains mined self-supervised by a VQ-VAE beat hand-crafted domain splits in advertising click-through-rate models, with a 2.3 percent revenue lift online.","keywords":["domain mining","domain adaptation","self-supervised learning","VQ-VAE","CTR prediction","online advertising","multi-domain recommendation","shared and specific network"],"falsifier":"Permutation falsifier: in the trained Adaptive2 on Avazu, keep all weights fixed but route each sample to a randomly permuted domain-specific network (same networks, random assignment of which sample goes to which). If AUC does not fall substantially from the reported 0.7850 toward the MLP baseline of 0.7759, the mined domain index is not actually carrying task-relevant routing information, and the gains would instead come from added capacity or the shared network.","tokens_in":14986,"feed_emoji":"🎯","tokens_out":10609,"duration_ms":91767,"temperature":0.7,"pith_summary":"The paper argues that the biggest bottleneck in multi-domain ad ranking is not the adaptation network but the definition of domains themselves: hand-crafted splits such as ad placement miss fine-grained distribution shifts that matter for click prediction. It proposes Adaptive2, which mines latent domains automatically with a vector-quantized variational autoencoder (VQ-VAE) and routes each sample to a domain-specific network alongside a shared network. On Avazu and Criteo it reports the best AUC among MLP, MMoE, PLE, ADIN, and PEPNet baselines, and shows that under comparable FLOPs the hand-crafted-domain baselines do not beat a single-domain MLP. When deployed in a live-streaming pre-ranking system, the method reports a 2.3% revenue increase and a 2.5% conversion increase. The point, if true, is that domain identity should be learned from the data rather than assumed from business metadata.","feed_headline":"Mining latent domains beats hand-crafted ad splits","feed_subtitle":"A VQ-VAE learns latent domains for CTR ranking, lifting revenue 2.3 percent and conversions 2.5 percent online.","key_machinery":"The load-bearing mechanism is the VQ-VAE domain mining module: a vector-quantized variational autoencoder whose codebook acts as a learnable set of latent domains. An encoder projects the concatenated user–item embedding into a continuous latent vector $z_e$, which is mapped to its nearest codebook entry $e_k$ by Euclidean distance; the index $k$ becomes the domain routing signal. The reconstruction objective is trained jointly with the downstream CTR loss, with a stop-gradient on the input so the mining module does not destabilize the feature embeddings. The authors' core assumption is that this self-supervised clustering yields codebooks that correspond to data distributions that are distinct across codes and similar within a code, making the mined index a meaningful domain label. The module plugs into different domain-adaptation backbones (the paper shows it improving both its own shared/specific network and PEPNet), which is why the authors describe it as a general tool for domain identification.","core_discovery":"Adaptive2 treats domain identification and domain adaptation as two separable tasks. A self-supervised VQ-VAE encoder maps each sample's feature embedding to one of $m$ codebook vectors; the chosen codebook index is the sample's latent domain. That index routes the sample through one of $m$ domain-specific networks, whose output is added to the output of a shared network before the click prediction head. The authors claim that this automatically mined domain structure captures fine-grained distributional differences that manually-selected domain features miss, and that this is why Adaptive2 outperforms baselines both offline (Avazu AUC 0.7850 vs 0.7823 for the best baseline ADIN; Criteo AUC 0.8097 vs 0.8068) and in online deployment. They further claim that previous domain-adaptation methods with hand-crafted domains perform no better than a single-domain MLP when compared at matched FLOPs, which they take as evidence that domain definition itself is the decisive factor.","pith_inferences":[],"forward_implications":["Domain definition should be treated as a first-class modeling problem in multi-domain CTR prediction, not a fixed input: hand-crafted domains under matched compute do not beat a single-domain MLP.","Self-supervised clustering with vector quantization can serve as a domain-identification module that transfers across network architectures (ADMM and PEPNet both improve when given the mined domain ids).","Because the domain signal is learned end-to-end, no expert annotation of domains is needed at training or serving time, which simplifies deployment.","Under matched FLOPs and parameter counts, the mined-domain model is more resource-efficient than the larger hand-crafted-domain baselines, which matters for industrial ROI.","Online, the mined domains give a 2.3% revenue lift and a 2.5% conversion lift over the previous online SOTA in the live-stream pre-ranking stage.","The same VQ-VAE routing idea transfers to other multi-domain ranking problems, such as e-commerce or short-video feeds, where business-defined scene IDs are too coarse; the paper only tests ads, but nothing in the mechanism is ad-specific.","The fair-FLOPs result implies that some published multi-domain gains may be artifacts of unequal compute budgets; re-benchmarking earlier methods at matched FLOPs could reorder the leaderboard.","A testable refinement is to make the number of latent domains adaptive, growing or pruning codebook entries, rather than a tuned hyperparameter, since the paper leaves $m$ fixed per dataset and acknowledges collapse risk."],"supporting_citations":[{"why":"Supplies the VQ-VAE quantization mechanism that performs the domain mining.","marker":"[37]"},{"why":"Defines the shared-and-specific ADIN architecture that Adaptive2 extends and serves as the strongest offline baseline.","marker":"[18]"},{"why":"Represents the STAR multi-domain CTR model, a hand-crafted-domain baseline that the paper argues is limited by fixed domain splits.","marker":"[31]"},{"why":"Provides the PEPNet baseline and the backbone used in the PEPNet+DMM variant showing the mined domain ids transfer across architectures.","marker":"[5]"},{"why":"Motivates the paper's focus on domain division as a methodological problem in multi-domain recommendation.","marker":"[16]"},{"why":"Supplies the preprocessed Avazu and Criteo data splits used in all offline experiments.","marker":"[8]"}],"fun_headline_variants":["Self-supervised VQ-VAE mines latent ad domains","Auto-learned domains beat hand-crafted ad splits","Adaptive2: domain mining lifts CTR and revenue","Domain mining outperforms manual domain definitions","Latent domain discovery improves ad adaptation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole mechanism rests on the premise that reconstruction-based clustering by the VQ-VAE produces latent domains that correspond to task-relevant distribution shifts; if the codebook groups data by features that have nothing to do with click behavior, routing by the mined domain index would not help beyond random assignment.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised VQ-VAE mines latent ad domains","Auto-learned domains beat hand-crafted ad splits","Adaptive2: domain mining lifts CTR and revenue","Domain mining outperforms manual domain definitions","Latent domain discovery improves ad adaptation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1335,"prompt_tokens":993,"completion_tokens":342,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":609,"completion_tokens_details":{"reasoning_tokens":271}},"tokens_in":609,"tokens_out":342,"duration_ms":3818,"temperature":1.0,"reasoning_tokens":271,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:05:40.812609+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Permutation falsifier: in the trained Adaptive2 on Avazu, keep all weights fixed but route each sample to a randomly permuted domain-specific network (same networks, random assignment of which sample goes to which). If AUC does not fall substantially from the reported 0.7850 toward the MLP baseline of 0.7759, the mined domain index is not actually carrying task-relevant routing information, and the gains would instead come from added capacity or the shared network.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the VQ-VAE quantization mechanism that performs the domain mining."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the shared-and-specific ADIN architecture that Adaptive2 extends and serves as the strongest offline baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the PEPNet baseline and the backbone used in the PEPNet+DMM variant showing the mined domain ids transfer across architectures."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates the paper's focus on domain division as a methodological problem in multi-domain recommendation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the preprocessed Avazu and Criteo data splits used in all offline experiments."}],"review_version":1}