{"id":"11cbc917-eabf-4fc0-81d2-c33a43e07474","arxiv_id":"2412.03471","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A meta-algorithm that learns cluster-specific embedding functions jointly with cluster assignments improves clustering and denoising over standard representation learning baselines.","lead":"The paper proposes that when data naturally falls into groups, a useful representation should be tailored to each group, and it builds a training procedure that learns these group-specific summaries together with the grouping itself. The approach is tested on four standard machine learning models and shows better clustering and denoising on small benchmark datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quantitative gains of PTAE over AE are confounded by the k-means penalty term (Eq. 3) and by comparing S from joint clustering to k-means on AE embeddings; an ablation with lambda=0 would settle it.","rationale":"The reader's weakest assumption (k known in advance) is valid and is explicitly acknowledged in Section VII, but the overestimation claim is only heuristic and is not tested. A more load-bearing concern is the empirical confound in the AE experiments: PTAE's objective (3) includes a k-means penalty that the standard AE objective lacks, and the clustering comparison uses S from joint optimization rather than k-means on the learned embedding. This means the paper's headline quantitative claim that PTAE outperforms AE does not necessarily support the central conceptual claim about cluster-specific representations. I would not reject the paper: the VAE (Section IV) and RBM (Section VI) results provide qualitative evidence that cluster-specific latent spaces can be meaningful and do not rely on the penalty, and the authors promise code. The right verdict is therefore CONDITIONAL, with the condition being an ablation that isolates the effect of partial tensorization from the effect of the added penalty and the joint clustering protocol. This is consistent with the reader's conditional verdict, though the emphasis shifts from the k-sensitivity issue to the experimental confound.","tokens_in":14254,"tokens_out":6506,"duration_ms":71547,"concrete_test":"On the five datasets of Figure 2, retrain PTAE with lambda=0 in Eq. (3), keeping all other settings, and compare its ARI and denoising MSE against: (a) AE2 with k-means on its embedding, and (b) a non-cluster-specific AE2 augmented with the same lambda*p penalty on the shared embedding. If PTAE(lambda=0) loses its margin over AE2+k-means, or if AE2+penalty matches PTAE, the reported benefit is not due to cluster-specific representations. Report the lambda value used in the paper and the mean over at least 5 seeds with standard errors.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that cluster-specific representations are intrinsically better than standard ones. The main quantitative evidence (Section III-B, Figure 2) does not isolate this factor. In Eq. (3), PTAE minimizes reconstruction plus lambda * p(tilde_x_{i,j}) with p(.) = ||g_{Psi_j}(g_Omega(tilde_x))||^2, a 'k-means penalty' absent from all AE baselines. Any improvement in clustering ARI could therefore come from this regularizer rather than from making the encoder cluster-specific. The clustering comparison is also structurally asymmetric: PTAE's S is the direct output of the joint Lloyd's-step optimization, while AE baselines are scored by running k-means on their embeddings. A fairer baseline would be a deep clustering method, or an AE with the same penalty but a shared encoder. The discussion of unknown k in Section VII is a real limitation, but it is secondary: even with k known, the reported gains do not demonstrate that cluster-specific structure, rather than the added penalty or the joint clustering protocol, is the cause.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes \"cluster specific representation learning,\" a downstream-agnostic formulation in which, given data with k assumed clusters, the learned representation is a concatenation of cluster-specific embedding functions applied to a shared representation. To control parameter growth, the authors introduce a partial tensorized variant (PTAE) that keeps a shared encoder and makes only a final layer cluster-specific, while still jointly optimizing a cluster assignment matrix S via an alternating scheme of gradient descent and Lloyd's steps. The idea is extended to Variational Autoencoders, contrastive losses, and Restricted Boltzmann Machines. The reported experiments compare PTAE with standard AEs and fully tensorized AEs (TAE) on clustering and denoising tasks, and qualitatively analyze VAE latent spaces. The central empirical claim is that PTAE outperforms standard AEs by a considerable margin and performs comparably to TAE with much lower model complexity.","tokens_in":14485,"tokens_out":2815,"duration_ms":31538,"significance":"If the central claim were rigorously established, the paper would make a useful contribution: it offers a principled, task-agnostic criterion for representation quality, and the partial-tensorization idea is a sensible way to control the added parameter cost of cluster-specific models. The paper also has a commendable breadth, showing that the same meta-formulation can be plugged into several representation-learning frameworks, and it provides code for reproduction. However, the current experimental evidence does not isolate the claimed mechanism: the PTAE objective differs from the AE baselines in two ways at once (a k-means penalty term and the joint clustering protocol), so the reported gains cannot yet be attributed to cluster specificity itself. The comparison also lacks error bars, statistical tests, and standard deep-clustering baselines. These are fixable issues, and the conceptual framework is worth pursuing, but the paper in its present form does not yet support its main quantitative claims.","major_comments":[{"comment":"The clustering comparison is not on equal footing: for TAE and PTAE the reported ARI is computed from the assignment matrix S that is part of the joint optimization, while for standard AE baselines the ARI is obtained by running k-means on the learned embeddings. This asymmetry alone can explain part of the observed gap, because the tensorized methods receive direct gradient feedback on S. To support the claim that cluster-specific representations are intrinsically better, the authors should report k-means ARI on PTAE embeddings as well, and also run the tensorized assignment procedure on the standard AE embeddings, so that both methods are scored under identical clustering protocols.","section":"Section III-B, Figure 2 (left)"},{"comment":"The PTAE objective contains the extra term -lambda p(x~_{i,j}) with p(.) = ||g_{Psi_j}(g_Omega(x~_{i,j}))||^2, a k-means penalty that is absent from every AE baseline. Any improvement in clustering or denoising could therefore be caused by this regularizer alone, rather than by making the encoder cluster-specific. The authors should run an ablation with lambda = 0, and ideally also compare against a shared-encoder AE that uses the same k-means penalty without cluster-specific heads. This is load-bearing for the paper's central claim, and the absence of this ablation makes the quantitative evidence in Figure 2 inconclusive.","section":"Equation (3), Section III-A and Figure 2"},{"comment":"The experimental evidence is presented without error bars, multiple restarts, or significance tests, and the main quantitative evaluation relies on two small toy datasets (parallel lines and 3D lines), one real low-dimensional dataset (penguin), and a 1000-sample subset of MNIST. Given that both the optimization and the clustering are sensitive to initialization and to the alternating updates, single-run point estimates are not sufficient to support the claimed 'considerable margin'. The authors should report means and variances over several random seeds, and ideally include standard deep clustering or deep embedding baselines (e.g., DEC or k-means-friendly autoencoders) to calibrate the magnitude of the reported ARI values.","section":"Section III-B, Figure 2 and Appendix VIII-B"},{"comment":"The discussion of an unknown number of clusters is purely heuristic. The claim that overestimating k makes 'the additional embedding functions simply learn copies of one of the cluster specific embedding functions' is not demonstrated anywhere in the paper; Appendix VIII-C only shows the underestimation case (six classes, two embedding functions). Since the method assumes k is known and all experiments set k to the true number of classes, the authors should either provide a diagnostic for choosing k, or empirically test the overestimation behavior to substantiate the claim in Section VII.","section":"Section VII and Appendix VIII-C"}],"minor_comments":[{"comment":"There is a typo in 'Lyod's step' in the paragraph describing the alternating optimization; it should read 'Lloyd's step'.","section":"Section II, Optimization"},{"comment":"The figure legend says 'k-means as benchmark' but the text describes the baseline as k-means on the original data; please make the protocol explicit in the caption.","section":"Section III-B, Figure 2 left"},{"comment":"The KL divergence term has a mismatched parenthesis: it should be KL(q_{Psi_j,Omega}(z_j|x_i) || p_{Phi_j,Theta}(z_j)), not KL(q_{Psi_j,Omega}(z_j|x_i) || p_{Phi_j,Theta}(z_j).","section":"Equation (6)"},{"comment":"The caption repeats 'Top row: embedding of test samples' after the middle-row description; the third row should presumably be labeled differently (e.g., 'Bottom row').","section":"Figure 3 caption"},{"comment":"The text says 'for the datasets considered in Figure 2 and Figure 8 (top row) we provide the runtime comparison in Figure 8 (bottom row)', but Figure 8 does not have a top row of datasets; both rows are runtime plots. Please correct the cross-reference.","section":"Appendix VIII-B, Runtime"},{"comment":"The reparameterization is written as z_{i,j} = mu_{i,j} + sigma^2_{i,j} * epsilon; this should be sigma_{i,j} (not sigma^2) for a standard Gaussian perturbation, or the notation for the variance should be clarified.","section":"Section IV-A, Sampling"}],"recommendation":"major_revision","confidential_remarks":"The paper is positioned as a general formulation but currently reads more like an extended abstract: the experimental section is thin for a journal submission, and the key causality claim (cluster specificity, not the penalty term or the joint-clustering protocol, drives the gains) is not tested. The missing lambda=0 ablation and the asymmetric clustering evaluation are the core issues. If the authors supply these experiments and the results hold, the paper could become acceptable; otherwise the central claim remains unsupported. I also note that the paper relies heavily on the authors' own prior work [9] without a critical comparison, but I see no reason to question novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The partial-tensorization idea is real but modest. Instead of making the whole encoder cluster-specific as in Esser et al. [9], the authors keep a shared encoder and add only a cluster-specific final layer. That is a sensible parameter-efficiency improvement, and the paper applies the same recipe across autoencoders, VAEs, contrastive losses, and RBMs. The writing is clear, the method is simple to understand, and the code is promised. The VAE latent visualizations are a nice qualitative touch, and the discussion of known-k limitations is honest, if brief.\n\nThe soft spot is the main quantitative claim. PTAE minimizes reconstruction plus a k-means penalty on the embedding, while the AE baselines do not have that penalty term. So the improved ARI and denoising MSE could come from the regularizer alone, not from making representations cluster-specific. The clustering comparison is also asymmetric: PTAE's assignment matrix S comes straight out of the joint optimization, while standard AEs are scored by running k-means on their embeddings afterward. An ablation with lambda=0, or a shared-encoder AE with the same penalty, would settle whether the architecture matters. Without that, the headline claim is not cleanly supported.\n\nSome other issues are minor but worth noting: no error bars, small and easy datasets, no deep clustering baselines, and a heuristic treatment of unknown k. The paper does not oversell itself too much, but the abstract says the experiments \"clearly show\" improved performance, which is stronger than what the design supports.\n\nThe citation pattern is fine; the authors build directly on [9] and cite relevant clustering and contrastive learning work. I would not call this a breakthrough, and I would not cite it as evidence that cluster-specific representations are intrinsically better. But the partial-tensorization recipe and the cross-framework demonstrations are worth having in the literature.\n\nIf this lands in my inbox, I would send it to review, with a strong instruction to the authors to add the lambda=0 ablation, fix the baseline asymmetry, and report variance. Without those, it is a workshop-level contribution; with them, it could be a solid conference paper.","headline":"A small, clearly explained parameter-efficiency trick for tensorized representation learning, but the experiments don't isolate what causes the reported gains.","tokens_in":14977,"tokens_out":1326,"would_cite":false,"duration_ms":16953,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A good representation is specific to each inherent cluster in the data, and a partially tensorized architecture makes this idea work across autoencoders, VAEs, contrastive learning, and RBMs.","keywords":["representation learning","cluster-specific embeddings","tensorized autoencoders","variational autoencoders","contrastive learning","restricted Boltzmann machines","unsupervised clustering","denoising"],"falsifier":"Train PTAE on a dataset with known cluster structure, first with $k$ equal to the true number of clusters and then with $k$ larger than the true count; in the overestimated case, check whether the extra per-cluster embedding functions actually converge to copies of existing ones as the paper predicts, and whether clustering ARI and denoising MSE stay stable or degrade — either outcome would settle the claim.","tokens_in":14075,"feed_emoji":"🧩","tokens_out":10366,"duration_ms":85964,"temperature":0.7,"pith_summary":"The paper proposes a downstream-agnostic definition of a good representation: when data have inherent clusters, the embedding function should be specific to each cluster rather than tuned to a single task. To make this practical, it introduces partial tensorization, which keeps most of the encoder shared and makes only a thin per-cluster layer cluster-specific, so the parameter cost grows slowly with the number of clusters. The authors instantiate this idea in autoencoders, variational autoencoders, contrastive learning, and restricted Boltzmann machines, and show on several datasets that the cluster-specific versions improve clustering and denoising over standard counterparts. The central claim is that the cluster-specific inductive bias, not extra model capacity, is what yields the better representations.","feed_headline":"Cluster-specific embeddings beat plain autoencoders","feed_subtitle":"A thin per-cluster layer captures cluster structure without a big parameter increase.","key_machinery":"The load-bearing mechanism is partial tensorization: the embedding function is factored into a shared network $g_\\Omega$ followed by $k$ small cluster-specific maps $g_{\\Psi_j}$, together with a cluster assignment matrix $S$ updated by k-means-style Lloyd steps. The shared part learns structures common to all clusters; the per-cluster part tunes the embedding to each cluster, so the method inherits the benefits of full tensorization without scaling the whole model with $k$. A k-means penalty $\\lambda \\|g_{\\Psi_j}(g_\\Omega(\\tilde{x}_{i,j}))\\|_2^2$ is added in the autoencoder setting to keep the latent space cluster-friendly.","core_discovery":"The paper's central claim is that a representation is good when it is specific to each inherent cluster in the data, and that this property can be learned jointly with cluster assignments through a partial tensorized objective: minimize $\\sum_{j=1}^k \\sum_{i=1}^n S_{j,i} \\mathcal{L}(g_{\\Psi_j}(g_\\Omega(x_i)))$ subject to $S$ being a valid soft assignment matrix, where $g_\\Omega$ is a shared encoder and each $g_{\\Psi_j}$ is a small cluster-specific layer. For reconstruction-based models, the same pattern is applied to encoder and decoder with cluster-centered inputs $\\tilde{x}_{i,j}=x_i-c_j$. Empirically, the partial tensorized autoencoder (PTAE) outperforms a standard autoencoder on clustering and denoising and matches the fully tensorized autoencoder while using far fewer parameters, because only the final layer is duplicated per cluster.","pith_inferences":["A direct extension the paper leaves implicit is monitoring trained per-cluster layers for redundancy: pairwise distances between the $g_{\\Psi_j}$ weight matrices could turn the heuristic claim about overestimated $k$ into a quantitative diagnostic.","The same shared-plus-per-cluster factorization could be applied at the output heads of a supervised network, effectively creating a lightweight mixture-of-experts that routes by latent cluster rather than by a hand-defined task.","Because the assignment matrix $S$ is updated by Lloyd steps, a mini-batch variant that assigns and updates on each batch could scale the method to streaming or very large datasets."],"forward_implications":["If the central claim holds, representation quality can be evaluated and improved without specifying a downstream task, because cluster specificity acts as a proxy for usefulness across tasks such as clustering and denoising.","Partial tensorization can be layered onto any representation learner with a small architectural change, so autoencoders, VAEs, contrastive models, and RBMs can all inherit cluster-specific embeddings.","New points are embedded by assigning them to the cluster whose embedding function gives the lowest loss, giving an explicit inference rule for unseen data.","Because the parameter overhead is confined to the per-cluster layer, the approach remains practical for larger numbers of clusters than fully tensorized versions."],"supporting_citations":[{"why":"Shows that linear autoencoders learn principal directions of the full dataset, which motivates splitting the embedding per cluster.","marker":"[5]"},{"why":"Defines the fully tensorized autoencoder and the joint clustering/embedding update that this paper extends to partial tensorization.","marker":"[9]"},{"why":"Defines the VAE objective that the tensorized VAE extends to cluster-specific latent spaces.","marker":"[10]"},{"why":"Defines the restricted Boltzmann machine energy and likelihood used to build the tensorized RBM.","marker":"[11]"},{"why":"Supplies the k-means++ initialization for the cluster assignment matrix S.","marker":"[12]"},{"why":"Provides the k-means penalty on embeddings used to enforce cluster-friendly latent structure.","marker":"[18]"},{"why":"Supplies the MNIST data used in the numerical evaluations of the AE, VAE, and RBM settings.","marker":"[19]"},{"why":"Supplies the penguin dataset, a real-data illustration of Simpson's paradox used for evaluation.","marker":"[20]"},{"why":"Formalizes the contrastive loss that the tensorized contrastive objective extends to cluster-specific embeddings.","marker":"[36]"}],"fun_headline_variants":["Thin per-cluster layer boosts autoencoder quality","Cluster-specific embeddings without a big parameter jump","Learn a small layer per cluster, gain better representation","Small cluster-specific tweak lifts autoencoder performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method requires the number of clusters $k$ to be known in advance, and every experiment sets $k$ to the true number of classes, so behavior under misspecified $k$ is only discussed heuristically.","fun_headline_variants_meta":{"raw":{"variants":["Thin per-cluster layer boosts autoencoder quality","Cluster-specific embeddings without a big parameter jump","Learn a small layer per cluster, gain better representation","Small cluster-specific tweak lifts autoencoder performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1504,"prompt_tokens":937,"completion_tokens":567,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":553,"completion_tokens_details":{"reasoning_tokens":507}},"tokens_in":553,"tokens_out":567,"duration_ms":5850,"temperature":1.0,"reasoning_tokens":507,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:20:59.888732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train PTAE on a dataset with known cluster structure, first with $k$ equal to the true number of clusters and then with $k$ larger than the true count; in the overestimated case, check whether the extra per-cluster embedding functions actually converge to copies of existing ones as the paper predicts, and whether clustering ARI and denoising MSE stay stable or degrade — either outcome would settle the claim.","supporting_citations":[{"cited_title":"Nonlinear principal component analysis using autoas- sociative neural networks,","cited_arxiv_id":null,"evidence_quote":"Shows that linear autoencoders learn principal directions of the full dataset, which motivates splitting the embedding per cluster."},{"cited_title":"Im- proved representation learning through tensorized autoencoders,","cited_arxiv_id":null,"evidence_quote":"Defines the fully tensorized autoencoder and the joint clustering/embedding update that this paper extends to partial tensorization."},{"cited_title":"Information processing in dynamical systems: Foun- dations of harmony theory,","cited_arxiv_id":null,"evidence_quote":"Defines the restricted Boltzmann machine energy and likelihood used to build the tensorized RBM."},{"cited_title":"K-means++: The advantages of careful seeding,","cited_arxiv_id":null,"evidence_quote":"Supplies the k-means++ initialization for the cluster assignment matrix S."},{"cited_title":"Towards k- means-friendly spaces: Simultaneous deep learning and clustering,","cited_arxiv_id":null,"evidence_quote":"Provides the k-means penalty on embeddings used to enforce cluster-friendly latent structure."},{"cited_title":"Mnist handwritten digit database,","cited_arxiv_id":null,"evidence_quote":"Supplies the MNIST data used in the numerical evaluations of the AE, VAE, and RBM settings."},{"cited_title":"Ecological sexual di- morphism and environmental variability within a community of antarctic penguins (genus pygoscelis),","cited_arxiv_id":null,"evidence_quote":"Supplies the penguin dataset, a real-data illustration of Simpson's paradox used for evaluation."},{"cited_title":"A theoretical analysis of contrastive unsupervised representation learning,","cited_arxiv_id":null,"evidence_quote":"Formalizes the contrastive loss that the tensorized contrastive objective extends to cluster-specific embeddings."}],"review_version":1}