REVIEW 4 major objections 6 minor 18 references
Infinite hierarchical contrastive clustering for personal digital envirotyping
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Infinite hierarchical contrastive clustering groups daily-life photos into personal environments and shared types without a preset cluster count, linking them to behaviors like smoking.
desk verdict A useful envirotyping method with an overstated 'infinite' claim: the stick-breaking term is a per-sample regularizer, not a nonparametric prior, but the core clustering direction is sound and worth refining. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The object that carries the argument is the joint loss $L = L_{\mathrm{ins}} + L_{\mathrm{clu}} + L_{\mathrm{ps}} + \lambda_{\mathrm{sb}} L_{\mathrm{sb}}$, computed from three heads attached to a residual convolutional encoder. The instance-level and cluster-level contrastive heads come from the base contrastive clustering method; the cluster head's softmax outputs $\pi_i(x)$ are mapped to stick-breaking weights via $\beta_i = \pi_i/(1-\sum_{j<i}\pi_j)$, the inverse of the identity $\pi_i = \beta_i \prod_{j=1}^{i-1}(1-\beta_j)$. Under the Griffiths–Engen–McCloskey construction each $\beta_i$ is supposed to follow Beta(1, $\alpha$), so the model adds the negative log of that density, weighted by $\lambda_{\mathrm{sb}}$, to the contrastive losses; smaller $\alpha$ concentrates probability on fewer sticks and thus fewer clusters. The participant-specific head adds a cross-entropy loss that predicts which participant took each image, which separates different participants' images within a cluster and produces distinct per-person sub-clusters representing specific environments.
What would settle it
Train IH-CC on the labeled convenience dataset, where the true structure is known (29 distinct environments, 6 environment types), and sweep $\alpha$ and $\lambda_{\mathrm{sb}}$ as well as the width of the cluster head. If the best agreement with the known labels occurs only for a narrow hand-tuned range of these hyperparameters, and the chosen cluster count moves roughly one-for-one with the cluster-head width at fixed $\alpha$, then the stick-breaking penalty is acting as a tunable regularizer rather than as a mechanism that infers the number of clusters from data.
Extended reading notes
Core claim
The central discovery the paper argues for is that the contrastive clustering recipe can be extended into a two-level environment discovery tool that does not fix the cluster count in advance. The cluster head's softmax probabilities are reinterpreted through the stick-breaking identity $\beta_i = \pi_i/(1-\sum_{j<i}\pi_j)$, and the negative log Beta(1, $\alpha$) density of these $\beta_i$ is added to the training loss, so the model can in principle use any number of clusters up to the head size and tends to use fewer when $\alpha$ is small. A second head predicts participant identity from the learned features, and its cross-entropy loss pushes each participant's distinct environments into well-separated sub-clusters inside the shared environment-type clusters. On the labeled household dataset the model reaches NMI 0.516 and accuracy 0.791, above the base contrastive clustering and deep clustering baselines, and on the 77-participant photoEMA dataset the derived clusters show environment-specific smoking, craving, and mood patterns.
Load-bearing premise
The load-bearing assumption is that the probability scores the network assigns to clusters can be treated as random draws from a stick-breaking process, so the chosen penalty hyperparameters, not the data alone, determine how many clusters emerge.
Editorial extensions
If this is right
- Digital envirotyping can proceed without a fixed environment taxonomy: the same trained model yields distinct personal environments at the sub-cluster level and shared environment types at the cluster level, with cluster granularity controlled by the stick-breaking hyperparameters.
- Including the participant-specific head yields tighter and better-separated per-participant sub-clusters within each environment-type cluster, as measured by higher silhouette scores and Dunn indices.
- Environment clusters can be linked to health outcomes: in the photoEMA analysis, bathrooms, daytime green areas, and daytime porches show the highest smoking rates, while indoor spaces and kitchens show lower rates.
- Because healthcare studies often collect multiple images per participant, the repeated-measures contrastive design applies beyond envirotyping to other image-based health monitoring settings.
- On the labeled dataset, the model assigns fewer clusters per participant than the base contrastive clustering model (8.3 vs 11.5), which supports the interpretation that similar environments are being consolidated rather than merely split.
Reading between the lines
- A direct test of the 'infinite' claim would hold the stick-breaking hyperparameters fixed, double the width of the cluster head, and see whether the effective cluster count stabilizes; the paper's Table 1, in which the count moves from 20 to 34 as $\alpha$ goes from 1.5 to 5.0, suggests the count may be driven by tuning rather than by data alone.
- Since the participant-specific head is trained to predict participant identity, it may be learning participant-specific shortcuts rather than environment content; checking whether clusters of the same room type from different participants are consistently merged would separate those two explanations.
- The same architecture could be applied to other repeated-sampling streams, such as wearable-camera images, ambient audio, or GPS trace segments, to discover personal settings without a pre-existing place taxonomy.
- A natural extension is to replace manual selection of $\alpha$ and $\lambda_{\mathrm{sb}}$ with a data-driven criterion, such as optimizing cluster quality against a small labeled validation set or a marginal-likelihood estimate; that would make the 'arbitrary number of clusters' property testable rather than a tuning outcome.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Infinite Hierarchical Contrastive Clustering (IH-CC), an extension of contrastive clustering for grouping daily-life photographs into distinct personal environments and, at a coarser level, into shared environment types. The proposed method adds a stick-breaking penalty on the softmax outputs of the cluster head (Section 3.2) and a participant-specific prediction head that encourages images from the same participant to form compact sub-clusters (Section 3.3). The method is evaluated on a small labeled convenience dataset of 484 images from six study-team members (Section 5) and applied to a photoEMA dataset from 77 smokers to illustrate associations between environment clusters and health outcomes (Section 6). The abstract and Section 8 claim that the method allows an arbitrary number of clusters and that the number of clusters is inferred from data rather than fixed a priori.
Significance. The paper targets a practically important problem—automatically identifying distinct environments from repeated daily-life images—and the contrastive-learning framing is well motivated by the parallel between image augmentations and variability in repeated environmental sampling. The participant-specific head is a simple, domain-appropriate idea for handling repeated measures in contrastive clustering, and the authors provide code and a publicly shareable demonstration dataset. However, the central theoretical claim of an 'infinite' and data-inferred cluster count is not supported by the construction as written; the stick-breaking term acts as a per-sample regularizer under a fixed truncation, with the number of clusters tracking the tuned hyperparameters. The evaluation is also limited by single-run results, manual labeling, and an ablation metric that is partly circular. If the claims are reframed to describe a tunable regularized contrastive clustering method with a participant-specific auxiliary head, the contribution is useful and of interest to the envirotyping community, but the current manuscript overstates the nonparametric and inferential aspects.
major comments (4)
- [Section 3.2, Eq. (1)-(2)] The central claim that placing a stick-breaking prior on predicted cluster probabilities 'allows an arbitrary number of clusters' and lets the number be 'inferred from data' (abstract, Section 3.2, Section 8) is not supported by the construction. In a genuine GEM/Dirichlet-process construction, a single stick-breaking draw defines global cluster proportions shared by all observations, and posterior inference over the partition determines the number of clusters. Here, Eq. (1)-(2) are applied per image: L_sb is a sum of per-sample negative log Beta(1, alpha) densities on transformed softmax outputs. There is no shared random measure and no posterior over K, so the stick-breaking term is a regularizer that biases each predicted probability vector toward an ordered, sparse pattern. The experiments confirm this reading: Table 1 shows the active cluster count moving from 20 to 34 as alpha goes from 1.5 to 5.0, and Section 5.2 states that alpha=1.5 was selected because the authors expected 10-20 clusters. With lambda=0, the count equals the CCH width K=40, so the number of clusters is bounded by the chosen truncation. The Limitations section also concedes that the SB prior 'still requires tuning.' I recommend reframing the contribution as a fixed-truncation, regularized contrastive clustering method with a tunable sparsity prior, rather than an infinite or data-inferred cluster count.
- [Section 5.2, Table 2] The comparative evaluation rests on a single run with manually assigned cluster labels, and no variance, repeated-seed statistics, or significance tests are reported. With only 484 images from six participants, the NMI/ACC differences in Table 2 may not be stable, and the comparison gives all baselines the IH-CC cluster count of 20, which is operationally reasonable but does not test the cluster-count selection mechanism itself. The reported TCL result (NMI 0.511, ACC 0.323) is also surprising and needs an explanation of the label-matching or evaluation procedure. These issues do not invalidate the qualitative observation that images of the same room tend to land in the same cluster, but they do not support the strength of the claimed improvement over baselines.
- [Section 7, Figure 5] The PSH ablation is partially circular. The Silhouette Score and Dunn Index are computed on participant-specific subclusters within each environment-type cluster, and these metrics measure exactly the participant separability that the cross-entropy loss L_ps is designed to enforce. Higher values under IH-CC with PSH may therefore reflect direct optimization of the loss rather than improved identification of distinct environments. A more convincing evaluation would use held-out labels of distinct environments, such as manually verified same-room image groups, or would show that the improved subcluster structure leads to better downstream outcome associations.
- [Section 6, Table 3] The outcome-linkage analysis averages cluster-level outcome rates and presents top- and bottom-ranked clusters without accounting for within-participant correlation (each participant contributes many images) or multiple testing across outcomes and clusters. The reported associations between cluster labels and smoking, craving, stress, and related outcomes should therefore be regarded as descriptive illustrations rather than statistically supported envirotyping-outcome links.
minor comments (6)
- [Section 3 heading] The heading contains a typo: 'infinite-hierachical' should be 'infinite-hierarchical'.
- [Section 3.2, Eq. (2)] The denominator in Eq. (2) has an index error: it should be a sum over j from 1 to i-1, i.e., 1 - sum_{j=1}^{i-1} pi_j, not '1 - P_{i=j}^{i-1} pi_j' as printed.
- [Section 3.3] The text says 'The PCH consists of a FC layer...' but the abbreviation PCH is not defined and appears to be a typo for PSH (Participant-Specific Head).
- [Section 5.2.1, Table 2] The definition of ACC and the procedure for matching predicted clusters to manual labels should be stated explicitly; the large gap between NMI (0.511) and ACC (0.323) for TCL suggests a label-matching issue that needs clarification.
- [Data and Code Availability] The code repository is referred to only as 'ih-cc-envirotyping'; a URL or persistent identifier should be provided.
- [Section 7, Figure 5] The caption states that including the PSH increases silhouette score and Dunn index, but the figure does not report error bars or the number of clusters/subclusters used; adding this information would help the reader judge the robustness of the ablation.
Circularity Check
The 'infinite' cluster count is set by tuned hyperparameters, not inferred from the data; the PSH ablation evaluates its own training objective.
-
fitted input called prediction
[Section 3.2, Eq. (1)-(2); Section 5.1 Table 1; Section 5.2 Cluster Evaluation]
"This implies that we may use the following formula to calculate the β_i (i > 1) from the predicted π_i: β_i = π_i / (1 - Σ_{j<i} π_j)... By tuning the parameters α and λ_sb of the SB prior, we can guide the model towards more (larger α) or fewer (smaller α) clusters without explicitly constraining their number... We hypothesized that between 10 and 20 clusters would be sufficient to distinguish distinct environments while clustering similar environments together. Thus, we set our α for the SB prior to α = 1.5."
The abstract and Discussion claim the number of clusters is 'inferred from data rather than fixed a priori,' but the reported cluster count is a direct function of the tuned hyperparameters. Eq. (2) is the exact algebraic inverse of Eq. (1), so the 'SB prior' is a deterministic penalty on the CCH softmax outputs, not a generative distribution over an infinite measure. Table 1 shows that with λ_sb = 0 the active cluster count equals the fixed CCH size K = 40, and that changing α moves the count from 20 to 34 with λ_sb fixed. The authors then choose α = 1.5 to target a hypothesized 10-20 cluster range and present the resulting 20 clusters as a data-driven output. The 'infinite/inferred from data' claim therefore reduces to the chosen α, λ_sb, and truncation K.
-
self definitional
[Section 3.3 (Participant-Specific Head); Section 7 (Ablation Study on PSH), Figure 5]
"A Participant-Specific Head (PSH) is incorporated to encourage the latent representations of images from different participants to be separable... In the photoEMA dataset, we calculated the Silhouette Score and Dunn Index to evaluate the compactness of participant-specific clusters... Figure 5 shows that the clusters from the IH-CC model with PSH have higher Silhouette Scores and Dunn Index values compared to the IH-CC model without PSH."
The PSH loss L_ps is a cross-entropy classification loss on participant ID, which directly trains the feature representations to separate participants. The ablation then evaluates participant-specific subclusters using Silhouette Score and Dunn Index, which measure exactly the participant-separation structure that L_ps was designed to enforce. The reported improvement is therefore a check of the training objective rather than an independent confirmation of a discovered hierarchical structure. This is a secondary evaluation circularity, but it does not affect the main clustering derivation.
full rationale
The paper's central derivation chain is the contrastive clustering loss plus a Beta stick-breaking penalty applied per sample. The claimed 'infinite' clustering does not arise from a Dirichlet-process posterior: Eq. (2) is the exact inverse of Eq. (1), so the stick-breaking penalty is a deterministic function of the network's finite softmax outputs. The number of active clusters is controlled by the tunable hyperparameters α and λ_sb and the fixed truncation K, as Table 1 explicitly shows. The authors set α = 1.5 because they expected 10-20 clusters, and then present the resulting 20 clusters as a model output, which is a fitted input presented as a prediction. The participant-specific head ablation is also self-referential: the loss supervises participant identity and the evaluation measures participant identity separation, so the positive result is partly built into the objective. There is no load-bearing self-citation: the cited CC and DP works are external and are used for context, not to justify the present claims. Overall, the central 'infinite and inferred from data' claim reduces to hyperparameter choice, while the PSH evaluation is partially circular, giving a score of 6.
Assumptions & free parameters
free parameters (4)
- alpha (Beta(1, alpha) concentration) =
1.5 (convenience sample), 10 (PhotoEMA)
- lambda_sb (SB prior weight) =
1
- CCH output size K (truncation) =
40
- Cluster size threshold for outcome analysis =
>10 images, retaining 66 of 83 clusters
assumptions (5)
- standard math Stick-breaking construction and Beta(1, alpha) properties (Eq. 1-2)
- domain assumption Data augmentations in contrastive learning mimic real variability in repeated photos of the same environment
- domain assumption All images of one distinct environment belong to the same participant
- ad hoc to paper Deterministic softmax cluster probabilities can be treated as draws from a stick-breaking process
- domain assumption Manually assigned cluster labels (e.g., living room, kitchen) are valid ground truth for cluster evaluation
Cite this review
Pith. "Pith review of Infinite hierarchical contrastive clustering for personal digital envirotyping." pith.science (2026). https://pith.science/paper/A2IIL7CL
@misc{pith2026250515022,
author = {Pith},
title = {Pith review of: Infinite hierarchical contrastive clustering for personal digital envirotyping},
year = {2026},
howpublished = {\url{https://pith.science/paper/A2IIL7CL}},
note = {Machine review of arXiv:2505.15022}
}
read the original abstract
Daily environments have profound influence on our health and behavior. Recent work has shown that digital envirotyping, where computer vision is applied to images of daily environments taken during ecological momentary assessment (EMA), can be used to identify meaningful relationships between environmental features and health outcomes of interest. To systematically study such effects on an individual level, it is helpful to group images into distinct environments encountered in an individual's daily life; these may then be analyzed, further grouped into related environments with similar features, and linked to health outcomes. Here we introduce infinite hierarchical contrastive clustering to address this challenge. Building on the established contrastive clustering framework, our method a) allows an arbitrary number of clusters without requiring the full Dirichlet Process machinery by placing a stick-breaking prior on predicted cluster probabilities; and b) encourages distinct environments to form well-defined sub-clusters within each cluster of related environments by incorporating a participant-specific prediction loss. Our experiments show that our model effectively identifies distinct personal environments and groups these environments into meaningful environment types. We then illustrate how the resulting clusters can be linked to various health outcomes, highlighting the potential of our approach to advance the envirotyping paradigm.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Covid-19 lockdown: housing built environment’s effects on mental health
Andrea Amerio, Andrea Brambilla, Alessandro Morganti, Andrea Aguglia, Davide Bianchi, Francesca Santi, Luigi Costantini, Anna Odone, Alessandra Costanza, Carlo Signorelli, et al. Covid-19 lockdown: housing built environment’s effects on mental health. International journal of environmental research and public health, 17 0 (16): 0 5973, 2020
work page 2020
-
[2]
Deep clustering for unsupervised learning of visual features
Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European conference on computer vision (ECCV), pages 132--149, 2018
work page 2018
-
[3]
Paidamoyo Chapfuwa, Chunyuan Li, Nikhil Mehta, Lawrence Carin, and Ricardo Henao. Survival cluster analysis. In Proceedings of the ACM Conference on Health, Inference, and Learning, pages 60--68, 2020
work page 2020
-
[4]
Identifying smoking environments from images of daily life with deep learning
Matthew M Engelhard, Jason A Oliver, Ricardo Henao, Matt Hallyburton, Lawrence E Carin, Cynthia Conklin, and F Joseph McClernon. Identifying smoking environments from images of daily life with deep learning. JAMA Network Open, 2 0 (8): 0 e197939--e197939, 2019
work page 2019
-
[5]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016
2016
-
[6]
Sara FL Kirk, Tarra L Penney, and T-LF McHugh. Characterizing the obesogenic environment: the state of the evidence with directions for future research. Obesity reviews, 11 0 (2): 0 109--117, 2010
work page 2010
-
[7]
A neural dirichlet process mixture model for task-free continual learning
Soochan Lee, Junsoo Ha, Dongsu Zhang, and Gunhee Kim. A neural dirichlet process mixture model for task-free continual learning. arXiv preprint arXiv:2001.00689, 2020
arXiv 2001
-
[8]
Yunfan Li, Peng Hu, Zitao Liu, Dezhong Peng, Joey Tianyi Zhou, and Xi Peng. Contrastive clustering. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 8547--8555, 2021
work page 2021
Show all 18 references
-
[9]
Twin contrastive learning for online clustering
Yunfan Li, Mouxing Yang, Dezhong Peng, Taihao Li, Jiantao Huang, and Xi Peng. Twin contrastive learning for online clustering. International Journal of Computer Vision, 130 0 (9): 0 2205--2221, 2022
2022
-
[10]
Hippocampal and insular response to smoking-related environments: neuroimaging evidence for drug-context effects in nicotine dependence
F Joseph McClernon, Cynthia A Conklin, Rachel V Kozink, R Alison Adcock, Maggie M Sweitzer, Merideth A Addicott, Ying-hui Chou, Nan-kuei Chen, Matthew B Hallyburton, and Anthony M DeVito. Hippocampal and insular response to smoking-related environments: neuroimaging evidence f...
2016
-
[11]
N2d:(not too) deep clustering via clustering the local manifold of an autoencoded embedding
Ryan McConville, Raul Santos-Rodriguez, Robert J Piechocki, and Ian Craddock. N2d:(not too) deep clustering via clustering the local manifold of an autoencoded embedding. In 2020 25th international conference on pattern recognition (ICPR), pages 5145--5152. IEEE, 2021
2020
-
[12]
Environmental psychology
Gabriel Moser and David Uzzell. Environmental psychology. Comprehensive handbook of psychology, 5: 0 419--445, 2003
2003
-
[13]
The infinite gaussian mixture model
Carl Rasmussen. The infinite gaussian mixture model. Advances in neural information processing systems, 12, 1999
1999
-
[14]
Living environment and its relationship to depressive mood: A systematic review
Nina Rautio, Svetlana Filatova, Heli Lehtiniemi, and Jouko Miettunen. Living environment and its relationship to depressive mood: A systematic review. International journal of social psychiatry, 64 0 (1): 0 92--103, 2018
2018
-
[15]
Neighborhood conditions and psychosocial outcomes among middle-aged african americans: A cross-sectional analysis
Maya Tabet, Erin A Sanders, Mario Schootman, Jen Jen Chang, Fredric D Wolinsky, Theodore K Malmstrom, and Douglas K Miller. Neighborhood conditions and psychosocial outcomes among middle-aged african americans: A cross-sectional analysis. Journal of primary care & community he...
2017
-
[16]
Scan: Learning to classify images without labels
Wouter Van Gansbeke, Simon Vandenhende, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool. Scan: Learning to classify images without labels. In European conference on computer vision, pages 268--285. Springer, 2020
2020
-
[17]
Joint unsupervised learning of deep representations and image clusters
Jianwei Yang, Devi Parikh, and Dhruv Batra. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5147--5156, 2016
2016
-
[18]
Graph contrastive clustering
Huasong Zhong, Jianlong Wu, Chong Chen, Jianqiang Huang, Minghua Deng, Liqiang Nie, Zhouchen Lin, and Xian-Sheng Hua. Graph contrastive clustering. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9224--9233, 2021
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.