REVIEW 4 major objections 6 minor 11 references
CMiNet: R package for learning the Consensus Microbiome Network
T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Ten microbiome algorithms merged into one consensus network
desk verdict A convenient R wrapper with a simple majority-vote consensus, but the paper's central reliability claim is unsupported by any ground-truth validation or comparison to individual methods. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the weighted consensus network matrix: an $n \times n$ matrix whose entries count how many of the included algorithms place an interaction between each pair of taxa, together with the thresholding step that converts those counts into a final network. The novel methodological ingredient is CMIMN, a conditional mutual information algorithm that captures non-linear conditional dependencies between taxa, complementing the nine existing methods. The weight-by-agreement construction does the argument's work: it converts a set of divergent binary network estimates into a single graded confidence score, lets users choose stringency via a threshold, and supplies Hamming distances as a measure of structural disagreement among methods.
What would settle it
Construct a synthetic microbial community with known interaction structure, simulate compositional count data from it, run CMiNet, and compare the consensus network with each individual method's output; if a single method recovers the true edges more accurately than the consensus, or if a consensus built from deliberately biased but mutually agreeing methods still looks confident, the claim that consensus implies reliability would be refuted.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that no individual network inference method should be trusted as the sole picture of microbial interactions, because different algorithms—correlation-based, inverse-covariance based, and compositional-data specific—produce substantially different networks from the same data. CMiNet addresses this by running all ten methods and recording, for each pair of taxa, how many methods place an edge between them; the result is a weighted consensus network matrix in which edge weight equals the level of cross-method agreement. The authors show that thresholding this weight changes network size monotonically (for example, threshold 9 gives 47 edges, threshold 8 gives 94), and they argue that edges confirmed by all or most methods are more stable and reliable than those found by a single algorithm. The package also provides Hamming-distance comparisons, visualization, and per-algorithm parameter control. The paper's claim is that this consensus output is a more stable and comprehensive representation of microbial interactions than any single-method network, while acknowledging that no gold standard exists against which to check accuracy.
Load-bearing premise
The load-bearing premise is that agreement among multiple inference methods is a trustworthy sign of a real microbial interaction, even though the paper does not test this against known true networks.
Editorial extensions
If this is right
- Researchers can set a threshold to trade off sensitivity and specificity: high thresholds keep only edges every method agrees on, while lower thresholds admit edges supported by most methods.
- Edge weights provide a per-interaction confidence score that can be used to prioritize candidate microbial interactions for experimental validation.
- The package broadens applicability by handling quantitative and compositional data through algorithm-specific defaults and customizable parameters.
- The consensus network can serve as a reference or benchmark when comparing newly developed network inference algorithms.
- Because the package is open-source R, reproducible consensus networks can become a routine part of microbiome analysis pipelines.
Reading between the lines
- The paper's confidence argument is only as strong as the independence of the methods' errors; if several algorithms share the same bias, their agreement may overstate confidence rather than accuracy, a possibility the authors do not test empirically.
- A natural testable extension is a synthetic benchmark with known interaction ground truth, comparing consensus recovery against each single method; if consensus does not beat the best method in such simulations, its value would be stability rather than accuracy.
- Since CMIMN is new and only described in a manuscript in preparation, the package's 'nine plus one' consensus depends on an algorithm not yet independently validated; a standalone description of CMIMN's properties would strengthen the consensus claim.
- The reported thresholding behavior suggests edge counts are highly concentrated near full agreement, so users may want guidance on choosing thresholds a priori rather than by inspecting network size.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents CMiNet, an R package that constructs a consensus microbiome network by integrating outputs from ten network inference methods (Pearson, Spearman, Bicor, SparCC, SpiecEasi-MB, SpiecEasi-glassso, SPRING, GCoDA, CCLasso, and a novel CMIMN). The package provides functions for network construction, threshold-based visualization, and Hamming distance comparison. The central claim is that the consensus network is more stable and comprehensive than any single method, and that it ensures more robust and reliable inference. The paper includes an example application with node/edge counts at thresholds, but no quantitative evaluation of accuracy.
Significance. If the consensus approach were validated, the package would be a convenient and potentially useful tool for microbiome researchers, bundling diverse methods in one interface with customisable thresholds. The strengths are open availability, broad coverage of methods, and a clear software structure with documented functions. However, the paper's advertised advantages (stability, reliability, comprehensiveness) are asserted rather than demonstrated. There is no simulation with known ground truth, no comparison against individual methods, no uncertainty quantification, and no description of the example dataset. As presented, the evidence is entirely internal to the method, making the main claims unsupported. A validated version of the package would be a solid contribution to applied microbiome research.
major comments (4)
- [Abstract; Example Application] The central claim, that CMiNet provides 'a more stable and comprehensive representation' (Abstract) and 'ensures more robust and reliable network inference' (Example Application), is not supported by any quantitative evaluation. No simulation with known ground truth is performed, no comparison of the consensus network to individual algorithms is reported, and no stability analysis across data subsamples is provided. The only numerical results are node and edge counts at thresholds in Figure 1b, which are descriptive and do not measure accuracy or robustness.
- [Example Application; Conclusion] The manuscript uses the absence of a gold standard ('the lack of a gold standard in microbiome research', Example Application) to justify the consensus approach, but it never demonstrates that agreement among algorithms is a valid proxy for accuracy. Many of the included methods share underlying assumptions (e.g., compositional log-ratio transformations or correlation heuristics), so biases may be common rather than independent; a majority vote could amplify shared false edges. A synthetic benchmark with known interactions is required to test whether consensus improves edge recovery relative to individual methods.
- [Introduction; Table 1] CMIMN, described as a novel algorithm, is never formally specified. There is no equation or algorithmic description for the conditional mutual information estimator, and the thresholds q1 and q2 are listed in Table 1 without any explanation of how they are chosen or what they control. The only reference for CMIMN is an in-preparation manuscript by the same authors [10], so the method cannot be independently evaluated or reproduced from this paper.
- [Example Application; Figure 1] The dataset used to generate Figure 1b is not described: the text does not state the data source, sample size, type of sequencing or count data, preprocessing, or number of taxa. Without this context, the reported node/edge counts at thresholds 8 and 9 cannot be interpreted or reproduced, and the example does not illustrate the method's utility in a concrete setting.
minor comments (6)
- [Abstract] The abstract says CMiNet incorporates 'nine widely used algorithms' but lists eight before introducing CMIMN; the Introduction lists nine widely recognized methods plus CMIMN, for a total of ten. Please make the counts consistent.
- [Introduction] The CMIMN bullet contains the typo 'cotructing'; it should be 'constructing'.
- [Table 1] The terms 'quantitative' and 'non-quantitative' are not defined. Clarify whether 'non-quantitative' means relative abundance/compositional data or something else.
- [Table 1] The algorithm labels 'SE_mb' and 'SE_glasso' in Table 1 do not match the names 'SpiecEas_MB' and 'SpiecEasi_Glasso' used in the Introduction; use consistent spelling throughout.
- [Conclusion] The statement that CMiNet 'can serve as a benchmark for comparing newly developed algorithms' is an overclaim in the absence of validation; consider rewording to say the package provides a common reference point.
- [Example Application] The choice of threshold (e.g., 9 vs. 8) is presented as a user preference without any guidance on how to select a threshold based on network properties; the package would benefit from a built-in diagnostic.
Circularity Check
Consensus reliability is asserted from the agreement counts that define the consensus, with no external validation.
-
self definitional
[Example Application, IMPLEMENTATIONS section, last paragraph]
"Given the significant differences in the outputs of various algorithms and the lack of a gold standard in microbiome research for evaluating relationships between microbes, using a consensus approach ensures more robust and reliable network inference."
The consensus network is built by counting how many algorithms confirm each edge; Figure 1b defines thresholds as 'edges confirmed by all ten methods' and 'confirmed by nine or all ten methods.' The paper's sole argument that this network is 'more robust and reliable' is that edges are confirmed by all or most algorithms—the same counts that define the consensus output. With no gold standard, synthetic benchmark, subsample stability analysis, or recovery comparison, the reliability claim is asserted to follow from the construction itself, so the conclusion restates the consensus definition as its own evidence.
full rationale
The manuscript is a software/application note, so most of its content (package functions, thresholds, example run) is not a derivation chain. The one load-bearing interpretive claim—that the consensus network is 'more robust and reliable'—is supported only by the fact that edges are confirmed by multiple algorithms. Because the consensus edge weight is precisely a count of confirming algorithms, this is a self-referential justification rather than a tested prediction. The CMIMN algorithm is cited to the authors' own in-preparation work [10], but since nine established methods also contribute to the consensus, this self-citation is not load-bearing for the consensus claim; it is a separate weakness. No equation-level circularity exists beyond the agreement-count tautology, and the package itself is functional and reproducible. Score 6 reflects partial circularity in the central reliability claim, not in the software implementation.
Assumptions & free parameters
free parameters (2)
- CMIMN q1 =
0.7
- CMIMN q2 =
0.95
assumptions (3)
- domain assumption The consensus of multiple algorithms is more reliable than any single algorithm
- domain assumption Agreement among methods is a proxy for accuracy in the absence of a gold standard
- ad hoc to paper CMIMN correctly estimates conditional dependencies
Cite this review
Pith. "Pith review of CMiNet: R package for learning the Consensus Microbiome Network." pith.science (2026). https://pith.science/paper/Q2ZTEOZC
@misc{pith2026241108309,
author = {Pith},
title = {Pith review of: CMiNet: R package for learning the Consensus Microbiome Network},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q2ZTEOZC}},
note = {Machine review of arXiv:2411.08309}
}
read the original abstract
Understanding complex interactions within microbiomes is essential for exploring their roles in health and disease. However, constructing reliable microbiome networks often poses a challenge due to variations in the output of different network inference algorithms. To address this issue, we present CMiNet, an R package designed to generate a consensus microbiome network by integrating results from multiple established network construction methods. CMiNet incorporates nine widely used algorithms, including Pearson, Spearman, Biweight Midcorrelation (Bicor), SparCC, SpiecEasi, SPRING, GCoDA, and CCLasso, along with a novel algorithm based on conditional mutual information (CMIMN). By combining the strengths of these algorithms, CMiNet generates a single, weighted consensus network that provides a more stable and comprehensive representation of microbial interactions. The package includes customizable functions for network construction, visualization, and analysis, allowing users to explore network structures at different threshold levels and assess connectivity and reliability. CMiNet is designed to handle both quantitative and compositional data, ensuring broad applicability for researchers aiming to understand the intricate relationships within microbiome communities. Availability: Source code is freely available at https://github.com/solislemuslab/CMiNet.
Figures
Reference graph
Works this paper leans on
-
[10]
Leveraging machine learning and enhanced network-based methods in potato disease interactions
Rosa Aghdam, Shan Shan, Richard Lankau, and Claudia Solís-Lemus. Leveraging machine learning and enhanced network-based methods in potato disease interactions. In preparation, 2024
work page 2024
-
[1]
Microbial interactions: from networks to models
Karoline Faust and Jeroen Raes. Microbial interactions: from networks to models. Nature Reviews Microbiology, 10(8):538–550, 2012
work page 2012
-
[2]
Netcomi: network construction and comparison for microbiome data in r
Stefanie Peschel, Christian L Müller, Erika V on Mutius, Anne-Laure Boulesteix, and Martin Depner. Netcomi: network construction and comparison for microbiome data in r. Briefings in bioinformatics, 22(4):bbaa290, 2021. 4
work page 2021
-
[3]
Human Limits in Machine Learning: Prediction of Plant Phenotypes Using Soil Microbiome Data
Rosa Aghdam, Xudong Tang, Shan Shan, Richard Lankau, and Claudia Solís-Lemus. Human limits in machine learning: Prediction of plant phenotypes using soil microbiome data. arXiv preprint arXiv:2306.11157, 2023
work page Pith review arXiv 2023
-
[4]
Networks as tools for defining emergent properties of microbiomes and their stability
Kacie T Kajihara and Nicole A Hynson. Networks as tools for defining emergent properties of microbiomes and their stability. Microbiome, 12(1):184, 2024
work page 2024
-
[5]
Jonathan Friedman and Eric J Alm. Inferring correlation networks from genomic survey data.PLoS computational biology, 8(9):e1002687, 2012
work page 2012
-
[6]
Sparse and compositionally robust inference of microbial ecological networks
Zachary D Kurtz, Christian L Müller, Emily R Miraldi, Dan R Littman, Martin J Blaser, and Richard A Bonneau. Sparse and compositionally robust inference of microbial ecological networks. PLoS computational biology, 11(5):e1004226, 2015
work page 2015
-
[7]
Grace Yoon, Irina Gaynanova, and Christian L Müller. Microbial networks in spring-semi-parametric rank-based correlation and partial correlation estimation for quantitative microbiome data. Frontiers in genetics, 10:516, 2019
work page 2019
Show all 11 references
-
[8]
gcoda: conditional dependence network inference for compositional data
Huaying Fang, Chengcheng Huang, Hongyu Zhao, and Minghua Deng. gcoda: conditional dependence network inference for compositional data. Journal of Computational Biology, 24(7):699–708, 2017
2017
-
[9]
Cclasso: correlation inference for compositional data through lasso
Huaying Fang, Chengcheng Huang, Hongyu Zhao, and Minghua Deng. Cclasso: correlation inference for compositional data through lasso. Bioinformatics, 31(19):3172–3180, 2015
2015
-
[11]
Minaa: Microbiome network alignment algorithm
Reed Nelson, Rosa Aghdam, and Claudia Solis-Lemus. Minaa: Microbiome network alignment algorithm. Journal of Open Source Software, 9(96):5448, 2024. 5
2024
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.