{"id":"1d8c6e94-2c5b-48ce-bf91-c662209c115a","arxiv_id":"2502.10350","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A taxonomically aware OTU clustering method that learns per-taxon genetic distance thresholds from reference sequences and clusters query sequences down the taxonomic hierarchy, with a full Illumina pipeline.","lead":"OptimOTU is a new method for grouping DNA sequences from environmental samples into species-like units, using a different genetic similarity cutoff for each taxonomic group. It comes with a complete data-processing pipeline for turning raw sequencing reads into ecological community tables.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No empirical validation supports the central claim: per-taxon optimized thresholds are untested, and the deployed clustering deviates from the single-linkage optimization, so the threshold's role is not established.","rationale":"The reader's conditional verdict is justified. The strongest claim is that taxonomically aware thresholds improve OTU clustering. For this to be true, (1) the threshold optimization must produce thresholds that generalize from reference sequences to query sequences, and (2) the clustering algorithm must apply those thresholds in a way consistent with the optimization. The paper provides no benchmark or cross-validation for (1), and it is internally questionable for (2): the optimization evaluates plain single-linkage partitions, while the clustering algorithm forces all sequences with the same taxonomic ID into a single core regardless of distance. This means the optimized threshold is only used for attaching unidentified sequences and for de novo clustering, not for grouping named taxa. Even if a benchmark showed the whole pipeline works, it would not establish that the threshold optimization is the reason. Therefore the load-bearing concern is the absence of any empirical test isolating the threshold mechanism. A mock-community benchmark comparing OptimOTU to a single-threshold baseline would settle this: if per-taxon thresholds do not beat a single threshold on known communities, the central claim fails. Agreement with the reader is partial because the reader focused on reference representativeness/generalization, which is the main issue, but we additionally flag the optimization/clustering mismatch. The verdict remains CONDITIONAL pending such validation.","tokens_in":18887,"tokens_out":6657,"duration_ms":68137,"concrete_test":"Run a mock-community benchmark with a defined set of species (e.g., a fungal ITS or arthropod COI mixture with known intra- and interspecific distances), including species with no close reference in the database. Optimize thresholds on a standard reference database, then run OptimOTU and compare its OTUs against the known species labels using AMI and F-measure. Compare to a single-threshold VSEARCH at 97%, to dnabarcoder, and to DADA2 ASVs, with error bars over multiple reference database compositions. If OptimOTU does not outperform the single-threshold baseline on the mock community, the central claim fails; this tests both threshold generalization and the utility of the taxonomy-constrained clustering.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that per-taxon optimized thresholds, derived from reference sequences, produce OTUs that better match species boundaries than a single global threshold. The paper contains no empirical validation of this claim: no mock community, no benchmark against existing tools (e.g., VSEARCH at 97%, dnabarcoder, DADA2+clustering), and no cross-validation of the threshold optimization. The only evidence is the algorithmic description and shipped code. Moreover, the thresholds are optimized on partitions from plain single-linkage hierarchical clustering of reference sequences, but the deployed clustering algorithm does not use single-linkage to group sequences with the same taxonomic label: cluster cores are formed by grouping all sequences with the same identification regardless of distance (Implementation section: 'cluster cores are formed by grouping sequences which have the same taxonomic identifications at the current rank'; Figure 3 example B). Thus the optimized threshold is not the threshold that determines the final partition for named taxa; it governs only closed-reference attachment of unidentified sequences and de novo clustering of pseudotaxa. The paper does not demonstrate that the threshold selected by matching single-linkage partitions to the reference taxonomy is the right radius for those attachment decisions, nor that per-taxon thresholds outperform a single global threshold in that role. The load-bearing assumption—that reference-based threshold optimization generalizes to query sequences, including taxa absent from the reference—is unsupported and, because of the optimization/clustering mismatch, not even internally established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents OptimOTU, an OTU clustering algorithm and bioinformatics pipeline for metabarcoding data. The algorithm first uses taxonomically identified reference sequences to estimate per-taxon genetic distance thresholds by comparing single-linkage partitions at various thresholds to reference taxonomy via AMI and other indices. Query sequences are then taxonomically classified, cluster cores are formed from identifications at each rank, unidentified sequences are attached by closed-reference clustering using optimized thresholds, and remaining sequences are clustered de novo into pseudotaxa. The paper also describes a full pipeline for paired-end Illumina data, including denoising, chimera removal, taxonomic assignment, and clustering, with parallelization options and multiple distance calculation methods. The R/C++ package and pipeline code are open source.","tokens_in":19167,"tokens_out":4259,"duration_ms":39911,"significance":"The per-taxon threshold idea is timely and the software is substantial: the implementation is open source, includes a test suite checking consistency of the clustering algorithms with base R hclust(), and the pipeline addresses real bioinformatics needs for large datasets. If the central claim is validated, taxonomically aware threshold optimization could improve OTU delimitation over global thresholds, especially in diverse communities with incomplete reference libraries. However, the paper's central claims are currently unsupported by empirical evidence: there is no benchmark, no mock community analysis, and no demonstration that thresholds generalize from reference sequences to query sequences. Because the clustering is constrained by the same taxonomy used for calibration, the reported congruence with taxonomy is partly by construction. The contribution is therefore a well-described algorithmic framework that needs substantial additional validation before the performance claims can be accepted.","major_comments":[{"comment":"The threshold optimization step is supervised calibration: thresholds are chosen to maximize AMI (or another index) between single-linkage partitions of reference sequences and the reference taxonomy. The same taxonomy is then used at the clustering stage to form cluster cores (all sequences with the same identification at the current rank are grouped regardless of distance). Consequently, any report that the final OTUs 'closely match' the taxonomy is partly by construction. The manuscript provides no independent validation—such as mock communities, leave-one-out cross-validation of thresholds, or a comparison against a global-threshold baseline—that would break this circularity and support the claim that optimized thresholds generalize to query sequences.","section":"Implementation — threshold optimization"},{"comment":"The optimization measures how well single-linkage partitions match the reference taxonomy, but the deployed clustering algorithm does not use single-linkage to form the final partition for taxonomically identified sequences. Cluster cores are formed by grouping all sequences with the same taxonomic identification regardless of distance (Figure 3, examples A and B), and species S5 is kept as one cluster despite one sequence lying outside the 'species-level threshold' from the others. The optimized threshold therefore governs only closed-reference attachment of unidentified sequences and de novo clustering of pseudotaxa, not the threshold that splits or merges named taxa. The paper does not demonstrate that thresholds selected by matching single-linkage partitions to the taxonomy are the right radii for those two roles, nor that per-taxon thresholds outperform a single global threshold in those roles. This disconnect needs to be addressed explicitly and tested.","section":"Implementation — clustering process"},{"comment":"The central claims of the paper—that OptimOTU produces OTUs that better reflect species boundaries than a single global threshold and that the pipeline scales to millions of reads per sample across tens of thousands of samples—are not supported by any empirical evaluation. There is no mock community analysis, no comparison with VSEARCH 97% clustering, DADA2 followed by clustering, or dnabarcoder, no cross-validation of the threshold optimization, and no runtime or memory benchmarks to substantiate the scaling statement in the abstract. Since these claims are quantitative and comparative, the current manuscript is a software description rather than a demonstrated method. A benchmark section with at least one public mock community and a comparison to standard tools is necessary to make the claims credible.","section":"Whole manuscript — no empirical validation"}],"minor_comments":[{"comment":"The text reads 'In the OptimOTU pipeine'; 'pipeine' should be 'pipeline'.","section":"Phase 1 — ASV table construction"},{"comment":"The text reads 'used to process demultipexed paired-end reads'; 'demultipexed' should be 'demultiplexed'.","section":"The OptimOTU pipeline"},{"comment":"The statement that SLINK 'operates at the theoretically optimal complexities of O(n2) time and O(n) space' should use superscript notation O(n^2), and if the complexity claim is intended literally, a citation or proof should be given; as written it is a typesetting issue.","section":"Clustering algorithms"},{"comment":"The sentence 'Dashed vertical lines indicate thresholds which optimize AMI for each rank, with ties resolved by selecting the median threshold and rounding down' is ambiguous: the median of which set of thresholds, and why rounding down? Please clarify.","section":"Threshold optimization — Figure 2 caption"},{"comment":"The citation for dnabarcoder (Vu, Nilsson, and Verkley 2022) appears to be a preprint posted on Authorea; if a peer-reviewed version is available, it should be cited instead.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a competent software description, but the absence of any validation is a serious gap for a methods journal. The authors should be pushed to add benchmarks, but I do not think the algorithmic idea is wrong; it is just unproven. I would not recommend reject at this stage. The relationship to dnabarcoder should also be clarified: the threshold optimization is explicitly similar, so the novelty rests on the taxonomically constrained hierarchical clustering and the full pipeline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: OptimOTU is a serious piece of software engineering, and the taxonomically aware clustering idea is worth discussing, but the paper doesn't demonstrate that it works. There's no mock community, no comparison to VSEARCH at 97% or dnabarcoder, no error analysis, and no scaling measurements. If you're in molecular ecology, this is a package you might want to watch, but the abstract's claims outrun the evidence.\n\nWhat's genuinely new: combining per-taxon optimized thresholds with taxonomy-constrained clustering and pseudotaxa for query sequences. Dnabarcoder optimizes cutoffs but doesn't use identifications to constrain cluster cores or prohibit merges. The implementation details matter: the R/C++ package, the SLINK and tree-based clustering algorithms with parallelization, external distance matrices via UNIX pipe, and the full `targets` pipeline for Illumina data. The code is on GitHub, and the algorithm is described precisely enough to reimplement. That's real work.\n\nThe soft spots are substantial. First, no empirical validation exists. The central claim is that the method produces OTUs that 'most closely match' descendant taxa, but there is no benchmark against existing tools, no mock community, no independent test of generalization. The only evidence is the algorithmic description.\n\nSecond, and more specifically, the threshold optimization is done on plain single-linkage partitions of reference sequences, but the deployed clustering doesn't use single-linkage for named taxa—it forms cluster cores from sequences with the same taxonomic label regardless of distance. So the optimized threshold only governs closed-reference attachment of unclassified sequences and de novo clustering of pseudotaxa. The paper doesn't show that a threshold tuned to make single-linkage clusters match taxonomy is the right radius for those attachment decisions. That's a mismatch between the calibration procedure and the deployment, not just a missing benchmark.\n\nThird, there's circularity in any reported congruence: the thresholds are fitted to the reference taxonomy, and the clustering is constrained by the same taxonomy, so the match is partly by construction.\n\nNone of this is fatal if the paper is repositioned as a software description with a research agenda. But as written, the 'optimal' claim is not supported. A serious referee should ask for a validation section with mock and real data, error bars, and a comparison to standard methods. My recommendation: send to peer review, but expect heavy revision.","headline":"Useful methods paper with a real software contribution, but the central claim about optimized thresholds is unproven: no benchmark, and the thresholds are tuned on a different clustering procedure than the one actually deployed.","tokens_in":19701,"tokens_out":2872,"would_cite":false,"duration_ms":28972,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OptimOTU clusters metabarcoding sequences using per-taxon genetic-distance thresholds learned from reference taxonomy, instead of one global cutoff, and claims this yields OTUs that better match species boundaries.","keywords":["OTU clustering","metabarcoding","taxonomically aware","genetic distance thresholds","single-linkage clustering","adjusted mutual information","taxonomic classification","bioinformatics pipeline"],"falsifier":"Construct a mock community with known species spanning several phyla, run OptimOTU on its sequences, and compare the recovered clusters to the true species either by adjusted Rand index or by counting merged and split species; if per-taxon optimized thresholds do not match true species boundaries better than the best single global threshold, the paper's central claim is wrong.","tokens_in":18736,"feed_emoji":"🧬","tokens_out":8302,"duration_ms":71245,"temperature":0.7,"pith_summary":"OptimOTU is a taxonomically aware method for clustering DNA metabarcoding sequences into operational taxonomic units (OTUs) — groups of similar sequences that ecological studies treat as species proxies. Instead of applying one genetic-distance cutoff to all sequences, it learns a separate optimal cutoff for each taxonomic group from a set of reference sequences with known taxonomy. The paper argues that these per-taxon thresholds, used together with the preliminary taxonomic identification of each query sequence, produce OTUs that reflect species boundaries more accurately than the common single-threshold approach. The method is distributed as software and is accompanied by a pipeline that processes paired-end Illumina reads into OTU tables at large scale.","feed_headline":"Per-taxon thresholds replace the one-size-fits-all OTU cutoff","feed_subtitle":"OptimOTU learns each group's best genetic distance from reference taxonomy, then clusters reads inside that group.","key_machinery":"The load-bearing mechanism is the per-taxon optimized threshold: a genetic-distance cutoff, chosen separately for each ancestor taxon and each taxonomic rank, that makes a single-linkage cut of reference sequences most closely reproduce their labelled descendant taxa as measured by adjusted mutual information. This threshold is what lets clustering adapt to taxa with different rates of within-species variation; thresholds are inherited down the hierarchy and are applied to pseudotaxa when a taxon lacks enough reference sequences to optimize its own.","core_discovery":"OptimOTU's central claim is that the genetic-distance threshold at which sequences should be grouped into OTUs is not a single number but a property of each taxon and each rank, and that the right thresholds can be learned from a reference set whose sequences already carry taxonomic labels. The algorithm builds a single-linkage hierarchy of the reference sequences, cuts it at a grid of candidate thresholds, and scores each cut against the reference taxonomy at every rank using adjusted mutual information; the threshold that best recovers the named descendant taxa is stored for that ancestor taxon. Query sequences are first given a preliminary taxonomic identification with a stopping condition, then clustered rank by rank. Within each rank, sequences that share an identification at that rank form cluster cores regardless of distance; unidentified sequences are attached to the nearest core by closed-reference clustering at the learned threshold, with iterations that mimic single linkage; and sequences still unattached are clustered de novo, receiving placeholder 'pseudotaxa' names. The output is a full hierarchical classification in which every OTU is either a named taxon or a pseudotaxon, and the clustering is constrained to be congruent with the preliminary taxonomy.","pith_inferences":["A consequence the paper does not draw is that the threshold optimization could be repurposed for other grouping objectives, such as functional genes or ecological guilds, by swapping the reference labels; nothing in the algorithm itself is species-specific.","The paper's reliance on reference taxonomy implies that a biased or mislabeled reference database will systematically miscalibrate every threshold inherited from it; this makes the method's real-world accuracy sensitive to database quality in ways a mock-community benchmark could quantify.","A testable extension would be to measure whether the benefit of per-taxon thresholds grows as the taxonomic breadth of a dataset increases, since the single-threshold approach should degrade most where lineages differ most in their intraspecific variation."],"forward_implications":["A single metabarcoding dataset spanning several phyla will no longer have to be clustered with one threshold that over-merges some groups and over-splits others; per-taxon thresholds should reduce both errors within the same dataset.","Because clustering is constrained by the preliminary taxonomy, the resulting OTU table comes with a complete rank-by-rank classification, with pseudotaxa standing in for organisms that have no named close relative in the reference database.","Optimized thresholds are a reusable resource: computing them once for a marker gene and taxonomic group lets future studies cluster new samples without repeating the expensive optimization step.","The pipe-based external distance interface removes the need to hold a full distance matrix in memory, which is what allows the pipeline to scale to datasets with millions of reads per sample."],"supporting_citations":[{"why":"Supplies the threshold-optimization approach that OptimOTU adapts: choosing a cutoff that makes clusters best match reference taxonomy.","marker":"Vu, Nilsson, and Verkley 2022"},{"why":"Provides the adjusted mutual information measure used to score each candidate threshold against the reference taxonomy.","marker":"Vinh, Epps, and Bailey 2010"},{"why":"Supplies the SLINK single-linkage algorithm used to build the hierarchical clustering tree in threshold optimization and clustering.","marker":"Sibson 1973"},{"why":"Supplies the Edlib edit-distance library used as one of the internal pairwise sequence alignment methods.","marker":"Šošić and Šikić 2017"},{"why":"Supplies the WFA2 wavefront alignment library, the default pairwise distance method in the package.","marker":"Marco-Sola et al. 2021"},{"why":"Supplies USEARCH for heuristic sparse-distance calculation and closed-reference search, and represents the single-threshold clustering approach the paper contrasts with.","marker":"Edgar 2010"}],"fun_headline_variants":["No single OTU cutoff: OptimOTU learns per-taxon thresholds","OptimOTU adapts OTU thresholds per taxon, not one-size-fits-all","Per-taxon OTU thresholds from reference taxonomy","Learn the right OTU threshold for each taxon automatically","Clustered OTUs: per-taxon distance thresholds beat global cutoff"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the reference sequences and their taxonomic labels are accurate and representative enough that the threshold which best matches the reference taxonomy for a taxon will also be the right threshold for query sequences from that taxon.","fun_headline_variants_meta":{"raw":{"variants":["No single OTU cutoff: OptimOTU learns per-taxon thresholds","OptimOTU adapts OTU thresholds per taxon, not one-size-fits-all","Per-taxon OTU thresholds from reference taxonomy","Learn the right OTU threshold for each taxon automatically","Clustered OTUs: per-taxon distance thresholds beat global cutoff"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000641,"raw_usage":{"total_tokens":3349,"prompt_tokens":1117,"completion_tokens":2232,"prompt_tokens_details":{"cached_tokens":1024},"prompt_cache_hit_tokens":1024,"prompt_cache_miss_tokens":93,"completion_tokens_details":{"reasoning_tokens":2140}},"tokens_in":93,"tokens_out":2232,"duration_ms":21500,"temperature":1.0,"reasoning_tokens":2140,"cache_read_input_tokens":1024,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T18:19:04.459436+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a mock community with known species spanning several phyla, run OptimOTU on its sequences, and compare the recovered clusters to the true species either by adjusted Rand index or by counting merged and split species; if per-taxon optimized thresholds do not match true species boundaries better than the best single global threshold, the paper's central claim is wrong.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the threshold-optimization approach that OptimOTU adapts: choosing a cutoff that makes clusters best match reference taxonomy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the adjusted mutual information measure used to score each candidate threshold against the reference taxonomy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies USEARCH for heuristic sparse-distance calculation and closed-reference search, and represents the single-threshold clustering approach the paper contrasts with."}],"review_version":1}