pith. sign in

arxiv: 2605.26268 · v2 · pith:TCFJ3FOInew · submitted 2026-05-25 · ⚛️ physics.soc-ph

Detecting Hierarchical Clusters and Estimating their Modularity Directly from Dendrograms

Alexandre Benatti , Luciano da F. Costa This is my paper

Pith reviewed 2026-06-29 19:01 UTC · model grok-4.3

classification ⚛️ physics.soc-ph
keywords hierarchical clusteringdendrogramsmodularitycommunity detectionpeak detectionmerging densityscale variable
0
0 comments X

The pith

Balancing the merging density function from a dendrogram followed by peak detection recovers hierarchical clusters and their modularity values.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper describes a procedure that starts from a dendrogram of data relationships and builds a merging density function that records how subclusters combine as a scale parameter increases. This function is then balanced across the scale, after which peak detection locates the positions and strengths of clusters at different resolutions. The approach is presented as sufficient to estimate both the clusters themselves and a measure of their hierarchical modularity without returning to the original data matrix. A reader would care because many pattern-recognition tasks rely on hierarchical representations yet currently require additional steps to quantify modularity once the dendrogram is obtained.

Core claim

After the mergings of subclusters along the scale variable are obtained to form a merging density function, balancing that function along the scale and applying peak detection estimates the respective hierarchical clusters and their hierarchical modularity within a specified resolution. The method is illustrated on example data and dendrograms, and the possibility of applying the procedure recursively is noted.

What carries the argument

The merging density function obtained by counting subclusters merges along the scale variable; balancing it and locating peaks extracts both cluster boundaries and modularity values.

If this is right

  • The same dendrogram can be processed at multiple resolutions to obtain nested cluster descriptions.
  • Modularity estimates become available directly from the hierarchy without separate community-detection passes.
  • Recursive application of the peak-detection step can refine clusters at finer scales.
  • The procedure applies to any dendrogram regardless of the original data type.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method could reduce computational cost for very large networks by operating only on the already-computed dendrogram.
  • It may extend naturally to settings where the scale variable represents time or another ordered parameter.
  • Handling of ties or plateaus in the merging density would need explicit rules if the method is to be applied automatically.

Load-bearing premise

Balancing the merging density function and detecting its peaks will correctly recover the true hierarchical clusters and modularity from the dendrogram alone.

What would settle it

Generate a dendrogram from data whose true hierarchical clusters are already known, apply the balancing-plus-peak procedure, and check whether the detected peaks and their modularity values match the known structure.

Figures

Figures reproduced from arXiv: 2605.26268 by Alexandre Benatti, Luciano da F. Costa.

Figure 1
Figure 1. Figure 1: Example of a less modular (a) and a more modular (c) data sets, with the x-y [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A simple dataset (a) and dendrograms obtained by using three distinct linkage [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Illustration of the estimation of the functions [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: A dendrogram with two main hierarchical levels is shown in (a). Its unbalanced [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Example of detected maximum and minimum peaks of [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Illustration of cluster detection for the dendrogram in (a). The two identified [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Illustration of the main variables required for modularity estimation, shown [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The hierarchical clusters obtained for the dendrogram in Figure 11 by using [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Four distinct dendrograms (a-d) and their respective modularity in terms of the [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Illustration of clusters hierarchically detected from a dendrogram at three [PITH_FULL_IMAGE:figures/full_fig_p015_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Superimposition of the identified delimiting levels on the original dendrogram [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Example of a dendrogram (a) obtained from the distribution of data elements [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Further analysis of the sub-dendrogram marked by the asterisk in Figure 12(a), [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Example of a dendrogram (a) obtained by average linkage criterion from the [PITH_FULL_IMAGE:figures/full_fig_p018_14.png] view at source ↗
read the original abstract

Identifying possible clusters in datasets and estimating their hierarchical modularity are central tasks in pattern recognition. In the present work, concepts and methodologies are described for performing these tasks while considering only the density of mergings obtained from hierarchical representations (dendrograms) of data inter-relationship along a scale variable. More specifically, the mergings of subclusters along the scale variable are obtained, yielding a respective merging density function. After this function is balanced along the scale variable, peak detection is applied in order to estimate, within a specified resolution, the respective hierarchical clusters and their hierarchical modularity. The potential of the reported approach is illustrated for some types of data and dendrograms, and the possibility of recursive cluster detection is also considered.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript describes a heuristic procedure to detect hierarchical clusters and estimate their modularity directly from dendrograms: a merging density function is computed from the sequence of subclusters along the scale variable; this function is balanced; peak detection is then applied at a chosen resolution to recover the clusters and their modularity values. The approach is illustrated on selected data types, and recursive application is discussed.

Significance. If the procedure can be shown to recover ground-truth structure reliably, it would supply a practical, dendrogram-only route to hierarchical modularity estimation that bypasses the original data matrix. The absence of any quantitative validation or comparison against known partitions, however, leaves the practical utility and accuracy of the estimates unestablished.

major comments (2)
  1. [Abstract and method description] The central claim rests on the balancing step of the merging density function followed by peak detection, yet no mathematical definition, pseudocode, or parameter specification for balancing is supplied (abstract and method description). Without this, it is impossible to determine whether the subsequent peak detection yields reproducible or meaningful modularity estimates.
  2. [Illustrations and results] No validation results, error metrics, or comparisons against ground-truth partitions are reported for any of the illustrated data types. This omission is load-bearing because the weakest assumption—that balancing plus peak detection recovers the true hierarchical clusters and modularity—remains untested.
minor comments (1)
  1. The abstract refers to “some types of data and dendrograms” without naming the specific datasets or dendrogram construction methods used in the illustrations, which hinders reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed comments. We address each major point below and describe the revisions that will be incorporated.

read point-by-point responses

  1. Referee: [Abstract and method description] The central claim rests on the balancing step of the merging density function followed by peak detection, yet no mathematical definition, pseudocode, or parameter specification for balancing is supplied (abstract and method description). Without this, it is impossible to determine whether the subsequent peak detection yields reproducible or meaningful modularity estimates.

    Authors: We agree that the absence of an explicit mathematical definition for the balancing operation prevents full reproducibility. The revised manuscript will add the precise formula used to balance the merging density function along the scale variable, together with pseudocode for the complete procedure and specification of the resolution parameter employed in peak detection. revision: yes

  2. Referee: [Illustrations and results] No validation results, error metrics, or comparisons against ground-truth partitions are reported for any of the illustrated data types. This omission is load-bearing because the weakest assumption—that balancing plus peak detection recovers the true hierarchical clusters and modularity—remains untested.

    Authors: The present manuscript emphasizes the methodological framework and provides illustrative applications rather than a comprehensive validation study. We acknowledge that quantitative assessment against known partitions is necessary to substantiate the recovery claims. The revision will include a dedicated validation section using synthetic datasets with planted hierarchical structure, reporting error metrics and direct comparisons to established hierarchical clustering methods. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper describes a heuristic procedure that computes a merging density function directly from a given dendrogram, balances that function along the scale variable, and applies peak detection to identify clusters and modularity values. No equations, definitions, or self-citations are presented that reduce the output modularity estimates or cluster detections to fitted parameters or inputs defined from the same data by construction. The method operates on the dendrogram structure as an external input and makes no claim of a first-principles derivation that collapses into its own assumptions. This is a standard non-circular empirical technique.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that a balanced merging density plus peak detection faithfully recovers hierarchical structure; no free parameters or invented entities are mentioned.

axioms (1)
  • domain assumption Balancing the merging density function along the scale variable followed by peak detection recovers the hierarchical clusters and their modularity.
    This premise is required for the peak-detection step to produce the claimed estimates.

pith-pipeline@v0.9.1-grok · 5649 in / 1080 out tokens · 39234 ms · 2026-06-29T19:01:50.607278+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references

  1. [1]

    R. O. Duda, P. E. Hart, and D. G. Stork.Pattern Classification. Wiley- Interscience New York, 2nd edition, 2000

    2000

  2. [2]

    Theodoridis and K

    S. Theodoridis and K. Koutroumbas.Pattern Recognition. Elsevier, 2006

    2006

  3. [3]

    A. K. Jain and R. C. Dubes.Algorithms for Clustering Data. Prentice- Hall, Inc., 1988

    1988

  4. [4]

    A. K. Jain, M. N. Murty, and P. J. Flynn. Data Clustering: A review. ACM Computing Surveys (CSUR), 31(3):264–323, 1999

    1999

  5. [5]

    M. Z. Rodriguez, C. H. Comin, D. Casanova, O. M. Bruno, D. R. Aman- cio, L. da F. Costa, and F A. Rodrigues. Clustering Algorithms: A comparative approach.PloS One, 14(1):e0210236, 2019

    2019

  6. [6]

    R. S. Hill. A Stopping Rule for Partitioning Dendrograms.Botanical Gazette, 141(3):321–324, 1980

    1980

  7. [7]

    de Ridder, J

    D. de Ridder, J. De Ridder, and M. J. T. Reinders. Pattern Recognition in Bioinformatics.Briefings in Bioinformatics, 14(5):633–647, 2013

    2013

  8. [8]

    Karna and K

    A. Karna and K. Gibert. Automatic identification of the number of clusters in hierarchical clustering.Neural Computing and Applications, 34(1):119–134, 2022

    2022

  9. [9]

    E. O. Brigham.The Fast Fourier Transform and Its Applications. Pren- tice Hall, Englewood Cliffs, NJ, 2 edition, 1988

    1988

  10. [10]

    Perona, T

    P. Perona, T. Shiota, and J. Malik. Anisotropic Diffusion. InGeometry- Driven Diffusion in Computer Vision, pages 73–92. Springer, 1994

    1994

  11. [11]

    M. J. Black, G. Sapiro, D. H. Marimont, and D. Heeger. Ro- bust Anisotropic Diffusion.IEEE Transactions on Image Processing, 7(3):421–432, 1998

    1998

  12. [12]

    Weickert.Anisotropic Diffusion in Image Processing

    J. Weickert.Anisotropic Diffusion in Image Processing. Teubner, 1998

    1998

  13. [13]

    scipy.signal.find peaks, 2025

    SciPy Community. scipy.signal.find peaks, 2025. Accessed: 2026-05-15

    2025

  14. [14]

    R. R. Sokal and F. J. Rohlf. The Comparison of Dendrograms by Ob- jective Methods.Taxon, 11(2):33–40, 1962

    1962

  15. [15]

    T. M. J. Fruchterman and E. M. Reingold. Graph Drawing by Force- Directed Placement.Software: Practice and Experience, 21(11):1129– 1164, 1991. 19

    1991