REVIEW 4 major objections 6 minor 38 references
GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read GPT-HTree claims that clustering founders into personas and fitting a decision tree inside each cluster yields explainable, segment-specific success estimates, with one cluster reaching 17.4% success versus a 1.9% random baseline.
desk verdict The LLM persona idea is worth a look, but the headline 9x success-rate claim is uninterpretable because the paper never calibrates the resampled cluster rates back to the original baseline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is GPT-HTree itself, a sequential framework combining CTGAN-based resampling to balance class distributions, agglomerative hierarchical clustering to form eight founder personas, per-cluster decision trees built with standard impurity-based splitting, and a structured GPT-4 prompt pipeline that turns z-score feature deviations and importance scores into natural-language persona summaries. The resampling is meant to widen the separation between successful and unsuccessful founders so the clusters and trees can learn sharper boundaries; the per-cluster trees are meant to provide local, explainable classification; and the LLM component is meant to bridge statistical outputs and human judgment.
What would settle it
Re-run the full pipeline on the original, unresampled 8,800-founder dataset with the same cluster definitions and per-cluster trees on held-out profiles; if the serial-exit cluster's success advantage drops from roughly 22x to near 1x, the headline multipliers are rebalancing artifacts rather than real-world probabilities.
Extended reading notes
Core claim
The paper claims that the GPT-HTree framework identifies explainable founder personas whose measured success rates far exceed the random baseline: one cluster, serial-exit founders, reaches 17.4% success, about nine times the 1.9% baseline, and is 22 times more likely to succeed than early professionals. The intended mechanism is that global decision trees fail on heterogeneous populations, whereas first grouping individuals by feature similarity and then learning localized decision paths captures the different determinants of success within each segment. The paper further claims that LLMs can convert quantitative cluster characteristics and feature-importance scores into human-readable persona descriptions, making the whole pipeline actionable for investor decision-making.
Load-bearing premise
The reported success rates are treated as real-world probabilities even though they were computed after synthetic resampling that deliberately widened the gap between clusters; if that resampling does not preserve the true link between founder features and success, the multipliers overstate reality.
Editorial extensions
If this is right
- If the resampled data preserve the real feature-success relationship, venture capitalists could use segment-specific probability estimates instead of one global baseline when sourcing deals.
- A new founder profile can be assigned to the nearest persona cluster and classified by that cluster's decision tree, giving an explanation and confidence score alongside the prediction.
- Feature-importance analysis within each cluster can reveal that success depends on different traits in different persona groups, which a single global decision tree would obscure.
- The same cluster-then-tree architecture with LLM-generated descriptions could be transferred to other high-stakes, heterogeneous domains such as healthcare, as the paper itself proposes.
Reading between the lines
- If the reported 9x and 22x multipliers later prove to be artifacts of the CTGAN rebalancing rather than real relationships, the practical value of the method may shift from the point estimates to the qualitative persona segmentation and the sourcing narratives it produces.
- A direct test that would strengthen the framework is whether the cluster personas remain stable and predictive on a fresh cohort of founders whose outcomes were not used in resampling or clustering.
- The LLM-generated persona descriptions could be evaluated by whether investors who read them make better sourcing decisions than investors given only the raw feature scores, since the stated goal is actionable insight rather than statistical novelty.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GPT-HTree, a pipeline that combines CTGAN resampling, hierarchical clustering, per-cluster decision trees, and GPT-4-generated persona descriptions, applied to a proprietary dataset of 8,800 startup founders. The central empirical claim is that the method identifies explainable founder clusters with success probabilities up to 17.4%, about nine times the 1.9% random baseline, and that serial-exit founders are 22x more likely to succeed than early professionals. The manuscript also describes the training, classification, and persona-generation algorithms, presents a prompt template, and defines feature-importance formulas.
Significance. If the quantitative claims were supported, the framework would provide a practical interpretable segmentation tool for venture-capital deal sourcing and a useful template for combining unsupervised clustering with LLM-based explanations. The paper contributes a clear algorithmic description (Algorithms 1-3), explicit decision-tree feature-importance definitions (Section 4.5), and a reusable prompt template (Figure 3). However, the current evidence does not establish the headline multipliers: the success-rate comparison is confounded by CTGAN resampling, the tables are internally inconsistent, and no held-out evaluation or baseline comparison is provided. The framework's contribution is therefore primarily conceptual at this stage.
major comments (4)
- [Section 3.2, Tables 1-2, Table 4] The headline claim in the abstract and Section 1.2 that GPT-HTree identifies clusters with success probabilities 'up to 9 times higher' than the 1.9% baseline relies on comparing Table 4's rates (e.g., 17.4% and 0.8%) with a real-world base rate. Section 3.2 explicitly shows that CTGAN resampling changed normalized success rates from 4.9-19.75% (Table 1) to 3.9-50.0% (Table 2), and Algorithm 1 inserts ResampleData before clustering. Table 4 is never labeled as pre- or post-resampling, and no calibration step maps resampled rates back to the original scale. If Table 4 is post-resampling, the comparison to 1.9% is invalid because balancing a 1.9% base rate changes the class distribution and hence the success rates; if it is pre-resampling, the paper does not say so or explain the relationship. The 9x and 22x multipliers are therefore not established.
- [Table 1 vs. Table 3] Table 3 is introduced as summarizing Table 1, but the two tables disagree on the same quantities. Table 1 reports the main cluster's success rate as 8.7% (48 successes / 103 total), while Table 3 reports 46.6% for the same cluster; for Sub S.1, Table 1 reports 19.75% (17/18) while Table 3 reports 94.4%; for Sub S.4, Table 1 reports 4.9% (11/53) while Table 3 reports 20.8%. These inconsistencies are not explained and make it impossible to trust the numerical basis for any downstream claim.
- [Section 5] No evaluation protocol is reported. The experimental section describes feature importance and cluster success rates on the same proprietary dataset used to build the clusters and trees, but it gives no held-out test set, cross-validation, error bars, or comparison against baselines such as a single global decision tree, logistic regression, or the GPTree method cited as [1]. The abstract and Section 3.6 claim accuracy and classification improvements, yet no classification accuracy, AUC, precision/recall, or calibration metric is reported anywhere. The descriptive success rates in Table 4 cannot support the predictive claims.
- [Algorithms 1-3 and Section 4] The evaluation is self-referential: hierarchical clustering is applied to the CTGAN-resampled data, decision trees are fit to the same clusters, and the reported success rates are computed on the same data; the LLM personas are generated from the same z-scores that define the clusters. In addition, Algorithm 1 takes a 'real world success rate' as input, but the manuscript never specifies its value or how it is used by ResampleData. Without an external validation set or an explicit calibration step, the clusters and their success rates are descriptive summaries of one resampled sample, not validated probability estimates. The Limitations section (6.1) further concedes potential LLM hallucination in feature engineering, which compounds this concern.
minor comments (6)
- [Figure 6] The caption for Figure 6 reads 'Resampling Techniques and Their Impact on Class Balance,' but the figure appears to illustrate a decision tree; the caption should match the figure content.
- [Sections 3.7 and 6.1] The model is referred to as GPT-4 in Section 3.7 but GPT-4o in Section 6.1; the manuscript should use one consistent name.
- [Section 3.6] The Gini impurity formula contains a formatting error ('cX i=1' instead of a summation over i=1 to c), which should be corrected.
- [Algorithm 1] Algorithm 1 appears to call SaveClusterResults twice in succession (lines 21-22) with very similar arguments; one of the calls is likely redundant or contains a typo.
- [References] Reference [38] lists the authors as 'L. X. M. S. A. C. and K. V.,' which appears malformed, and reference [1] is incomplete; both should be corrected.
- [Section 5] The proprietary dataset is not available, and no reproducibility statement or code release is mentioned; this limits the reader's ability to verify the reported results.
Circularity Check
The headline 9x/22x success multipliers are outputs of the paper's own resampling transformation and a self-cited baseline, not independent predictions.
-
fitted input called prediction
[Section 3.2 (Resampling Analysis), Section 3.8.1 Algorithm 1, Section 5.1.2 Table 4]
"Algorithm 1, line 3: Dbalanced ← ResampleData(D′, real world success rate). ... After resampling (Table 4), this range expanded notably from 3.9% to 50.0%, creating more distinct separation between high and low-performing groups. ... the post-resampling distribution provides clearer differentiation between performance tiers. For instance, we can now more clearly distinguish between high-performing clusters (50.0%), medium-performing clusters (25.0-28.1%), and lower-performing clusters (3.9%)."
Algorithm 1 places ResampleData before hierarchical clustering and decision trees, so every cluster success rate reported later is computed on the balanced synthetic data. Section 3.2 explicitly credits resampling with expanding the success-rate spread from 4.9-19.75% to 3.9-50.0%. The headline 17.4% (Table 4) is never labeled as pre- or post-resampling and is never calibrated back to the original 1.9% base rate; if it is a post-resampling rate, the claimed 9x multiplier is an artifact of the balancing input, not a measured population probability. If it is pre-resampling, the relationship between the resampled clusters and this table is left unspecified. Either way, the paper presents the output of its own resampling-dependent pipeline as an empirical success probability.
-
self citation load bearing
[Section 1.2 (Empirical Context), Abstract, References [1]]
"the probability of a startup achieving random success—defined as becoming a unicorn or reaching significant exit milestones—is approximately 1.9% [1]. ... uncovering clusters that outperform the random success rate by 9x, a significant jump from the baseline probability of 1.9% to 17.4%. Reference [1]: S. Xiong, Y. Ihlamur, F. Alican, and Y. A., 'Gptree: Towards explainable decision-making via llm-powered decision trees,' arXiv, 2024."
The 1.9% baseline is the denominator of the paper's central 9x claim. It is taken from reference [1], the authors' own prior arXiv paper on GPTree (current authors include Fuat Alican and Yigit Ihlamur). No independent source, external dataset, or derivation in this paper establishes the 1.9% figure, so the headline multiplier's scale rests on a self-citation chain rather than an externally verified constant.
full rationale
The paper's claimed derivation chain—1.9% baseline, CTGAN resampling, hierarchical clustering, decision-tree feature importance, and cluster success rates—does not stand outside its own inputs. The baseline is imported from the authors' own GPTree paper; Algorithm 1 explicitly resamples the data before clustering; and the paper's own Section 3.2 text describes resampling as widening the success-rate spread from 4.9-19.75% to 3.9-50.0%. No holdout set, external benchmark, or calibration step connects the reported cluster success probabilities back to the original 1.9% population scale. The LLM-generated persona descriptions are summaries of the same z-scores used to characterize the clusters, not independent evidence. The internal table inconsistency (Table 3 is said to summarize Table 1 but shows different success rates) reinforces that the reported rates are not reliable independent measurements. I therefore find partial circularity: the headline multipliers reduce, at least in part, to the resampling transformation and to a self-cited baseline.
Assumptions & free parameters
free parameters (6)
- n_main_clusters =
8
- min_subcluster_size =
not reported
- max_depth =
3
- significance_threshold =
not reported
- CTGAN resampling target / real_world_success_rate =
1.9% (input)
- LLM decoding parameters =
temperature 0.7, top_p 0.95, max_tokens 1000, frequency_penalty 0.5
assumptions (5)
- domain assumption CTGAN-generated synthetic rows preserve the true relationship between features and success labels
- domain assumption z-scores and normal-distribution modeling of features are sufficient to characterize cluster-defining traits
- standard math Gini impurity based feature importance correctly identifies determinants of success
- domain assumption GPT-4 outputs are faithful, reproducible summaries of the quantitative cluster statistics
- domain assumption Success labels in the proprietary dataset are accurate and complete
Cite this review
Pith. "Pith review of GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification." pith.science (2026). https://pith.science/paper/F66CRFOS
@misc{pith2026250113743,
author = {Pith},
title = {Pith review of: GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/F66CRFOS}},
note = {Machine review of arXiv:2501.13743}
}
read the original abstract
This paper introduces GPT-HTree, a framework combining hierarchical clustering, decision trees, and large language models (LLMs) to address this challenge. By leveraging hierarchical clustering to segment individuals based on salient features, resampling techniques to balance class distributions, and decision trees to tailor classification paths within each cluster, GPT-HTree ensures both accuracy and interpretability. LLMs enhance the framework by generating human-readable cluster descriptions, bridging quantitative analysis with actionable insights.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Gptree: Towards explainable decision- making via llm-powered decision trees,
S. Xiong, Y. Ihlamur, F. Alican, and Y. A., “Gptree: Towards explainable decision- making via llm-powered decision trees,” arXiv, 2024
work page 2024
-
[2]
Large language models for constructing and optimizing machine learning workflows: A survey,
Y. Gu, H. You, J. Cao, and M. Yu, “Large language models for constructing and optimizing machine learning workflows: A survey,”arXiv preprint arXiv:2411.10478, 2024
arXiv 2024
-
[3]
R. Duda, P. Hart, and D. Stork, Pattern Classification . New York: Wiley, 2 ed., 2001
work page 2001
- [4]
-
[5]
On the handling of continuous-values attributes in decision tree generation,
U. M. Fayyad and K. B. Irani, “On the handling of continuous-values attributes in decision tree generation,” Machine Learning, 1992
work page 1992
-
[6]
L. Breiman, J. Friedman, R. Olshen, and C. Stone, Classification and Regression Trees. New York: Chapman Hall, 1993
work page 1993
-
[7]
Quinlan, Programs for Machine Learning
J. Quinlan, Programs for Machine Learning . San Francisco: Morgan Kaufmann, 1993
work page 1993
-
[8]
Improved use of continuous attributes in c4.5,
J. Quinlan, “Improved use of continuous attributes in c4.5,” Journal of Artificial Intelligence, 1996
work page 1996
Show all 38 references
-
[9]
Multivariate decision trees,
C. Brodley and P. Utgoff, “Multivariate decision trees,” Machine Learning, 1995
1995
-
[10]
A comparative study on feature selection in text catego- rization,
Y. Yang and J. Pedersen, “A comparative study on feature selection in text catego- rization,” in Proceedings of the 14th International Conference on Machine Learning (ICML’97), 1997
1997
-
[11]
Some applications of tree based modeling to speech and language index- ing,
M. Riley, “Some applications of tree based modeling to speech and language index- ing,” 1989. 20
1989
-
[12]
Compact decision trees with cluster validity for speech recognition,
J. Chien, C. Huang, and S. Chen, “Compact decision trees with cluster validity for speech recognition,” in Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002
2002
-
[13]
A decision tree system for finding genes in dna,
S. Salzberg, A. Delcher, K. Fasman, and J. Henderson, “A decision tree system for finding genes in dna,” Journal of Computational Biology , 1998
1998
-
[14]
Web document clustering: A feasibility demonstration,
O. Zamir and O. Etzioni, “Web document clustering: A feasibility demonstration,” in Research and Development in Information Retrieval , 1998
1998
-
[15]
A personalized recommender system based on web usage mining and decision tree induction,
Y. Cho, J. Kim, and S. Kim, “A personalized recommender system based on web usage mining and decision tree induction,” Expert Systems with Applications , 2002
2002
-
[16]
Large language models for automated data science: Introducing caafe for context-aware automated feature engineering,
N. Hollmann, S. M¨ uller, and F. Hutter, “Large language models for automated data science: Introducing caafe for context-aware automated feature engineering,” in Advances in Neural Information Processing Systems , 2024
2024
-
[17]
Grammar prompting for domain-specific language generation with large language models,
B. Wang, Z. Wang, X. Wang, Y. Cao, R. A. Saurous, and Y. Kim, “Grammar prompting for domain-specific language generation with large language models,” in Advances in Neural Information Processing Systems , 2024
2024
-
[18]
Gpt-4 technical report,
J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023
2023 arXiv
-
[19]
Llama: Open and efficient foundation language models,
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi` ere, N. Goyal, E. Hambro, F. Azhar, et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023
2023 arXiv
-
[20]
Bliva: A simple multimodal llm for better handling of text-rich visual questions,
W. Hu, Y. Xu, Y. Li, W. Li, Z. Chen, and Z. Tu, “Bliva: A simple multimodal llm for better handling of text-rich visual questions,” in Proceedings of the AAAI Conference on Artificial Intelligence , 2024
2024
-
[21]
Link-context learning for multimodal llms,
Y. Tai, W. Fan, Z. Zhang, and Z. Liu, “Link-context learning for multimodal llms,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, 2024
2024
-
[22]
Autom3l: An automated multi- modal machine learning framework with large language models,
D. Luo, C. Feng, Y. Nong, and Y. Shen, “Autom3l: An automated multi- modal machine learning framework with large language models,” arXiv preprint arXiv:2408.00665, 2024
2024 arXiv
-
[23]
Plan, generate and match: Scientific workflow recommendation with large language models,
Y. Gu, J. Cao, Y. Guo, S. Qian, and W. Guan, “Plan, generate and match: Scientific workflow recommendation with large language models,” in International Conference on Service-Oriented Computing , Springer, 2023
2023
-
[24]
Conversa- tional process modelling: state of the art, applications, and implications in practice,
N. Klievtsova, J.-V. Benzin, T. Kampik, J. Mangler, and S. Rinderle-Ma, “Conversa- tional process modelling: state of the art, applications, and implications in practice,” in International Conference on Business Process Management , Springer, 2023
2023
-
[25]
Multimodal chain- of-thought reasoning in language models,
Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. Smola, “Multimodal chain- of-thought reasoning in language models,” arXiv preprint arXiv:2302.00923 , 2023. 21
2023 arXiv
-
[26]
Verbalized machine learning: Re- visiting machine learning with language models,
T. Z. Xiao, R. Bamler, B. Sch¨ olkopf, and W. Liu, “Verbalized machine learning: Re- visiting machine learning with language models,” arXiv preprint arXiv:2406.04344 , 2024
2024 arXiv
-
[27]
Data interpreter: An llm agent for data science,
S. Hong, Y. Lin, B. Liu, B. Wu, D. Li, J. Chen, J. Zhang, J. Wang, L. Zhang, M. Zhuge, et al. , “Data interpreter: An llm agent for data science,” arXiv preprint arXiv:2402.18679, 2024
2024 arXiv
-
[28]
Substrat: A subset-based optimization strategy for faster automl,
T. Lazebnik, A. Somech, and A. I. Weinberg, “Substrat: A subset-based optimization strategy for faster automl,” Proceedings of the VLDB Endowment , 2022
2022
-
[29]
Automated evo- lutionary approach for the design of composite machine learning pipelines,
N. O. Nikitin, P. Vychuzhanin, M. Sarafanov, I. S. Polonskaia, I. Revin, I. V. Bara- banova, G. Maximov, A. V. Kalyuzhnaya, and A. Boukhanovsky, “Automated evo- lutionary approach for the design of composite machine learning pipelines,” Future Generation Computer Systems , 2022
2022
-
[30]
Diffusionnag: Predictor-guided neural architecture generation with diffusion models,
S. An, H. Lee, J. Jo, S. Lee, and S. J. Hwang, “Diffusionnag: Predictor-guided neural architecture generation with diffusion models,” in The Twelfth International Conference on Learning Representations, 2023
2023
-
[31]
Github copilot ai pair programmer: Asset or liability?,
A. M. Dakhel, V. Majdinasab, A. Nikanjam, F. Khomh, M. C. Desmarais, and Z. M. J. Jiang, “Github copilot ai pair programmer: Asset or liability?,” 2023
2023
-
[32]
How to support ml end-user programmers through a conversational agent,
E. J. Arteaga Garcia, J. F. N. Pimentel, Z. Feng, M. Gerosa, I. Steinmacher, and A. Sarma, “How to support ml end-user programmers through a conversational agent,” in Proceedings of the 46th IEEE/ACM International Conference on Soft- ware Engineering, 2024
2024
-
[33]
Mlcopilot: Unleashing the power of large language models in solving machine learning tasks,
L. Zhang, Y. Zhang, K. Ren, D. Li, and Y. Yang, “Mlcopilot: Unleashing the power of large language models in solving machine learning tasks,” arXiv preprint arXiv:2304.14979, 2023
2023 arXiv
-
[34]
Optimized feature gen- eration for tabular data via llms with decision tree reasoning,
J. Nam, K. Kim, S. Oh, J. Tack, J. Kim, and J. Shin, “Optimized feature gen- eration for tabular data via llms with decision tree reasoning,” arXiv preprint arXiv:2406.08527, 2024
2024 arXiv
-
[35]
Dynamic and adaptive feature generation with llm,
X. Zhang, J. Zhang, B. Rekabdar, Y. Zhou, P. Wang, and K. Liu, “Dynamic and adaptive feature generation with llm,” arXiv preprint arXiv:2406.03505 , 2024
2024
-
[36]
On the opportunities and risks of foundation models,
R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al., “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258 , 2021
2021 arXiv
-
[37]
A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,
Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang, “A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,”High-Confidence Computing, 2024
2024
-
[38]
Modeling tabular data using conditional gan,
L. X. M. S. A. C. and K. V., “Modeling tabular data using conditional gan,” 2019. 22
2019
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.