REVIEW 4 major objections 6 minor 1 cited by
CTS-Bench: Benchmarking Graph Coarsening Trade-offs for GNNs in Clock Tree Synthesis
T0 review · 4 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read The paper shows that generic graph coarsening, while cutting memory by 17.2x and training time by 3x, destroys the structural information needed to predict clock skew, driving R^2 below zero on unseen designs.
desk verdict Useful new dataset, but the central negative-R2 claim is confounded by unequal model capacity between the raw and clustered arms. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The benchmark's multi-scale representation pair: a Raw graph whose nodes are standard cells with one-hop flip-flop neighborhoods, and a Clustered graph built by a three-step physics-aware algorithm (atomic BFS clusters around flip-flops, high-spread filtering, and gravity-vector-aligned merging that respects control nets and physical proximity), achieving about 13.3x compression. The paired representations plus the normalized Pareto Gap scoring framework are what allow the paper to separate efficiency gains from fidelity loss.
What would settle it
Retrain the clustered GNNs with the same hidden size and learning rate as the raw models and run the same held-out divider design. If R^2 returns to positive values or approaches the raw model's 0.0-0.2 range, the coarsening-harm claim is weakened. Alternatively, replace the benchmark's custom clustering baseline with a standard multilevel graph partitioner at the same compression ratio and compare skew R^2.
Extended reading notes
Core claim
The paper's core discovery is an accuracy-efficiency trade-off with a sharp local-metric cliff. Using clock skew prediction as a representative CTS task, models trained on the benchmark's raw gate-level graphs reach roughly 0.90 R^2 on seen designs, while models trained on clustered graphs—despite preserving global aggregates—lose the fine-grained spatial resolution needed to tell high-skew from low-skew regions, frequently dropping below zero and to -2.23 on an unseen architecture. The authors interpret this as evidence that coarsening removes structural information essential to clock distribution even when global physical metrics such as power and wirelength remain unchanged.
Load-bearing premise
The central claim that coarsening itself destroys skew fidelity rests on a comparison where the clustered models use 16-unit hidden layers and 0.0005 learning rate while the raw models use 64 units and 0.001; if those are equated, the fidelity loss could shrink or disappear.
Editorial extensions
If this is right
- Coarsened graphs remain a viable proxy for global metrics (total power, wirelength) on seen designs, with MAE comparable to raw graphs, making them useful on memory-limited hardware.
- For clock skew, clustered models are not merely slightly worse; their negative R^2 means predictions are worse than predicting the mean, so any CTS-critical use of coarsening must preserve local structural information.
- The 17.2x VRAM reduction and 3x training speedup shift GNN training from memory-bound to compute-bound, suggesting clustered representations can support larger batches or larger designs on the same accelerator.
- Zero-shot generalization is weak even on raw graphs (R^2 0.0-0.2 on the unseen design), so coarsening alone cannot explain all generalization failure; training data diversity matters too.
Reading between the lines
- If the clustered models were given the same hidden dimensions and learning rate as the raw models, part of the fidelity gap might close; the paper's Table 3 confounds coarsening with capacity.
- The benchmark's Pareto Gap scores could be reused as placement-quality labels independent of any GNN, potentially enabling placement optimization before CTS.
- A natural next experiment is task-aware coarsening that explicitly preserves flop-to-flop distances or skew-critical edges; the benchmark's paired graphs make this directly testable.
- Because only one held-out architecture is used for zero-shot evaluation, the headline negative R^2 numbers should be read as evidence of fragility rather than a universal law; a multi-design holdout would sharpen the claim.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CTS-Bench, a dataset of 4,860 post-placement, pre-CTS physical design solutions across five open-source architectures, with paired raw gate-level graphs and clustered proxy graphs plus 15 post-CTS ground-truth metrics. Using GCG/GraphSAGE/GATv2 multi-task regression on skew, power, and wirelength, the authors report that clustering reduces peak VRAM by up to 17.2x and training time by 3x, but that clustered models frequently achieve negative R² for clock-skew prediction, especially on an unseen Zipdiv hold-out. The paper concludes that generic graph clustering can fundamentally compromise CTS learning objectives even when global physical metrics remain unchanged, and positions CTS-Bench as a testbed for task-aware coarsening and GNN acceleration.
Significance. If the empirical claims are established, CTS-Bench would be a useful community resource: it is one of few open ML-EDA benchmarks focused specifically on the placement-CTS interface, it provides paired multi-scale graph representations, and the containerized OpenLane pipeline supports reproducible extensions. The central headline result—that coarsening destroys local CTS fidelity while preserving global metrics—is precisely the kind of trade-off that the community needs quantified. However, the current experimental design does not support that headline: the Raw and Clustered arms differ in model capacity and learning rate, the 'global metrics unchanged' wording is contradicted by the paper's own MAE numbers, and only one custom, paper-specific clustering algorithm is tested. The dataset itself is potentially valuable, but the causal claim about 'generic graph clustering' is not yet demonstrated.
major comments (4)
- [§3, Table 3] The Raw-vs-Clustered comparison is confounded by unequal model capacity. Table 3 sets Raw hidden dim 64 / LR 0.001 / placement-MLP 32 / CTS-MLP 16, while Clustered uses hidden dim 16 / LR 0.0005 / placement-MLP 8 / CTS-MLP 4. The Clustered arm is therefore weaker on every axis, so the observed R² collapse—including the headline GCN -2.23 on Zipdiv—cannot be attributed to graph coarsening without an equal-capacity control. Please retrain both representations with identical hidden dimension, learning rate, and MLP sizes, or run a matched capacity/lr sweep; also report mean/std over at least 5 seeds. This is load-bearing for the abstract's claim that 'generic graph clustering techniques can fundamentally compromise CTS learning objectives.'
- [Abstract and §4.2, Figures 5–6] The statement that global physical metrics 'remain unchanged' is not supported by the presented numbers. For GCN on seen data, Power MAE and Wirelength MAE are both 0.06 for Raw (Fig. 5) and 0.10 for Clustered (Fig. 6), a ~67% relative increase. Similar increases appear for the other backbones. Please either revise the wording to describe a modest degradation, with confidence intervals, or provide a statistical test showing the difference is not significant. Since the paper's conclusion is explicitly framed as 'even when global physical metrics remain unchanged,' this mismatch is central to the trade-off message.
- [§2.1, §2.2, and §4] The term 'generic graph clustering' overstates the evidence. The only coarsening evaluated is the custom three-step BFS/gravity-merge method with paper-specific thresholds (spread >0.05, Manhattan distance <0.05, cosine >0.9). There is no comparison to standard coarseners such as METIS, spectral clustering, or algebraic multigrid, nor to a simple randomized contraction control. Therefore the conclusion that 'generic graph clustering techniques' fail is not established; at most it applies to this particular clustering recipe. Please add at least one standard task-agnostic baseline and one randomized baseline with matched node counts.
- [§2.1 and §3] The dataset accounting is unclear. The text says 'We generate 486 unique placements across all five designs' but also 'an additional 500 data points from a Zipdiv core for zero-shot generalization testing.' If 4,860 = 486 placements × 10 CTS variants, then the Zipdiv points cannot be 'additional' to 486 placements unless the arithmetic is different. Please give a per-architecture table of placements and data points, state explicitly which architectures are in the training split and which form Zipdiv, and confirm that no Zipdiv data is used in training. This is needed for the zero-shot claim to be reproducible.
minor comments (6)
- [§1] The roadmap sentence says 'Section 1 reviews related benchmarking efforts' but the related-work discussion appears in the same section as the introduction; Section 2 is the methodology. Please correct the cross-reference.
- [References] References [19] and [20] are the same paper (EDA-schema, GLSVLSI 2024) and are duplicated. Please merge or remove one.
- [Abstract and §3] The metric is called 'Mean Average Error' in Section 3; this should be 'Mean Absolute Error' (MAE).
- [§2.2] The 'Pareto Gap' / 'Total Pareto Distance' is defined and listed as a contribution but is not used in any experiment or reported result. Either use it in evaluation or remove it to avoid over-claiming.
- [Figure 3 and §2.1] The axis labels switch between 'Raw Nodes' / 'Clustered Nodes' and 'raw gate-level graphs/clustered proxy graphs.' Also, the compression ratio 13.3x is for node count, while the VRAM reduction is 17.2x; please clarify the relationship, since memory reduction may also reflect smaller hidden dimensions in the clustered models.
- [Table 1] The table lists 10 randomization knobs but the text says 'randomize seven placement parameters.' Please reconcile the count, and note which knobs are applied at CTS stage.
Circularity Check
No circular derivation: CTS-Bench reports direct empirical evaluations with no fitted-parameter feedback or self-citation chain.
full rationale
The paper is an empirical benchmark study rather than a derivation, and I found no step in which a claimed prediction or result is equivalent to its inputs by construction. The core claims — that clustered graphs reduce VRAM and training time and degrade clock-skew R^2 — come from direct measurements and evaluations (Figures 4–6, Section 4) against ground-truth CTS metrics; no fitted parameter is renamed as a prediction, and the Pareto Gap metric (Eqs. 1–2) is defined from dataset minima but is not used as a prediction target or fed back into the model evaluations. The clustering algorithm is described procedurally (Section 2.1) and is not derived from the downstream accuracy results. There is no load-bearing self-citation: the cited prior works are standard ML/EDA references and none is invoked to justify the central accuracy-efficiency trade-off. The skeptical concern about unequal model capacity (Table 3: Raw hidden dim 64 / LR 0.001 versus Clustered hidden dim 16 / LR 0.0005) is a legitimate threat to the causal interpretation of the Raw-vs-Clustered comparison, and the claim that 'global physical metrics remain unchanged' is not supported by the reported MAE rise from 0.06 to 0.10. However, these issues concern experimental validity and overstatement, not circularity: they do not make the measured outcomes true by definition or reduce the conclusion to its own inputs. Per the review rules, such confounds belong under correctness risk, not circularity, and I therefore report no significant circularity.
Assumptions & free parameters
free parameters (4)
- Spread threshold for high-spread filtering =
0.05 (standard deviation)
- Gravity merge distance threshold =
Manhattan distance < 0.05
- Gravity vector cosine similarity threshold =
>0.9
- Clustered model hidden dimension / learning rate =
hidden 16, LR 0.0005 (vs Raw 64, 0.001)
assumptions (4)
- domain assumption OpenLane/OpenROAD with Sky130 PDK produces designs representative of industrial CTS behavior.
- domain assumption Flip-flops plus one-hop fanout logic with geometric, type, and switching-activity features are sufficient to predict clock skew.
- domain assumption Random 80/20 split over 4,860 points (10 CTS variants per placement) does not leak placement identity.
- ad hoc to paper The BFS atomic clusters plus gravity merge is representative of 'generic graph clustering'.
Cite this review
Pith. "Pith review of CTS-Bench: Benchmarking Graph Coarsening Trade-offs for GNNs in Clock Tree Synthesis." pith.science (2026). https://pith.science/paper/X6SCUGV5
@misc{pith2026260219330,
author = {Pith},
title = {Pith review of: CTS-Bench: Benchmarking Graph Coarsening Trade-offs for GNNs in Clock Tree Synthesis},
year = {2026},
howpublished = {\url{https://pith.science/paper/X6SCUGV5}},
note = {Machine review of arXiv:2602.19330}
}
abstract
Graph Neural Networks (GNNs) are increasingly explored for physical design analysis in Electronic Design Automation, particularly for modeling Clock Tree Synthesis behavior such as clock skew and buffering complexity. However, practical deployment remains limited due to the prohibitive memory and runtime cost of operating on raw gate-level netlists. Graph coarsening is commonly used to improve scalability, yet its impact on CTS-critical learning objectives is not well characterized. This paper introduces CTS-Bench, a benchmark suite for systematically evaluating the trade-offs between graph coarsening, prediction accuracy, and computational efficiency in GNN-based CTS analysis. CTS-Bench consists of 4,860 converged physical design solutions spanning five architectures and provides paired raw gate-level and clustered graph representations derived from post-placement designs. Using clock skew prediction as a representative CTS task, we demonstrate a clear accuracy-efficiency trade-off. While graph coarsening reduces GPU memory usage by up to 17.2x and accelerates training by up to 3x, it also removes structural information essential for modeling clock distribution, frequently resulting in negative $R^2$ scores under zero-shot evaluation. Our findings indicate that generic graph clustering techniques can fundamentally compromise CTS learning objectives, even when global physical metrics remain unchanged. CTS-Bench enables principled evaluation of CTS-aware graph coarsening strategies, supports benchmarking of GNN architectures and accelerators under realistic physical design constraints, and provides a foundation for developing learning-assisted CTS analysis and optimization techniques.
Figures
Forward citations
Cited by 1 Pith paper
-
SwiftCTS: Fast Cross-Design Prediction and Pareto Optimization of Clock Tree Metrics via Few-Shot Calibration
SwiftCTS combines physics-informed gradient-boosted models with K-shot multiplicative calibration to enable fast, low-error prediction and Pareto optimization of clock tree metrics on unseen macro architectures withou...
Reference graph
Works this paper leans on
-
[1]
Cheng, Vidya A
Tutu Ajayi, David Blaauw, Tuck-Boon Chan, C.-K. Cheng, Vidya A. Chhabria, D. K. Choo, M. Coltella, Ronald G. Dreslinski, Mateus Fogac¸a, S. Mehdi Hashemi, A. A. Ibrahim, Andrew B. Kahng, M. Kim, J. Li, Z. Liang, Uday Mallappa, Paul I. Pénzes, Geraldo Pradipta, Sherief Reda, Austin Rovinski, Kambiz Samadi, Sachin S. Sapatnekar, Lawrence K. Saul, Carl Seche...
2019
-
[2]
Charles J. Alpert. 1998. The ISPD98 circuit benchmark suite. InProceedings of the 1998 International Symposium on Physical Design(Monterey, California, USA) (ISPD ’98). Association for Computing Machinery, New York, NY, USA, 80–85. doi:10.1145/274535.274546
arXiv 1998
-
[3]
Shaked Brody, Uri Alon, and Eran Yahav. 2022. How Attentive are Graph At- tention Networks? arXiv:2105.14491 [cs.LG] https://arxiv.org/abs/2105. 14491
arXiv 2022
-
[4]
Ruoyu Cheng and Junchi Yan. 2021. On joint learning for solving placement and routing in chip design.Advances in Neural Information Processing Systems34 (2021), 16508–16519
2021
-
[5]
Truong Xuan Dinh and Tien-Dung Do. 2024. Graph-Based Ranking Techniques for Improving VLSI Placement. In2024 IEEE 15th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON). IEEE, 715–719
2024
-
[6]
Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson
Vijay Prakash Dwivedi, Chaitanya K. Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. 2023. Benchmarking Graph Neural Networks. Journal of Machine Learning Research24, 43 (2023), 1–48. http://jmlr.org/ papers/v24/22-0567.html
2023
-
[7]
Vijay Prakash Dwivedi, Ladislav Rampášek, Mikhail Galkin, Ali Parviz, Guy Wolf, Anh Tuan Luu, and Dominique Beaini. 2023. Long Range Graph Benchmark. arXiv:2206.08164 [cs.LG]https://arxiv.org/abs/2206.08164
arXiv 2023
-
[8]
Hamilton, Rex Ying, and Jure Leskovec
William L. Hamilton, Rex Ying, and Jure Leskovec. 2018. Inductive Representation Learning on Large Graphs. arXiv:1706.02216 [cs.SI] https://arxiv.org/abs/ 1706.02216
arXiv 2018
Show all 25 references
-
[9]
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. 2021. Open Graph Benchmark: Datasets for Machine Learning on Graphs. arXiv:2005.00687 [cs.LG] https://arxiv. org/abs/2005.00687
2021 arXiv
-
[10]
Yuting Hu, Jiajie Li, Florian Klemme, Gi-Joon Nam, Tengfei Ma, Hussam Am- rouch, and Jinjun Xiong. 2023. SyncTREE: Fast Timing Analysis for Integrated Circuit Design through a Physics-Informed Tree-Based Graph Neural Network. InAdvances in Neural Information Processing Systems...
2023
-
[11]
Guyue Huang, Jingbo Hu, Yifan He, Jialong Liu, et al. 2021. Machine Learning for Electronic Design Automation: A Survey.ACM Trans. Des. Autom. Electron. Syst.26, 5 (2021). doi:10.1145/3451179
2021 doi
-
[12]
Xun Jiang, zhuomin chai, Yuxiang Zhao, Yibo Lin, Runsheng Wang, and Ru Huang
-
[13]
Myung-Chul Kim, Jin Hu, Jiajia Li, and Natarajan Viswanathan. 2015. ICCAD- 2015 CAD contest in incremental timing-driven placement and benchmark suite. In2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). 921–926. doi:10.1109/ICCAD.2015.7372671
2015
-
[14]
Kipf and Max Welling
Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. arXiv:1609.02907 [cs.LG] https://arxiv.org/ abs/1609.02907
2017 arXiv
-
[15]
Yi-Chen Lu, Sai Pentapati, and Sung Kyu Lim. 2020. VLSI placement optimization using graph neural networks. InProceedings of the 34th Advances in Neural Information Processing Systems (NeurIPS) Workshop on ML for Systems, Virtual. 6–12
2020
-
[16]
Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Erik Johnson, Omkar Pathak, Sungjun Bae, et al
-
[17]
Alpert, Paul Villarrubia, Bruce Winter, and Mehmet Yildiz
Gi-Joon Nam, Charles J. Alpert, Paul Villarrubia, Bruce Winter, and Mehmet Yildiz. 2005. The ISPD2005 placement contest and benchmark suite. InProceedings of the 2005 International Symposium on Physical Design(San Francisco, California, 6 USA)(ISPD ’05). Association for Comput...
2005
-
[18]
Cristhian Roman-Vicharra, Yiran Chen, and Jiang Hu. 2024. Flip-Flop Centric Incremental Placement for Simultaneous Timing and Clock Network Power Optimization. InProceedings of the 2024 ACM/IEEE International Symposium on Machine Learning for CAD(Salt Lake City, UT, USA)(MLCAD...
2024
-
[19]
Pratik Shrestha, Alec Aversa, Saran Phatharodom, and Ioannis Savidis. 2024. EDA-schema: A graph datamodel schema and open dataset for digital design automation. InProceedings of the Great Lakes Symposium on VLSI 2024. 69–77
2024
-
[20]
Pratik Shrestha, Alec Aversa, Saran Phatharodom, and Ioannis Savidis. 2024. EDA-schema: A Graph Datamodel Schema and Open Dataset for Digital Design Automation. InProceedings of the Great Lakes Symposium on VLSI 2024(Clearwa- ter, FL, USA)(GLSVLSI ’24). Association for Computi...
2024
-
[21]
Natarajan Viswanathan, Charles J Alpert, Cliff Sze, Zhuo Li, Gi-Joon Nam, and Jarrod A Roy. 2011. The ISPD-2011 routability-driven placement contest and benchmark suite. InProceedings of the 2011 international symposium on Physical design. 141–146
2011
-
[22]
Jiang Wu, Chunhe Ni, Hongbo Wang, and Jingyi Chen. 2025. Graph neural networks for efficient clock tree synthesis optimization in complex SoC designs. Applied and Computational Engineering150 (2025), 101–111
2025
-
[23]
Zhiyao Zhou, Sheng Zhou, Bochao Mao, Xuanyi Zhou, Jiawei Chen, Qiaoyu Tan, Daochen Zha, Yan Feng, Chun Chen, and Can Wang. 2023. OpenGSL: A Comprehensive Benchmark for Graph Structure Learning. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globers...
2023
-
[2021]
doi:10.1038/s41586-021-03544-w
A Graph Placement Methodology for Fast Chip Design.Nature594 (2021), 207–212. doi:10.1038/s41586-021-03544-w
2021 doi
-
[2024]
InThe Twelfth International Conference on Learning Representations
CircuitNet 2.0: An Advanced Dataset for Promoting Machine Learning Innovations in Realistic Chip Design Environment. InThe Twelfth International Conference on Learning Representations. https://openreview.net/forum?id= nMFSUjxMIl
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.