The reviewed record of science sign in
Pith

arxiv: 2606.04126 · v1 · pith:DI2SL46G · submitted 2026-06-02 · cs.AR · cs.AI· cs.SE

HighTide: An Agent-Curated Open-Source VLSI Benchmark Suite

Benjamin Goldblatt , Paolo Pedroso , Farhad Modaresi , Ethan Sifferman , Matthew R. Guthaus This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-28 07:47 UTCgrok-4.3pith:DI2SL46Grecord.jsonopen to challenge →

classification cs.AR cs.AIcs.SE
keywords VLSI benchmark suiteopen-source hardwareAI-assisted curationRTL-to-GDS flowagent skillsBazel compilationdecision logstechnology nodes
0
0 comments X

The pith

HighTide supplies an open-source VLSI benchmark suite maintained by twelve AI agent skills and per-design decision logs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents HighTide as an evolving benchmark suite for VLSI that collects diverse open-source designs across multiple languages and technology nodes. It adds Bazel-based incremental RTL-to-GDS flows with remote caching and verification steps for stable releases. Twelve agent skills manage the design lifecycle, flow optimization, tool reference, and meta-maintenance. Per-design decision logs record tuning choices to serve as long-term memory. The suite is built to expand with the open-source hardware community.

Core claim

HighTide is an evolving AI-assisted benchmark suite that assembles open-source VLSI designs spanning multiple languages and nodes, applies Bazel-based incremental RTL-to-GDS compilation with remote caching, and uses twelve agent skills plus per-design decision logs for curation across the design lifecycle, flow optimization, tool reference, and meta-maintenance, together with verification infrastructure for stable releases.

What carries the argument

Twelve agent skills covering design lifecycle, flow optimization, tool reference, and meta-maintenance, backed by per-design decision logs that record tuning rationale.

If this is right

  • Bazel-based flows deliver incremental RTL-to-GDS compilation and remote caching for repeated runs.
  • Verification infrastructure produces stable releases of the benchmark suite.
  • Decision logs preserve rationale for design and flow choices across the collection.
  • The suite structure supports addition of new designs as the open-source hardware ecosystem develops.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Decision logs could be mined later to extract patterns for automated flow tuning.
  • The agent-skill approach might transfer to benchmark maintenance in adjacent fields such as software or embedded systems.
  • Public availability of the logs could let external groups audit or extend the curation process.

Load-bearing premise

The twelve agent skills and per-design decision logs will sustain effective curation and maintenance across the suite with little ongoing manual effort.

What would settle it

Track whether the suite requires repeated manual interventions to stay current after the initial release when no new agent development occurs.

Figures

Figures reproduced from arXiv: 2606.04126 by Benjamin Goldblatt, Ethan Sifferman, Farhad Modaresi, Matthew R. Guthaus, Paolo Pedroso.

Figure 1
Figure 1. Figure 1: HighTide workflow. The flow takes one of two in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Placement view for three HighTide designs (ASAP7): a small CNN accelerator with dense macros, the most macro-dense [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Instance area share (ASAP7) for a selection of High [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
read the original abstract

We introduce HighTide, an evolving AI-assisted benchmark suite. Specifically, the contributions are: (i) a diverse open-source suite spanning multiple design languages and technology nodes, (ii) Bazel-based incremental RTL-to-GDS compilation with remote caching, (iii) AI-assisted design curation through twelve agent skills covering the design lifecycle, flow optimization, tool reference, and meta-maintenance, backed by per-design decision logs that serve as long-term memory of tuning rationale across the suite, and (iv) an infrastructure with RTL compilation verification for stable releases. The suite is publicly available and designed to grow with the open-source hardware ecosystem.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces HighTide, an evolving AI-assisted open-source VLSI benchmark suite. Its contributions are (i) a diverse collection of designs spanning multiple design languages and technology nodes, (ii) Bazel-based incremental RTL-to-GDS compilation with remote caching, (iii) AI-assisted curation via twelve agent skills covering the design lifecycle, flow optimization, tool reference, and meta-maintenance, supported by per-design decision logs as long-term memory, and (iv) an infrastructure providing RTL compilation verification for stable releases. The suite is publicly available and intended to grow with the open-source hardware ecosystem.

Significance. If the agent-based curation and supporting infrastructure deliver on the claims, HighTide could become a useful, community-maintainable resource for VLSI and open-source hardware research, lowering barriers to reproducible experiments across languages and nodes. The explicit provision of decision logs and public release are strengths that aid reproducibility and extensibility.

major comments (2)
  1. [Abstract] Abstract, contribution (iii): The central claim that the twelve agent skills plus per-design decision logs enable effective long-term curation with reduced manual intervention is asserted without any quantitative evidence (e.g., intervention counts, error rates, or A/B comparisons against manual maintenance) as the suite evolves. This is load-bearing for the title and stated purpose.
  2. [Abstract] Abstract, contribution (ii) and (iv): No results are reported on the practical benefits of the Bazel-based incremental flow (e.g., build-time reductions or cache-hit rates) or on the verification infrastructure (e.g., release stability metrics or failure rates), leaving the claimed advantages of the compilation and release system unverified.
minor comments (1)
  1. The abstract would be clearer if it stated the current number of designs, languages, and nodes covered rather than describing them only qualitatively.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed review and constructive feedback. We agree that quantitative evidence is important to substantiate the claims in the abstract and will strengthen the manuscript accordingly. Below we respond point-by-point to the major comments.

read point-by-point responses

  1. Referee: [Abstract] Abstract, contribution (iii): The central claim that the twelve agent skills plus per-design decision logs enable effective long-term curation with reduced manual intervention is asserted without any quantitative evidence (e.g., intervention counts, error rates, or A/B comparisons against manual maintenance) as the suite evolves. This is load-bearing for the title and stated purpose.

    Authors: We acknowledge that the current manuscript presents the agent skills and decision logs as enabling reduced manual intervention but does not include quantitative metrics such as intervention counts, error rates, or direct comparisons. This is a valid observation. In the revised version we will add data drawn from the curation process, including counts of agent-initiated vs. manual changes across releases, observed error rates in skill execution, and qualitative discussion of how the decision logs have reduced repeated manual tuning. Comprehensive A/B testing against purely manual maintenance is not yet feasible given the evolving nature of the suite, but we will report the available longitudinal data from our internal logs. revision: yes

  2. Referee: [Abstract] Abstract, contribution (ii) and (iv): No results are reported on the practical benefits of the Bazel-based incremental flow (e.g., build-time reductions or cache-hit rates) or on the verification infrastructure (e.g., release stability metrics or failure rates), leaving the claimed advantages of the compilation and release system unverified.

    Authors: We agree that the manuscript would be strengthened by reporting concrete measurements of the Bazel-based flow and verification infrastructure. In the revision we will include build-time comparisons (with and without incremental caching), observed cache-hit rates from our CI runs, and release stability metrics such as the fraction of designs that pass RTL compilation verification on each stable release. These data exist in our internal logs and will be added to the paper. revision: yes

Circularity Check

0 steps flagged

No circularity; paper is purely descriptive with no derivations or predictions

full rationale

The manuscript is a tool/dataset description paper introducing HighTide as an open-source VLSI benchmark suite. It lists four contributions (diverse designs, Bazel infrastructure, twelve agent skills for curation, and verification) but contains no equations, fitted parameters, predictions, or derivation chains. No self-citation load-bearing steps, self-definitional constructs, or renamings of known results appear. The curation method is presented as a stated contribution rather than derived from prior fitted inputs or self-citations that reduce to the paper's own claims. The paper is therefore self-contained against external benchmarks with no circular reduction possible.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The contribution is a software artifact and curation process rather than a mathematical or empirical derivation; no free parameters, axioms, or invented entities are invoked.

pith-pipeline@v0.9.1-grok · 5647 in / 1159 out tokens · 21962 ms · 2026-06-28T07:47:27.363904+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 4 canonical work pages

  1. [1]

    ICDAD Contests

    2026. ICDAD Contests. https://www.iccad-contest.org/History.html. Accessed: 2026-03-06

    2026

  2. [2]

    ISPD Contests

    2026. ISPD Contests. https://ispd.cc/ispd2026/index.php?page=contests. Ac- cessed: 2026-03-06

    2026

  3. [3]

    MLCAD Contests

    2026. MLCAD Contests. https://mlcad.org/symposium. Accessed: 2026-03-06

    2026

  4. [4]

    ABK Group. 2022. FakeRAM 2.0: Updated SRAM compiler with improved area and timing models. https://github.com/ABKGroup/FakeRAM2.0

    2022

  5. [5]

    Albrecht et al

    C. Albrecht et al . 2005. IWLS 2005 Benchmarks. http://iwls.org/iwls2005/ benchmarks.html. Accessed: 2026-03-09

    2005

  6. [6]

    A. Amid, D. Biancolin, A. Gonzalez, D. Gruber, S. Karandikar, H. Liew, A. Magyar, H. Mao, A. Ou, N. Pemberton, P. Rigge, C. Schmidt, J. Wright, J. Zhao, Y. S. Shao, K. Asanović, and B. Nikolić. 2020. Chipyard: Integrated Design, Simulation, and Implementation Framework for Custom SoCs.IEEE Micro40, 4 (2020), 10–21

    2020

  7. [7]

    Anonymous. 2025. Ternip: Ternary-Precision Matrix-Multiplication Inference Accelerator. Repository URL omitted for anonymous review

    2025

  8. [8]

    Anthropic. 2025. Claude Code: An Agentic Coding Tool. https://docs.anthropic. com/en/docs/agents-and-tools/claude-code/overview

    2025

  9. [9]

    Apple Inc. 2021. Apple at Work: M1 Overview. https://www.apple.com/ua/ business/mac/pdf/Apple-at-Work-M1-Overview.pdf

    2021

  10. [10]

    Asanović et al

    K. Asanović et al. 2016.The Rocket Chip Generator. Technical Report UCB/EECS- 2016-17. EECS Department, University of California, Berkeley. https://github. com/chipsalliance/rocket-chip

    2016

  11. [11]

    Balkind, M

    J. Balkind, M. McKeown, Y. Fu, T. Nguyen, Y. Zhou, A. Lavrov, M. Shahrad, A. Fuchs, S. Payne, X. Liang, M. Matl, and D. Wentzlaff. 2016. OpenPiton: An Open Source Manycore Research Framework.SIGARCH CAN44, 2 (2016), 217–232

    2016

  12. [12]

    Bespoke Silicon Group. 2019. bsg_fakeram: Black-box SRAM macro generator. https://github.com/bespoke-silicon-group/bsg_fakeram

    2019

  13. [13]

    Blanchard

    A. Blanchard. 2019. Microwatt. https://github.com/antonblanchard/microwatt

    2019

  14. [14]

    J. Bush. 2014. NyuziProcessor: GPGPU Microprocessor Architecture. https: //github.com/jbush001/NyuziProcessor

    2014

  15. [15]

    Z. Chai, Y. Zhao, W. Liu, Y. Lin, X. Wang, and Y. Wei. 2023. CircuitNet: An Open-Source Dataset for Machine Learning in VLSI CAD Applications with Improved Domain-Specific Evaluation Metric and Learning Strategies.TCAD (2023)

    2023

  16. [16]

    B.-C. Chen. 2024. CNN Accelerator Based on Eyeriss v2. https://github.com/ BoooC/CNN-Accelerator-Based-on-Eyeriss-v2

    2024

  17. [17]

    V. A. Chhabria, W. Jiang, A. B. Kahng, R. Liang, H. Ren, and S. S. Sapatnekar

  18. [18]

    Toward an ML EDA Commons: Establishing Standards, Accessibility, and Reproducibility in ML-driven EDA Research. InISPD

  19. [19]

    L. T. Clark, V. Vashishtha, L. Shifren, A. Gujja, S. Sez, C. Torber, K. Sankhala, R. Somani, G. Tagore, S. Ahmed, and D. K. R. Swamy. 2016. ASAP7: A 7-nm FinFET predictive process design kit.Microelectronics Journal53 (2016), 105–115

    2016

  20. [20]

    EnjoyDigital. 2012. LiteEth. https://github.com/enjoy-digital/liteeth

    2012

  21. [21]

    EnjoyDigital. 2014. LiteDRAM. https://github.com/enjoy-digital/litedram

    2014

  22. [22]

    EnjoyDigital. 2015. LitePCIe. https://github.com/enjoy-digital/litepcie

    2015

  23. [23]

    Fischer, M

    T. Fischer, M. Paulin, W. Roenninger, F. Zaruba, F. Schuiki, and L. Benini. 2025. FlooNoC: A 645-Gb/s/link 0.15-pJ/B/hop Open-Source NoC With Wide Physical Links and End-to-End AXI4 Parallel Multistream Support.IEEE TVLSI33, 4 (2025), 1094–1107

    2025

  24. [24]

    H. Genç, S. Kim, A. Amid, A. Haj-Ali, V. Iyer, P. Prakash, J. Zhao, D. Gruber, H. Liew, H. Mao, A. Ou, C. Schmidt, S. Steffl, J. Wright, I. Stoica, J. Ragan-Kelley, K. Asanović, B. Nikolić, and Y. S. Shao. 2021. Gemmini: Enabling Systematic Deep- Learning Architecture Evaluation via Full-Stack Integration. InDAC. 769–774

    2021

  25. [25]

    Gopalakrishnan, P

    V. Gopalakrishnan, P. Sankaranarayanan, S. P. Sandeep, S. Garg, M. Zeng, C. Alpert, Z. Alon, and R. Reis. 2025. Invited Paper: MLCAD 2025 Contest on ReSynthAI: Physical-aware Logic Resynthesis using AI. InMLCAD. 1–7. doi:10. 1109/MLCAD65511.2025.11189135

    work page arXiv 2025

  26. [26]

    Herveille

    R. Herveille. 2002. JPEG Encoder. https://opencores.org/projects/video_systems

    2002

  27. [27]

    iEDA Project. 2024. iEDA: Open-Source Intelligent Physical Design Toolkit. https://github.com/OSCC-Project/iEDA

    2024

  28. [28]

    Jiang, Z

    X. Jiang, Z. Chai, Y. Zhao, Y. Lin, X. Wang, and Y. Wei. 2024. CircuitNet 2.0: An Advanced Dataset for Promoting Machine Learning Innovations in Realistic Chip Design Environment. InICLR

    2024

  29. [29]

    Liang, W

    R. Liang, W. Jiang, H. Ren, V. A. Chhabria, and A. B. Kahng. 2023. Invited Paper: CircuitOps: An ML Infrastructure Enabling Generative AI for VLSI Circuit Optimization. InICCAD. 1–6

    2023

  30. [30]

    Liang, H

    R. Liang, H. Ren, V. A. Chhabria, B. Khailany, and A. B. Kahng. 2024. GPU/ML- Enhanced Large Scale Global Routing Contest. InISPD. 269–274

    2024

  31. [31]

    M-Labs. 2011. Migen: A Python Toolbox for Building Complex Digital Hardware. https://git.m-labs.hk/M-Labs/migen

    2011

  32. [32]

    D. Meza. 2024. Coral TPU NPU. https://github.com/danielmeza/Coral-TPU-NPU

    2024

  33. [33]

    NVIDIA. 2018. NVDLA: Open Source Hardware. https://github.com/nvdla/hw

    2018

  34. [34]

    OpenCores. 1999. OpenCores: Open Source IP Cores. https://opencores.org/

    1999

  35. [35]

    OpenROAD Project. 2019. OpenROAD-flow-scripts. https://github.com/The- OpenROAD-Project/OpenROAD-flow-scripts

    2019

  36. [36]

    Pan, C.-C

    J. Pan, C.-C. Chang, Z. Xie, Y. Chen, and H. Li. 2025. EDALearn: A Comprehensive RTL-to-Signoff EDA Benchmark for Democratized and Reproducible ML for EDA Research. https://arxiv.org/abs/2312.01674

    work page arXiv 2025

  37. [37]

    Petrisko, F

    D. Petrisko, F. Gao, S. Churiwala, R. Chien, J. Guo, E. Atingogbe, V. Ajayi, W. Ard, H. Cao, A. Casorso, M. Horeni, P. Jamieson, S. Li, M. Singhal, L. VandenBerghe, V. Vuppalapati, S. Singh, B. Zimmer, and M. Taylor. 2020. BlackParrot: An Agile Open-Source RISC-V Multicore for Accelerator SoCs.IEEE Micro40, 4 (2020), 93–102

    2020

  38. [38]

    Dreaming

    ROIC.AI. 2026. Anthropic Unveils “Dreaming” Feature to Help AI Agents Self-Improve. https://www.roic.ai/news/anthropic-unveils-dreaming-feature- to-help-ai-agents-self-improve-05-06-2026. Accessed: 2026-05-08

    2026

  39. [39]

    Schmidt and A

    C. Schmidt and A. Izraelevitz. 2013. A Fast Parameterized SHA3 Accelerator. https://github.com/ucb-bar/sha3

    2013

  40. [40]

    SkyWater Technology and Google. 2020. SkyWater SKY130 Open Source PDK. https://github.com/google/skywater-pdk

    2020

  41. [41]

    G. Smecher. 2022. Minimax. https://github.com/gsmecher/minimax

    2022

  42. [42]

    J. E. Stine, I. Castellanos, M. Wood, J. Henson, F. Love, W. R. Davis, P. D. Franzon, M. Bucher, S. Basavarajaiah, J. Oh, and R. Jenkal. 2007. FreePDK: An Open- Source Variation-Aware Design Kit. InInternational Conference on Microelectronic Systems Education (MSE). 173–174

    2007

  43. [43]

    Takamaeda-Yamazaki

    S. Takamaeda-Yamazaki. 2015. Pyverilog: A Python-Based Hardware Design Processing Toolkit for Verilog HDL. InARC. 451–460

    2015

  44. [44]

    Takamaeda-Yamazaki et al

    S. Takamaeda-Yamazaki et al. 2019. NNgen. https://github.com/NNgen/nngen

    2019

  45. [45]

    The OpenROAD Project. 2024. bazel-orfs. https://github.com/The-OpenROAD- Project/bazel-orfs

    2024

  46. [46]

    B. Tine, K. P. Emaminejad, B. Benton, and H. Kim. 2021. Vortex: Extending the RISC-V ISA for GPGPU and 3D-Graphics. InMICRO. 754–766. doi:10.1145/ 3466752.3480128

    work page arXiv 2021

  47. [47]

    Usselmann

    R. Usselmann. 2002. AES Core. https://opencores.org/projects/aes_core

    2002

  48. [48]

    Zaruba and L

    F. Zaruba and L. Benini. 2019. The Cost of Application-Class Processing: Energy and Performance Analysis of a Linux-Ready 1.7-GHz 64-Bit RISC-V Core in 22-nm FDSOI Technology.IEEE Transactions on Very Large Scale Integration (VLSI) Systems27, 11 (2019), 2629–2640. doi:10.1109/TVLSI.2019.2926114

    work page doi:10.1109/tvlsi.2019.2926114 2019

  49. [49]

    Zaruba, F

    F. Zaruba, F. Schuiki, T. Hoefler, and L. Benini. 2021. Snitch: A Tiny Pseudo Dual-Issue Processor for Area and Energy Efficient Execution of Floating-Point Intensive Workloads.IEEE Trans. Comput.70, 11 (2021). Benjamin Goldblatt, Paolo Pedroso, Farhad Modaresi, Ethan Sifferman, and Matthew R. Guthaus Appendix A Design Overview Design Language Cell Count ...

    2021