{"id":"f5d6c052-5349-43cb-9eb8-007764775bc2","arxiv_id":"2503.12946","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Open3DBench is an open-source 3D-IC benchmark with Open3D-Tiling and Open3D-DMP placement methods that report 51% area, 24% wirelength, 31% timing, and 6% power gains versus 2D flows.","lead":"The paper introduces Open3DBench, an open-source benchmark for 3D-IC backend design built on OpenROAD, with two new placement algorithms and reported PPA gains over 2D. A smart generalist might read it to see how open tools could help develop stacked chips without expensive commercial software.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"PPA gains rest on whether the open-source flow fully implements all modules (partitioning/placement/routing/RC/thermal) without commercial-tool fallbacks","rationale":"The reader's weakest assumption is exactly the load-bearing point for the experimental claim. Because the review was abstract-only, the full text must still be checked for concrete evidence that every listed module runs open-source; the concrete_test above directly tests that condition.","tokens_in":1738,"tokens_out":314,"duration_ms":22943,"concrete_test":"Clone the Open3DBench repository, execute the end-to-end flow on one of the reported benchmarks using only OpenROAD and the supplied open-source scripts (no commercial EDA licenses); if RC extraction or thermal simulation cannot complete or if the PPA numbers deviate >10 % from the paper tables, the claimed gains are not independently verifiable.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline experimental improvements (area 51.19 %, wirelength 24.06 %, timing 30.84 %, power 5.72 %) are obtained from Open3D-Tiling and Open3D-DMP inside the claimed modular flow. That flow is asserted to support 3D partitioning, placement, 3D routing, RC extraction and thermal simulation while aligning with commercial-tool flows. If any of those steps still depend on non-open components or if the in-house scripts are incomplete, the reported deltas versus 2D baselines cannot be reproduced or attributed to the open-source methods alone.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces Open3DBench, an open-source 3D-IC backend benchmark built on OpenROAD-flow-scripts. It proposes two placement algorithms (Open3D-Tiling for regular macro placement and Open3D-DMP for cross-die wirelength optimization via DREAMPlace) and a modular flow supporting 3D partitioning, placement, 3D routing, RC extraction, and thermal simulation. Experimental results claim PPA gains versus 2D flows (area 51.19%, wirelength 24.06%, timing 30.84%, power 5.72%) and note that wirelength improvement does not guarantee PPA gains, positioning the benchmark as a reproducible platform bridging open-source and commercial 3D-IC flows.","tokens_in":1856,"tokens_out":533,"duration_ms":28260,"significance":"If the open-source components fully implement the claimed modules without undisclosed commercial dependencies and the results prove reproducible, the work would deliver a valuable standardized benchmark and two baseline algorithms for 3D-IC research. The explicit release of the benchmark and algorithms constitutes a concrete strength that could enable community-driven PPA-driven method development.","major_comments":[{"comment":"§3 (Proposed Flow description): the assertion that the flow 'supports modular integration of 3D partitioning, placement, 3D routing, RC extraction, and thermal simulation' while aligning with commercial-tool flows does not map each module to its open-source implementation or identify any remaining commercial-tool fallbacks; this directly affects whether the reported PPA deltas can be attributed to the open-source methods alone.","section":"§3"},{"comment":"§5 (Experimental Results): the headline improvements (area 51.19%, wirelength 24.06%, timing 30.84%, power 5.72%) are reported without stating the number of test cases, statistical significance tests, run-to-run variance, or data-selection criteria, leaving the robustness of the cross-flow comparison unclear.","section":"§5"}],"minor_comments":[{"comment":"Figure captions and axis labels should explicitly distinguish 2D baseline from the two 3D variants (Open3D-Tiling, Open3D-DMP) for immediate readability.","section":null},{"comment":"Ensure first-use definitions for all acronyms (PPA, DMP, RC) and consistent notation for the two proposed algorithms across text and tables.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments, which highlight opportunities to improve transparency and robustness in the manuscript. We address each major comment below and will incorporate revisions to strengthen the paper.","responses":[{"response":"We agree that the current §3 description lacks an explicit mapping. The revised manuscript will include a new table (or expanded subsection) that maps each module (3D partitioning, placement, 3D routing, RC extraction, thermal simulation) to its concrete open-source implementation within OpenROAD-flow-scripts, along with any identified commercial-tool fallbacks or dependencies. This will allow readers to directly assess attribution of the reported PPA deltas.","revision_made":"yes","referee_comment":"[§3] §3 (Proposed Flow description): the assertion that the flow 'supports modular integration of 3D partitioning, placement, 3D routing, RC extraction, and thermal simulation' while aligning with commercial-tool flows does not map each module to its open-source implementation or identify any remaining commercial-tool fallbacks; this directly affects whether the reported PPA deltas can be attributed to the open-source methods alone."},{"response":"We acknowledge the need for greater experimental detail. The revised §5 will explicitly state the number of test cases, the data-selection criteria used, any statistical significance tests applied, and run-to-run variance (where relevant) to substantiate the cross-flow PPA comparisons and improve reproducibility.","revision_made":"yes","referee_comment":"[§5] §5 (Experimental Results): the headline improvements (area 51.19%, wirelength 24.06%, timing 30.84%, power 5.72%) are reported without stating the number of test cases, statistical significance tests, run-to-run variance, or data-selection criteria, leaving the robustness of the cross-flow comparison unclear."}],"tokens_in":1445,"tokens_out":400,"duration_ms":23015,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"Open3DBench and the two placement algorithms are the main new pieces. The paper builds a benchmark on OpenROAD that covers partitioning, placement, routing, RC extraction, and thermal simulation in a modular way, then adds Open3D-Tiling for regular macro placement and Open3D-DMP that co-places across dies using DREAMPlace. This is concrete new content for a subfield that has lacked shared open testbeds.","headline":"Open3DBench supplies an open 3D-IC flow on OpenROAD plus two placement routines, which could help standardize experiments if the implementation stays fully open.","tokens_in":2330,"tokens_out":166,"would_cite":false,"duration_ms":32516,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"Open-source 3D-IC EDA benchmark unrelated to RS forcing chain or J-cost structures","alignment":"orthogonal","rationale":"Paper describes Open3DBench, a modular OpenROAD-based flow for 3D partitioning/placement/routing/RC/thermal in face-to-face hybrid-bonded ICs, with two placement heuristics (Open3D-Tiling, Open3D-DMP) and PPA metrics. Central machinery is VLSI EDA tooling and empirical wirelength/PPA deltas; no reference to recognition cost J(x), phi-ladder, 8-tick periodicity, Alexander duality for D=3, or any RS forcing theorem. Domain (cs.AR) lies outside RS scope.","tokens_in":52942,"confidence":"high","tokens_out":157,"duration_ms":7619,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Open3DBench supplies an open-source flow for 3D-IC backend design that reports over 50 percent area reduction versus 2D methods.","keywords":["3D-IC","benchmark","backend implementation","PPA evaluation","placement algorithms","OpenROAD","wirelength","thermal simulation"],"falsifier":"Running the same benchmark designs through a commercial 3D-IC tool chain and finding no area, timing, or power gains would falsify the reported improvements.","tokens_in":2651,"feed_emoji":"📊","tokens_out":689,"duration_ms":24111,"temperature":0.7,"pith_summary":"The paper introduces Open3DBench, a benchmark platform built on open flow scripts to evaluate 3D integrated circuit backend steps including partitioning, placement, routing, extraction, and thermal analysis. It supplies two placement approaches that together produce measured gains in area, wirelength, timing, and power over standard 2D designs. Experiments on the platform also show that shorter wirelength does not reliably deliver better overall power-performance-area results. The work positions the benchmark as a reproducible way to test 3D methods without relying solely on closed commercial tools.","feed_headline":"Benchmark shows 51 percent area cut for 3D chip designs","feed_subtitle":"Open platform tests 3D flows and finds wirelength gains alone miss some power and timing benefits.","key_machinery":"Modular 3D-IC flow supporting partitioning, placement, 3D routing, RC extraction, and thermal simulation together with the Open3D-Tiling and Open3D-DMP placement algorithms.","core_discovery":"Open3DBench enables modular 3D-IC backend implementation and PPA evaluation on an open platform. Two placement algorithms are presented: Open3D-Tiling for regular macro placement and Open3D-DMP for cross-die co-placement. On test cases the flow records 51.19 percent area reduction, 24.06 percent wirelength reduction, 30.84 percent timing improvement, and 5.72 percent power reduction relative to 2D flows, while demonstrating that wirelength gains alone do not ensure PPA gains and therefore PPA-driven methods are required.","pith_inferences":["The benchmark could serve as a starting point for teams developing new 3D partitioning or routing heuristics.","Future extensions might add support for multi-tier stacking beyond two dies.","Widespread use could reduce reliance on proprietary 3D-IC tool suites for academic and small-team research.","The observation that wirelength and PPA are imperfectly correlated may guide placement research toward multi-objective optimizers."],"forward_implications":["3D-IC backend flows can deliver simultaneous reductions in area, wirelength, timing, and power.","Placement methods must optimize directly for PPA metrics rather than wirelength alone.","Standardized open benchmarks allow consistent comparison of 3D EDA techniques.","Modular integration of thermal simulation is feasible within open backend flows."],"fun_headline_variants":["Open3DBench cuts 3D-IC area 51% vs 2D flows","Benchmark: 51% area and 24% wirelength in 3D","Open3DBench: wirelength gains miss 3D PPA benefits","3D-IC open benchmark improves timing 30% over 2D"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The open modular flow produces results that align with those from advanced 3D-IC design practices that combine commercial tools and custom scripts.","fun_headline_variants_meta":{"raw":{"variants":["Open3DBench cuts 3D-IC area 51% vs 2D flows","Benchmark: 51% area and 24% wirelength in 3D","Open3DBench: wirelength gains miss 3D PPA benefits","3D-IC open benchmark improves timing 30% over 2D"]},"model":"grok-4.3","cost_usd":0.009101,"raw_usage":{"total_tokens":4098,"prompt_tokens":700,"num_sources_used":0,"completion_tokens":85,"cost_in_usd_ticks":91012000,"prompt_tokens_details":{"text_tokens":700,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3313,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":700,"tokens_out":85,"duration_ms":25433,"temperature":1.0,"reasoning_tokens":3313,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-22T23:55:00.304892+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the same benchmark designs through a commercial 3D-IC tool chain and finding no area, timing, or power gains would falsify the reported improvements.","supporting_citations":[],"review_version":1}