{"id":"9bfe79b1-9937-483f-83ed-10425ad69678","arxiv_id":"2506.06817","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"ASPO modifies Bayesian optimization with a categorical kernel, smooth constraint penalties, and checkpoint-based evaluation acceleration, and reports faster and better soft-processor configurations on three RISC-V cores.","lead":"This paper describes ASPO, a Bayesian optimization framework for tuning soft processors on FPGAs that adds categorical parameter handling, constraint checking, and synthesis-checkpoint reuse. The authors report up to 35% faster execution on a multiply benchmark and up to 74% shorter design time versus prior BO methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (2) encodes 'x1 in A and x2 in B' instead of 'if x1 in A then x2 in B', so ASPO rejects valid configurations and the constraint-awareness claim is unsupported as written.","rationale":"The paper's central contribution is constraint-aware Bayesian optimization. For that contribution to work, the smooth constraint functions must faithfully encode the stated constraints. Eq. (2)-(3) do not: taking min(c1,c2) requires both interval memberships, whereas an implication requires only that either the antecedent is false or the consequent is true. This is a direct logical error, not a matter of tuning or empirical uncertainty. It makes the reported invalid-design rates and design-quality numbers unreliable, because the optimizer may discard good designs and effectively enforce a different constraint set than the one claimed. The reader's stated weakest assumption, that weighted Euclidean distance predicts synthesis time, is a real but secondary concern; even a perfect cost proxy would not fix the constraint encoding. I therefore agree with the REJECT verdict, though only partially with the reader's framing of the weakest assumption. The paper does describe a concrete system and a plausible checkpoint-reuse mechanism, but the mathematical core of the constraint contribution is wrong as written, so the current version cannot be accepted. A corrected encoding with rerun experiments would be needed before the empirical claims can be evaluated.","tokens_in":13583,"tokens_out":5802,"duration_ms":58726,"concrete_test":"Compute Eq. (2) for the first BOOM conditional in Table III with x1=icache_nWays=4 and x2=nSets=64: c_1=-(4-64)(4-128)<0, so C_b<0 even though the implication is true. Then enumerate the feasible set under the intended implication and under Eq. (2) to confirm that valid configurations with antecedent false are rejected. If this is confirmed, rerun the BOOM multiply and spmv experiments with the corrected encoding max(-c_1,c_2) and compare TDT and EET; any material shift would show the headline results depend on the incorrect constraints.","verdict_should_be":"REJECT","load_bearing_attack":"Section III-B defines a basic conditional constraint 'if x1 lies in [a1,b1], then x2 must lie in [a2,b2]' via Eq. (2)-(3): C_b(x)=min_m c_m(x_m), c_m(x_m)=-(x_m-a_m)(x_m-b_m). Because c_m is nonnegative iff x_m is inside its interval, C_b(x)>=0 iff x1 in [a1,b1] AND x2 in [a2,b2]. This is a conjunction, not an implication. The intended rule is satisfied whenever x1 is outside the antecedent interval regardless of x2; the paper's function rejects those valid configurations. A correct smooth encoding would be max(-c_1(x1), c_2(x2)). The error propagates through the disjunctive and conjunctive combinators in Eq. (4)-(5), so the feasibility mechanism central to the paper does not implement the constraints listed in Table III. For example, for 'if icache nWays in [64,128] then nSets in [2,4]', the valid configuration (nWays=4, nSets=64) has c_nWays(4)<0 and hence C_b<0, and would be treated as invalid. The reported 34.6% EET improvement and up to 74% TDT savings are therefore not grounded in a correct constraint-aware optimizer, and the central claim is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ASPO, a Bayesian optimization (BO) framework for tuning FPGA-based soft processors. ASPO introduces three modifications to standard BO: a covariance kernel customized for categorical parameters, smooth constraint functions to encode parameter dependencies, and a cost-aware acquisition function that penalizes candidate configurations that are far from previously synthesized checkpoints, enabling reuse of incremental synthesis results. The approach is evaluated on three RISC-V soft processors (EL2 VeeR, RocketChip, BOOM) using seven benchmarks, reporting reductions in estimated execution time (EET) and total design time (TDT) compared to baselines such as VBO, BOOM-Explorer, RCBO, and Boomerang.","tokens_in":13865,"tokens_out":5374,"duration_ms":58302,"significance":"If the claims hold, ASPO would be a practically useful contribution to FPGA design-space exploration, particularly for combining categorical parameters, parameter constraints, and synthesis-cost awareness in a BO framework. The paper provides open-source artifacts and targets real soft-processor designs, which increases its potential impact. However, the current manuscript contains a clear mathematical error in the core conditional-constraint encoding, which undermines the central 'constraint-aware' claim. The reported quantitative results are therefore not grounded in the described algorithm, and the checkpoint-proxy mechanism is not empirically validated. The paper's contribution is potentially salvageable, but a major revision with corrected formulations and re-run experiments is needed before the claims can be assessed.","major_comments":[{"comment":"The constraint function C_b(x) = min_m c_m(x_m) with c_m(x_m) = -(x_m - a_m)(x_m - b_m) does not implement the conditional constraint 'if x1 in [a1,b1] then x2 in [a2,b2]'. Since c_m is nonnegative exactly when x_m lies in its interval, C_b is nonnegative if and only if both x1 and x2 lie in their respective intervals. This is a conjunction, not an implication. A configuration with x1 outside [a1,b1] has c1(x1) < 0, hence C_b < 0, and is rejected even though the conditional constraint is vacuously satisfied. A correct smooth encoding would be C_b(x) = max(-c_1(x_1), c_2(x_2)), which is nonnegative when x1 is outside the antecedent interval or when x2 is inside the consequent interval. This error propagates through the disjunctive and conjunctive combinators in Eq. (4)-(5), and therefore all constraints listed in Table III are mis-encoded by the described method. Because the feasibility function is central to the algorithm, the reported Invalid Design Rate (IDR), Total Design Time (TDT), and Estimated Execution Time (EET) results in Tables VI and VII are obtained with a different, more restrictive feasible region than the one specified. The paper's central claim of constraint-aware optimization is unsupported as written.","section":"Section III-B, Eq. (2)-(3)"},{"comment":"The cost-aware acquisition function and the checkpoint configuration matcher assume that the weighted Euclidean distance d(x,q) is a reliable proxy for FPGA synthesis time. However, the paper provides no experiment or measurement showing this correlation. The weights w* are optimized on a small random subset (Eq. 8) and the cooling parameters lambda0 and k are 'determined empirically through experiments', but no sensitivity or validation is reported. Without such evidence, the reported TDT savings (e.g., up to 74% vs. Boomerang) could be dominated by the growth of the checkpoint database or by changes in the search distribution rather than by the cost-aware mechanism. This is a load-bearing point for the design-time improvement claims and requires either a direct correlation study or ablation experiments demonstrating the mechanism's effect.","section":"Section IV-A, Eq. (7)-(10)"},{"comment":"The description of the categorical covariance kernel is internally unclear and appears technically incorrect as stated. The text says that after one-hot encoding and normalization, 'the covariance function sigma^2(x,x'') for any sample that can be transformed to the same one-hot vector becomes zero' and that this 'prevents the repetitive selection of identical soft processor designs'. If the covariance between an evaluated point and a candidate that maps to the same one-hot vector is zero, the Gaussian process treats those points as uncorrelated, which would not discourage re-selection; it would instead treat the candidate as unexplored. A kernel value of one (or a deterministic tie) would be needed to achieve the claimed behavior. The authors should clarify the intended kernel definition and correct the description; as written, this part of the algorithm does not support the stated objective.","section":"Section III-A"}],"minor_comments":[{"comment":"The acronym 'ASPO' is written as 'ASOP' in the fourth paragraph; please correct the typo.","section":"Section I, Introduction"},{"comment":"The dagger symbol (†) for Hill Climbing TDT values is not explained in the table caption or the surrounding text; clarify whether HC's TDT is excluded from comparison or reported for completeness.","section":"Table VI"},{"comment":"The optimization of w* is stated to use 'heuristic search techniques' on a random subset, but no details of the heuristic, its convergence, or the subset size are given; this makes the checkpoint-matching results difficult to reproduce.","section":"Section IV-A, Eq. (8)"},{"comment":"The paper reports IDR and TDT for each method, but the experimental setup does not state the number of BO iterations or the stopping criterion used by each method; please specify these to allow interpretation of the convergence times.","section":"Section V-C, Table VI"}],"recommendation":"major_revision","confidential_remarks":"The main mathematical error in Eq. (2)-(3) is severe and invalidates the reported experimental results as they stand. However, it is a local, fixable flaw: the conditional constraint can be encoded by max(-c1, c2) (or a smoother variant), and the experiments can in principle be re-run with the corrected feasibility function. The checkpoint-proxy issue is also addressable with additional validation. I am recommending major_revision rather than reject because the paper's overall direction is plausible and the authors provide an open-source platform that could be re-evaluated. The editor should require a corrected constraint formulation, re-run experiments, and a validation of the synthesis-time proxy before considering the paper for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper has a useful engineering core and some real empirical work, but the central constraint-encoding formula is mathematically wrong, and that flaw is load-bearing. I agree with the stress-test note: Eq. (2) with the min over interval memberships encodes a conjunction, not the stated implication. So the checker rejects configurations where the antecedent is false, like (nWays=4, nSets=64) for the constraint 'if icache nWays in [64,128] then nSets in [2,4]'. That entirely undermines the constraint-awareness claim, since the optimizer is being told that valid designs are invalid. The error propagates through the disjunctive/conjunctive combinators, so the whole constraint handling section is unsound as written.\n\nWhat is genuinely new is the integration: one-hot encoding for categoricals, a smooth penalty for constraints, cost-aware acquisition, and checkpoint reuse. The checkpoint reuse idea is sensible, and the experimental setup covers three real soft processors with seven benchmarks, which is substantial. The reported 74% TDT reduction over Boomerang on BOOM design time is plausible and worth chasing down.\n\nBut other soft spots pile up. The kernel description in Section III-A is under-specified and seems self-contradictory: it first says samples mapping to the same one-hot vector share the same kernel value, then says the covariance becomes zero for those samples. That would break the GP. The experiments are single runs without error bars, and the code is not actually available because the web address is removed. The cost-aware acquisition uses weighted Euclidean distance as a proxy for synthesis time but never validates that correlation. These are not all fatal on their own, but they add up.\n\nSo: the math error alone is enough for a reject. The empirical speedups might still be real, but the paper as written cannot be trusted. I would send it to peer review because the topic matters and the system is nontrivial, but I would expect the outcome to be rejection in this form. A corrected constraint function, a proper kernel definition, shared code, and repeated runs would make this worth revisiting.","headline":"The system integration is real but the core constraint formula is wrong, so the paper's central claim is unsupported and it should be rejected, with a path to revision.","tokens_in":14406,"tokens_out":2867,"would_cite":false,"duration_ms":31883,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ASPO claims that Bayesian optimization can be made to respect categorical parameters and hardware constraints by customizing its kernel and acquisition function, yielding faster soft-processor designs in less design time.","keywords":["Bayesian optimization","FPGA","soft processors","categorical parameters","constraint handling","incremental synthesis","design space exploration","RISC-V"],"falsifier":"Measure actual FPGA synthesis times for many pairs of configurations at varying weighted distances and test whether the minimum weighted Euclidean distance to the checkpoint database predicts synthesis time; if the correlation is weak or absent, the cost-aware penalty is steering on a bad proxy and the reported design-time savings would not hold for new processors.","tokens_in":13358,"feed_emoji":"⚙️","tokens_out":8683,"duration_ms":90712,"temperature":0.7,"pith_summary":"ASPO is an attempt to make Bayesian optimization actually work for FPGA-based soft processors—processor designs written in software and deployed on reconfigurable chips—by changing BO's internals rather than bolting on workarounds. The paper's claim is that three customizations—a covariance kernel that respects categorical parameters such as branch-predictor type, smooth functions that encode if-then and divisibility constraints from the processor specification, and an acquisition function that penalizes candidates far from already-synthesized configurations—let the optimizer find valid, fast designs while spending far less wall-clock time on synthesis. If this holds, designers of soft processors can replace manual tuning with an automated search that respects hardware constraints and reports, on the tested RISC-V processors and seven benchmarks, better designs than the default configuration and competing methods in most tasks. The headline numbers are a 34.6% reduction in execution time on the BOOM processor's multiply benchmark and up to 74% lower total design time than a leading hardware-oriented Bayesian optimization baseline.","feed_headline":"Bayesian optimizer for FPGAs cuts design time up to 74%","feed_subtitle":"A cost-aware search that respects hardware constraints finds faster RISC-V soft processors in fewer hours.","key_machinery":"The load-bearing machinery is a modified Gaussian-process Bayesian optimizer with three custom pieces. First, a categorical covariance kernel: one-hot encoded categorical parameters are collapsed to their dominant category before covariance is computed, making similarity depend on actual category identity and preventing redundant sampling of identical designs. Second, smooth constraint-checking functions built from min and max over simple quadratic if-then conditions, plus a sine-based divisibility check, so that conjunctive and disjunctive parameter constraints can guide gradient-based acquisition maximization. Third, a cost-aware acquisition function of the form $\\alpha_{\\text{cool}}(x,t) = \\frac{\\alpha(x)}{\\lambda(t)\\,\\hat{c}(x)}$, where $\\hat{c}(x)$ is the minimum weighted Euclidean distance from the candidate to a configuration in the synthesis checkpoint database and $\\lambda(t)$ decays exponentially over iterations; this is paired with an orthogonal-array warm start and a weighted-distance checkpoint matcher for incremental FPGA synthesis. Together these pieces convert a black-box parameter search into one that respects specification constraints and synthesis cost while exploring the design space.","core_discovery":"The paper's central discovery is that the three obstacles to applying Bayesian optimization to soft processors—categorical parameters, Boolean parameter constraints, and expensive FPGA synthesis—can be absorbed into the BO mechanism itself rather than worked around. Categorical parameters are one-hot encoded and handled by a customized covariance kernel that maps the continuous relaxation back to a hard category assignment, so the optimizer stops resampling identical designs. Constraints are encoded as smooth, differentiable functions using min and max compositions of quadratic if-then conditions and a sine-based divisibility check, so conjunctive and disjunctive specifications become numeric feasibility signals that a gradient-based inner solver can respect. Evaluation cost is folded into a cost-aware acquisition function that penalizes candidates far from already-synthesized configurations, with a decaying penalty schedule, and a checkpoint database plus weighted-distance matcher enables incremental synthesis reuse. The paper reports that this combination yields valid, high-performance configurations faster: a 34.6% execution-time improvement on the BOOM multiply benchmark and design-time reductions up to 74% compared to a state-of-the-art physical-aware BO baseline.","pith_inferences":["The weighted-distance synthesis-time proxy is never validated against measured synthesis times; if the proxy is poor, the cost-aware penalty could bias the search toward cheap-to-synthesize but mediocre configurations, and a direct correlation study would settle this.","The customized categorical kernel and min/max constraint encoding are generic mechanisms that could transfer to other constrained hardware tuning problems, such as high-level synthesis pragma search or accelerator configuration, where the same categorical-plus-constraint structure appears.","The cooling schedule's two parameters are set empirically, so the advantage may be sensitive to them; an adaptive schedule driven by observed synthesis times might remove that tuning burden.","The reported design-time savings conflate two effects—checkpoint reuse and the cost-aware acquisition penalty—so an ablation separating them would show which one drives the improvement."],"forward_implications":["Constraint-heavy processors become tractable: the invalid-design rate drops on most tasks for the larger processors, so designers waste less time on configurations that fail specification or resource checks.","Design time scales better with processor complexity: the savings are largest on the processor with the longest per-evaluation synthesis time, where total design time falls by up to 74% relative to a leading physical-aware BO approach.","Design quality does not suffer for the speedup: ASPO-optimized configurations had the best estimated execution time in 17 of 21 processor-benchmark tasks, including a 34.6% improvement on the multiply benchmark.","Because the framework exposes a configuration interface and supports designer-specified processors, the same constraint encoding and checkpoint reuse can be applied to other configurable RISC-V soft cores with modest modification."],"supporting_citations":[{"why":"Motivates the need for a kernel that handles categorical and integer variables in Gaussian-process BO.","marker":"[2]"},{"why":"Provides the BOOM design specification and constraint examples used to construct the constraint-checking functions.","marker":"[3]"},{"why":"Serves as the physical-aware design-space exploration baseline whose total design time ASPO is compared against.","marker":"[4]"},{"why":"Supplies the vanilla Bayesian optimization baseline for the productivity and design-quality comparisons.","marker":"[9]"},{"why":"Gives the resource-constraint BO method for FPGA soft processors, the closest prior approach and a main comparison point.","marker":"[10]"},{"why":"Demonstrates FPGA incremental synthesis using checkpoint files, the capability the checkpoint database exploits.","marker":"[20]"},{"why":"Provides the cost-aware Bayesian optimization formulation that the acquisition penalty adapts.","marker":"[22]"},{"why":"Supplies the orthogonal-array sampling strategy used for warm-start configuration coverage.","marker":"[21]"}],"fun_headline_variants":["Constraint-aware Bayesian optimizer cuts FPGA design time by 74%","ASPO: BO with categorical constraints reduces FPGA design time 74%","Bayesian optimization for FPGA soft processors: 74% less design time","Custom BO kernel honors hard constraints, speeds FPGA soft processor design"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole design-time speedup rests on the assumption that the minimum weighted Euclidean distance to an already-synthesized configuration predicts how long FPGA synthesis will take, but the paper presents no measurement of that correlation.","fun_headline_variants_meta":{"raw":{"variants":["Constraint-aware Bayesian optimizer cuts FPGA design time by 74%","ASPO: BO with categorical constraints reduces FPGA design time 74%","Bayesian optimization for FPGA soft processors: 74% less design time","Custom BO kernel honors hard constraints, speeds FPGA soft processor design"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000963,"raw_usage":{"total_tokens":4128,"prompt_tokens":998,"completion_tokens":3130,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":3055}},"tokens_in":614,"tokens_out":3130,"duration_ms":25240,"temperature":1.0,"reasoning_tokens":3055,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:48:55.735797+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure actual FPGA synthesis times for many pairs of configurations at varying weighted distances and test whether the minimum weighted Euclidean distance to the checkpoint database predicts synthesis time; if the correlation is weak or absent, the cost-aware penalty is steering on a bad proxy and the reported design-time savings would not hold for new processors.","supporting_citations":[{"cited_title":"Boomerang: Physical-Aware De- sign Space Exploration Framework on RISC-V SonicBOOM Microar- chitecture,","cited_arxiv_id":null,"evidence_quote":"Serves as the physical-aware design-space exploration baseline whose total design time ASPO is compared against."},{"cited_title":"Is Vanilla Bayesian Optimization Enough for High-Dimensional Architec- ture Design Optimization?","cited_arxiv_id":null,"evidence_quote":"Supplies the vanilla Bayesian optimization baseline for the productivity and design-quality comparisons."},{"cited_title":"Resource-Constraint Bayesian Optimiza- tion for Soft Processors on FPGAs,","cited_arxiv_id":null,"evidence_quote":"Gives the resource-constraint BO method for FPGA soft processors, the closest prior approach and a main comparison point."},{"cited_title":"Efficient design space exploration via statistical sampling and AdaBoost learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the orthogonal-array sampling strategy used for warm-start configuration coverage."}],"review_version":1}