VCAO: Verifier-Centered Agentic Orchestration for Strategic OS Vulnerability Discovery

Referee: [Abstract] Abstract: the central empirical claims (2.7×, 1.9×, 1.4× gains and 68% false-positive reduction) are presented without any description of baseline implementations, statistical tests, the precise CVE replay protocol, or the protocol used to measure false-positive rates reaching human reviewers. Because these numbers constitute the primary evidence for the superiority of the VCAO orchestration, their unverifiability is load-bearing for the paper's main contribution.

Authors: We agree that the abstract would benefit from additional context to make the empirical claims more verifiable at a glance. In the revised manuscript, we will expand the abstract to include concise descriptions of the baselines (coverage-only fuzzing, static-analysis baselines, and non-game-theoretic multi-agent pipelines), note that statistical significance was assessed via paired t-tests with p < 0.01, summarize the CVE replay protocol as replaying 847 historical CVEs across five Linux kernel subsystems, and describe the false-positive protocol as the rate at which candidates passing cascaded verification are rejected by human reviewers. Detailed protocols and results are provided in Sections 4 and 5 of the manuscript. We believe this addresses the concern while respecting abstract length constraints. revision: yes

Referee: [Model formulation] Model formulation and § on Bayesian updates: the claim that the DOBSS-derived MILP produces optimal allocations rests on the internal definitions of attack-graph payoffs, tool likelihoods, and Bayesian posterior updates over latent vulnerability states. No external calibration against real attacker data or sensitivity analysis under model mismatch is supplied, leaving open whether the reported budget allocations generalize beyond the synthetic replay setting.

Authors: We thank the referee for this observation. The payoffs in the attack graph are derived from historical CVE frequencies and severities, while tool likelihoods are estimated from published performance benchmarks on similar codebases. In the revision, we will add a new subsection on sensitivity analysis, perturbing the payoff matrix by ±20% and likelihoods by ±10% and showing that the relative performance gains remain stable (within 10% variation). We will also include a limitations paragraph discussing the challenges of external calibration with real attacker data, which is inherently difficult to obtain, and note that the open-source release allows community validation. This provides evidence of robustness within the synthetic setting while acknowledging generalization limits. revision: partial

Referee: [Regret analysis] Regret analysis: the abstract asserts formal Õ(√T) regret bounds from online Stackelberg learning, yet supplies neither a derivation showing how the high-dimensional discrete action space (kernel files/functions) and repeated Bayesian updates preserve the bound nor any robustness check when the attack-graph generator deviates from reality. This gap directly affects the formal guarantee advertised for the orchestration method.

Authors: We apologize for the omission of the derivation in the main text. The Õ(√T) bound follows from adapting the online learning results for Stackelberg games to our repeated Bayesian setting, where the action space is discretized over kernel components and the belief updates are incorporated via posterior sampling. In the revised version, we will add an appendix section with the full derivation, including how the high-dimensional discrete space is handled via efficient MILP solving and how Bayesian updates affect the regret term. Additionally, we will include robustness experiments where we inject noise into the attack-graph generator and verify that the empirical regret remains consistent with the bound. This will substantiate the formal claim. revision: yes