{"id":"1c456e7b-ef38-4254-850d-380b4b4d2e0f","arxiv_id":"2501.14997","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"DrBO uses Bayesian optimization with low-rank graph embeddings and trained surrogate models to recover causal DAGs, reaching lower structural error than prior score-based methods on several benchmarks.","lead":"DrBO is a new search algorithm that recovers cause-and-effect diagrams from observational data by using Bayesian optimization to decide which diagrams to score next. It reports reaching more accurate diagrams with far fewer expensive score evaluations than prior score-based causal discovery methods, which matters for fields where causal structure must be inferred from data without experiments.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The low-rank DAG family with k=8 is not shown to contain the ground truth, so the dense-graph near-zero SHD may rely on pruning; the 'accurate DAG' claim is conditional.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the sufficiency of the rank-8 low-rank DAG representation is not guaranteed and is only justified empirically. This is indeed the most important gap for the central claim, because the method optimizes exclusively over the image of tau(p,R); any DAG outside this image is unreachable regardless of sample size or time. The paper's own statement in Section 4.1 admits this limitation, and the ablation in Figure 3(a) does not directly test whether the plateau at k=8 is due to representability or to optimization difficulty. My proposed test, running the same pipeline at full rank with a larger budget, would distinguish these causes: if full rank converges to SHD 0 while k=8 does not, the low-rank family is restrictive; if full rank also plateaus, then the search/surrogate, not representability, is the bottleneck. This does not change the verdict because the paper is already CONDITIONAL and the concern was already flagged; it reinforces the need for a representability check before ACCEPT. The other potential issues, such as runtime comparisons and small number of datasets, are secondary and do not threaten the central claim as directly as the representability gap.","tokens_in":31805,"tokens_out":17375,"duration_ms":172447,"concrete_test":"Re-run the Figure 1(a) experiments with DrBO using full rank (k=d=30) on the same five 30ER8 datasets with a larger budget, e.g., 500,000 DAG evaluations, and compare the best SHD achieved by k=8 versus full rank. If full rank reaches SHD 0 while k=8 plateaus above 0, then the default low-rank family is the limiting factor for the 'accurate DAG' claim. If both reach SHD 0, then the low-rank family is empirically sufficient and the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 defines tau(p,R)=H(grad(p)) circle H(R R^T) and defaults to k=8. The paper explicitly concedes that for k<d this map may not cover all DAGs, and supports sufficiency only empirically. This is the load-bearing assumption: if the ground-truth DAG is not in the image of tau for k=8, then the central claim that DrBO 'can find the accurate DAG' cannot hold for those graphs. Technically, {H(R R^T) : R in R^{d x k}} is the family of positive supports of rank-k PSD matrices; for dense random graphs such as 30ER8 (edge probability about 0.55) this is a strongly restricted family. The reported SHD around 1.6 on 30ER8 may therefore reflect a representable supergraph plus the final pruning step, rather than the low-rank family containing the true DAG. The ablation in Figure 3(a) compares ranks and shows better sample efficiency for lower ranks, but it does not establish whether the full-rank representation reaches SHD 0 at convergence or whether k=8 plateaus because of representability limits. Without such evidence, the dense-graph results do not support the claim that the low-rank family suffices.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DrBO, a Bayesian optimization (BO) framework for score-based causal discovery from observational data. DrBO maps DAGs to a low-rank continuous representation tau(p,R)=H(grad(p)) odot H(R R^T) with default rank k=8, then uses batch Thompson sampling with dropout neural networks as scalable surrogates. The surrogates predict node-wise local score components (e.g., log-MSE), which are combined to form an acquisition function; candidate DAGs are proposed in a trust region around the current best representation, evaluated with the true BIC score, and the final output is the highest-scoring evaluated DAG, optionally pruned. The paper reports extensive experiments on synthetic linear-Gaussian, nonlinear GP, discrete, and real-world networks, showing that DrBO reaches low SHD with far fewer DAG evaluations than gradient-based (NOTEARS+TMPI, DAGMA, COSMO, GOLEM) and RL-based (CORL, ALIAS) baselines, especially on dense 30-node and 100-node graphs. Ablations examine rank, surrogate model choice, indirect local-score modeling, continual training, and hyperparameters. The authors provide code and release the implementation.","tokens_in":32074,"tokens_out":17445,"duration_ms":155104,"significance":"If the empirical findings hold, DrBO is a substantial practical advance: it demonstrates that BO with scalable neural surrogates can be competitive for high-dimensional discrete structure search, achieving near-perfect recovery on dense graphs where baselines remain far from the truth. The paper is notable for the breadth of settings (linear, nonlinear, discrete, standardized, large-scale), the per-component ablations, and the fact that the evaluation protocol is not circular: the final DAG is selected by the true BIC among evaluated candidates, while the dropout surrogates only guide candidate generation. The low-rank parameterization is a clever way to reduce the search dimension, and the paper honestly acknowledges the representational caveat for k<d. The main limitation is that the central claim that the accurate DAG is found rests on an empirical, not proven, sufficiency of the rank-8 family.","major_comments":[{"comment":"The search-space map tau(p,R)=H(grad(p)) odot H(R R^T) with default rank k=8 is the central device that turns Eq. (1) into an unconstrained problem, and the paper's own text concedes that for k<d this family may not contain every DAG. This is load-bearing for the abstract claim that DrBO 'can find the accurate DAG': on dense 30ER8 graphs the reported near-zero SHD (Figure 1(a), final SHD ~1.6) is computed after the pruning step described in Section 4.6, so a representable supergraph plus pruning could produce the same result even if the ground-truth DAG is not in the image of tau(.,.) for k=8. The rank ablation in Figure 3(a) only shows that lower ranks reach low SHD faster; it does not report whether the k=8 curve converges to SHD=0 with more evaluations, nor whether the full-rank representation (which can represent any DAG) eventually reaches SHD=0 on the same 30ER8 instances. Please add a direct test of representational sufficiency: for example, run k=d on 30ER8 with a large budget and report the unpruned best-SHD, report the SHD of the best unpruned DAG for k=8, or check membership of the ground-truth DAGs in the image of tau for k=8. Without such evidence, the dense-graph results remain conditional on an unverified assumption.","section":"Section 4.1 (Eq. (4)), Figure 3(a), Section 4.6"},{"comment":"The claim that DrBO finds the accurate DAG in 'less time' is not supported by the per-evaluation runtime data in Table 7: at 50,000 evaluations DrBO takes 22.9 minutes on 30ER8, while ALIAS, CORL, and COSMO take 2.4, 5.2, and 2.8 minutes respectively, and DAGMA takes <0.1 minutes. The 'less time' statement is only true in the sense of time to reach a target SHD (e.g., Figure 1(a), fourth column). Please either qualify the abstract and Section 5 to state the time-to-accuracy interpretation, or provide a comparison in which DrBO is faster in wall-clock for the same evaluation budget; as written, the abstract's 'less time' is contradicted by the authors' own table.","section":"Abstract, Section 5.1, Table 7"}],"minor_comments":[{"comment":"In the proof of Lemma 1, 'multiplying this adjacency matrix with the second term' should read 'taking the Hadamard product with the second term'; the current wording suggests matrix multiplication.","section":"Appendix A.1"},{"comment":"The first-column bar panels report TPR, Precision, Recall, F1, and FDR without numeric labels on the bars; providing the values (or a companion table) would make the aggregate comparison reproducible without reading the text.","section":"Figure 1"},{"comment":"The DAGMA runtime entries are reported as 0.0 minutes; given that these values are used to support runtime claims, reporting more significant digits would be more informative.","section":"Table 7"},{"comment":"The phrase 'absolute overall performance' is vague; it should be stated explicitly that DrBO achieves the best value on all five metrics simultaneously.","section":"Section 5.1"},{"comment":"The BnLearn experiments generate continuous data from the real network structures; stating this in Section 5.2 (rather than only in the appendix) would prevent a misreading that discrete real data are being modeled.","section":"Appendix D.3 / Section 5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is already published at ICLR 2025; this report is for a journal submission. The core method is sound and the experiments are extensive, but the authors should address the low-rank representability gap and the runtime wording before archival journal publication. The second major comment is relatively easy to fix by rewording; the first requires either new experiments or a softened claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely new combination—BO for observational score-based DAG learning, with a low-rank DAG encoding and node-wise dropout surrogates—and the experiments are extensive and largely convincing. The headline result, near-zero SHD on dense 30- and 100-node linear-Gaussian problems, is worth taking seriously, and the ablations show each component earns its place. I'd send it to referees.\n\nWhat's new: prior BO work in causal discovery was active/interventional; this is the first to optimize a score purely from observational data. The low-rank representation (edge potential as R R^T with k=8) and the local surrogate decomposition are sensible, and the continual training trick is what makes the method scale to tens of thousands of evaluations. The paper is honest about the main weakness: the k<d low-rank family is not proven to contain the true DAG. The paper states this in Sec 4.1 and supports it only empirically. That's a real limitation, and the stress-test's worry that the reported SHD relies on pruning is fair. But it doesn't sink the paper: the experiments include SHD 0 on 20ER8 and near-zero on 30ER8, which is strong evidence that the restricted family is practically sufficient for these benchmarks. Still, a theoretical characterization or a failure-mode analysis of rank selection would strengthen it.\n\nThe 'less time' claim is overbroad. From Tables 7-9, DrBO's per-evaluation cost is much higher than DAGMA's; it wins on accuracy and often on time-to-target-SHD, but the abstract's blanket 'less time' is not supported. The fixed-time plots in Figure 1 help, but the text should be more careful.\n\nOther soft spots: only 5 random datasets for main comparisons (fine for a conference, but confidence intervals are wide on some baselines), and the surrogate architecture has several hyperparameters that are empirically tuned. None of these are fatal.\n\nBottom line: a solid, useful paper for anyone doing score-based causal discovery, especially on dense or high-dimensional graphs. It deserves peer review and probably publication after minor revisions; the authors should soften the runtime claim and add a discussion of low-rank representability limits.","headline":"A solid BO-based DAG learner with genuinely strong empirical results; the unproven low-rank sufficiency and the overbroad 'less time' claim are the main caveats.","tokens_in":32610,"tokens_out":2387,"would_cite":true,"duration_ms":21742,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DrBO leverages Bayesian optimization over a low-rank, acyclicity-free DAG representation with dropout-network surrogates to recover high-scoring causal DAGs from observational data in fewer trials and less time than state-of-the-art…","keywords":["causal discovery","Bayesian optimization","score-based DAG learning","low-rank representation","dropout neural networks","continual learning","observational data","structural Hamming distance"],"falsifier":"Take a ground-truth adjacency $A$ from the paper's own 30-node ER8 benchmark and solve the feasibility problem: does there exist $p\\in\\mathbb{R}^{d}$ and $R\\in\\mathbb{R}^{d\\times 8}$ such that $\\tau(p,R)=A$? If even one such graph is infeasible at the default rank $k=8$, then unlimited evaluations cannot recover that exact DAG, and the paper's sufficiency claim for 'very complex graphs' would need qualification.","tokens_in":31563,"feed_emoji":"🎯","tokens_out":9896,"duration_ms":83562,"temperature":0.7,"pith_summary":"This paper tries to establish that score-based causal discovery from purely observational data can be made substantially more sample-efficient by treating DAG search as a Bayesian optimization problem. It proposes DrBO, which optimizes the BIC score over a low-dimensional unconstrained representation of DAGs: each node gets a potential and a rank-$k$ embedding vector, and an edge exists only when the potential increases and the embedding dot product is positive. Dropout neural networks learn the node-wise score components from past evaluations and combine them into an acquisition function, so the method spends its evaluations on the most promising DAG candidates. Across dense 30- and 100-node linear-Gaussian graphs, nonlinear additive-noise data, and several real structures, DrBO is claimed to reach near-zero structural error in fewer evaluations and less wall-clock time than gradient-based and reinforcement-learning baselines. If true, this would make accurate causal discovery practical in settings where score evaluations are expensive or trial budgets are tight.","feed_headline":"Bayesian optimization recovers dense causal graphs almost exactly","feed_subtitle":"Low-rank DAG search plus dropout surrogates cuts evaluations and runtime versus state-of-the-art baselines.","key_machinery":"The central object is the low-rank DAG parametrization $\\tau(p,R):=H(\\operatorname{grad}(p))\\odot H(RR^{\\top})$, where $H$ is the entry-wise Heaviside step, $\\operatorname{grad}(p)$ records differences $p_j-p_i$, and $R\\in\\mathbb{R}^{d\\times k}$ with $k\\ll d$. It converts the combinatorial acyclicity constraint into a potential ordering: edges only go from lower to higher potential, so cycles are impossible, and the embedding-dot-product factor decides which of those potential-consistent edges are present. This map is what makes Bayesian optimization feasible, because the search variable $z=(p,R)$ has only $d(1+k)$ dimensions and lies in a known bounded box. The other load-bearing component is the surrogate: $d$ dropout single-layer networks, each predicting one node's local score component from the binary parent-set indicator, trained continually with a reservoir replay buffer, so that Thompson-sampling-style acquisition is cheap and scales with the number of evaluations.","core_discovery":"The central claim is that the high-scoring DAG search can be solved effectively by Bayesian optimization once the DAG space is re-parametrized into a continuous, low-rank, acyclicity-free form and the score is modeled node-wise rather than globally. The paper proves that the map $\\tau(p,R)=H(\\operatorname{grad}(p))\\odot H(RR^{\\top})$ always outputs a DAG and is scale-invariant, so optimization can be restricted to a fixed hypercube of dimension $d(1+k)$. On top of this, DrBO trains $d$ independent dropout networks, one per node, to predict local score components such as the log-MSE terms of BIC, then forms acquisition values by combining sampled local scores according to the BIC rule; a continual-training replay buffer keeps retraining cost low. The paper's empirical conclusion is that this design finds near-optimal DAGs, often with structural Hamming distance close to zero where comparison methods remain well above 100, using fewer trials and less time.","pith_inferences":["The authors leave implicit that the node-wise surrogate trick only requires the score to decompose additively over nodes; any consistent score with a local decomposition can drop into DrBO unchanged, making it a template for score-family-specific search.","Their ablation shows lower ranks give more unique DAGs per random batch and faster convergence, so rank $k$ could be annealed upward during search as a built-in curriculum, a variant the paper does not test.","The representability condition is graph-side, not data-side: checking whether a candidate DAG is in the image of $\\tau$ at fixed $k$ is independent of sample size, so the feasibility question the paper leaves open is directly testable on its own benchmarks."],"forward_implications":["Score-based causal discovery can be driven by Bayesian optimization instead of gradient or reinforcement-learning search, which matters when each score evaluation is expensive.","On dense 30-node and 100-node linear-Gaussian graphs, DrBO reaches structural Hamming distance near 1–2 where baselines remain above 100 SHD at the same evaluation budget.","The same method transfers to nonlinear additive-noise data, binary and discrete data, BGe scores, and standardized data, with SHD often close to zero.","Because the surrogate scales with the number of evaluations and with dimensionality, DrBO can run into tens of thousands of trials on 100-node graphs, where GP-based Bayesian optimization would be impractical."],"supporting_citations":[{"why":"Supplies the Vec2DAG operator that DrBO compresses to low rank, and the ALIAS baseline used for comparison.","marker":"Duong et al., 2024"},{"why":"Supplies the low-rank DAG assumption that reduces the search space from $O(d^2)$ to $O(d(1+k))$ dimensions.","marker":"Fang et al., 2023"},{"why":"Provides the constraint-free smooth acyclic orientation idea and the COSMO baseline that DrBO must beat.","marker":"Massidda et al., 2024"},{"why":"Justifies interpreting dropout forward passes as approximate posterior samples, which is what makes the acquisition function cheap.","marker":"Gal & Ghahramani, 2016"},{"why":"Provides reservoir sampling, the replay mechanism that lets DrBO train continually without full retraining.","marker":"Vitter, 1985"},{"why":"Defines the BIC score that DrBO optimizes in the main experiments.","marker":"Schwarz, 1978"},{"why":"Provides the gradient-based DAGMA baseline and the acyclicity characterization that DrBO compares against.","marker":"Bello et al., 2022"},{"why":"Provides the CORL reinforcement-learning baseline and the ordering-based DAG generation context that motivates the unconstrained representation.","marker":"Wang et al., 2021"}],"fun_headline_variants":["Bayesian optimization finds causal graphs in fewer trials","Dropout nets accelerate causal graph search","Near-exact causal graphs via Bayesian optimization","DrBO recovers dense DAGs with fewer evaluations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a fixed low rank, by default $k=8$, is rich enough to represent the true DAG, or at least some equally high-scoring DAG, in the $\\tau$ family; the paper gives empirical evidence that it suffices for its test graphs, but concedes that this is not guaranteed when $k<d$.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian optimization finds causal graphs in fewer trials","Dropout nets accelerate causal graph search","Near-exact causal graphs via Bayesian optimization","DrBO recovers dense DAGs with fewer evaluations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000641,"raw_usage":{"total_tokens":2954,"prompt_tokens":950,"completion_tokens":2004,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":1955}},"tokens_in":566,"tokens_out":2004,"duration_ms":22364,"temperature":1.0,"reasoning_tokens":1955,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:43:45.450543+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a ground-truth adjacency $A$ from the paper's own 30-node ER8 benchmark and solve the feasibility problem: does there exist $p\\in\\mathbb{R}^{d}$ and $R\\in\\mathbb{R}^{d\\times 8}$ such that $\\tau(p,R)=A$? If even one such graph is infeasible at the default rank $k=8$, then unlimited evaluations cannot recover that exact DAG, and the paper's sufficiency claim for 'very complex graphs' would need qualification.","supporting_citations":[{"cited_title":"Reinforcement Learning for Causal Discovery without Acyclicity Constraints","cited_arxiv_id":"2408.13448","evidence_quote":"Supplies the Vec2DAG operator that DrBO compresses to low rank, and the ALIAS baseline used for comparison."},{"cited_title":"On low-rank directed acyclic graphs and causal structure learning","cited_arxiv_id":null,"evidence_quote":"Supplies the low-rank DAG assumption that reduces the search space from $O(d^2)$ to $O(d(1+k))$ dimensions."},{"cited_title":"Constraint-free structure learning with smooth acyclic orientations","cited_arxiv_id":null,"evidence_quote":"Provides the constraint-free smooth acyclic orientation idea and the COSMO baseline that DrBO must beat."},{"cited_title":"Dropout as a bayesian approximation: Representing model uncertainty in deep learning","cited_arxiv_id":null,"evidence_quote":"Justifies interpreting dropout forward passes as approximate posterior samples, which is what makes the acquisition function cheap."},{"cited_title":"Estimating the dimension of a model","cited_arxiv_id":null,"evidence_quote":"Defines the BIC score that DrBO optimizes in the main experiments."}],"review_version":1}