{"id":"7703715c-8d3e-4ee1-8e80-19b6d173a846","arxiv_id":"2411.16142","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"Using SyPI time-series causal discovery to build an adjacency matrix improves out-of-distribution mobility forecasts compared to fixed baselines, but the evaluation lacks sparsity controls and error bars.","lead":"This paper learns a causal adjacency matrix for graph-based mobility forecasting by testing which past time series of one region help predict another, then uses that matrix in a standard graph neural network. On COVID-era New York mobility data, the causal matrix beat distance, correlation, and attention matrices for out-of-distribution forecasting.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The OOD gains are not attributable to causal content: no sparsity-matched or random-edge control isolates the causal mechanism, so the central claim is unsupported.","rationale":"The reader's verdict is REJECT, and I agree that the paper should not be accepted in its current form. However, I identify the most load-bearing concern slightly differently from the reader's formal weakest_assumption. The reader emphasizes the stability of causal relations during the Omicron surge; that is a real concern about generalizability. But the more fundamental problem is that the experimental design cannot attribute the OOD gain to causal content at all. Because the CAL matrix changes sparsity, edge set, and weighting jointly, and because the attention baseline is artificially restricted to CAL's edge support, the comparison in Table II is confounded. A random or purely correlation-based matrix with the same number of edges could plausibly match the reported RMSE, which would remove the causal interpretation entirely. This is a correctness risk rather than a novelty or presentation issue, and it directly undermines the abstract's central claim. Thus I do not change the reader's rejection, but I would ground the rejection primarily in the missing sparsity-matched controls rather than in the stability assumption. The proposed concrete test is a single, feasible experiment that would settle the attribution: if sparsity-matched non-causal controls perform the same, the paper's causal claim is not supported; if CAL clearly outperforms such controls, then the causal mechanism becomes credible and a revised version could move toward conditional acceptance.","tokens_in":8712,"tokens_out":4479,"duration_ms":46252,"concrete_test":"Run the downstream GCN on the OOD test split with three control adjacency matrices: (1) a random binary matrix with exactly CAL's number of edges and a similar degree sequence, averaged over 100 draws; (2) a Pearson-correlation matrix thresholded to exactly CAL's edge count; (3) an attention matrix thresholded to its own top edges with the same edge count, rather than forced onto CAL's edge support. Report RMSE/MAE for Table II horizons with standard errors across random seeds. If any control is statistically indistinguishable from CAL on the OOD window, the reported improvement cannot be attributed to causal discovery, and the central claim fails as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that CAL captures causal relations and that this causal content drives the reported OOD improvement (Abstract, Section IV, Table II). The experiments cannot support that attribution because the CAL adjacency differs from every baseline simultaneously in edge-selection rule, edge count, and edge weights. In Section V.B.2, the attention baseline is forced to use 'the same edges as the SyPI adjacency matrix,' so it is not an independent attention-derived graph; the correlation baseline uses a fixed threshold of 0.75 with no attempt to match edge count; and no random-sparsification control is reported. Consequently, the 10-50% RMSE improvements in Section V.C.1 could stem from graph sparsity or from a favorable edge-support draw rather than from causal validity. The stability assumption in Section II.B, that causal relations remain constant across distributions, is also untested: the matrix is learned from the first 50 weeks and applied to the February 2022 Omicron surge (Section V.B.3). But the more fundamental gap is that no experiment isolates the causal mechanism from mere edge pruning. Additionally, the pre-selection size M in Section IV.A.3 is not reported, so the edge count is not held fixed across methods. Without a control that matches CAL's sparsity while randomizing or non-causally selecting edges, the link between causal discovery and OOD performance is an assertion, not a demonstrated result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Causal Adjacency Learning (CAL), an upstream module that discovers causal relations among nodes in spatiotemporal graph data using a kernel-based conditional independence test followed by the SyPI algorithm, and encodes the results into an adjacency matrix for a downstream spatiotemporal GCN. The method is evaluated on SafeGraph weekly mobility data for 172 ZIP codes in the New York area over 90 weeks, with the last 16 weeks (including the February 2022 Omicron surge) treated as out-of-distribution test data. The paper reports that the CAL adjacency matrix improves RMSE/MAE compared with distance-, correlation-, and attention-based adjacency matrices, and it claims that the learned matrix captures causal relations that transfer to the downstream task.","tokens_in":8927,"tokens_out":5625,"duration_ms":53472,"significance":"If fully supported, the paper would make a useful contribution: a transferable, interpretable causal adjacency matrix that improves out-of-distribution spatiotemporal prediction is a valuable module for traffic and mobility forecasting. The idea of extending temporal causal feature selection to spatiotemporal graphs is reasonable and timely. The authors also provide a clear algorithmic description and a geospatial visualization that is suggestive of interpretability. However, the current experimental design does not isolate the causal mechanism from sparsity or edge-selection effects, and no ground-truth validation of the causal edges is provided. The significance is therefore not yet established beyond a promising initial study.","major_comments":[{"comment":"The attention baseline is not an independent attention-derived graph: it is forced to keep the same edges as the SyPI adjacency matrix and only varies the weights. Thus the comparison to Attention in Table II does not test whether attention-based edge selection is inferior to causal edge selection; it only compares edge weights on a fixed edge set. Please construct an attention baseline with its own top-k edge selection and also match edge counts across all baselines.","section":"Section V.B.2, Table II"},{"comment":"There is no control that randomizes or non-causally selects edges while preserving the sparsity and edge count of CAL. Because the CAL adjacency differs from every baseline in edge-selection rule, edge count, and edge weights, the reported 10–50% RMSE improvements cannot be attributed to causal content; they may arise from the particular edge support or from increased sparsity. Add at least a random-edge control with the same number of edges and a correlation-based baseline with edge count matched to CAL.","section":"Section V.C.1, Table II"},{"comment":"The spatial pre-selection size M is never reported in the experiment section, although it is a tunable hyperparameter that determines the candidate edge set and hence the final sparsity of the causal adjacency matrix. Without reporting M and the resulting edge count for each method, the comparison in Table II and the sparsity discussion in Section V.C.1 are not reproducible.","section":"Section IV.A.3 and Section V.B"},{"comment":"The stability assumption that causal relations are invariant across distributions is adopted without testing. The matrix is learned from the first 50 weeks and applied to the last 16 weeks containing the February 2022 Omicron surge; no experiment checks whether the recovered edges would be stable across windows or whether a matrix estimated on the OOD period would differ. Please validate on synthetic data with known ground truth and/or report rolling-window stability results.","section":"Section II.B and Section V.B.3"},{"comment":"The text says “Given different random seeds,” but Table II reports only point estimates without standard deviations, confidence intervals, or significance tests. Since some gains are small (e.g., at T+3, CAL is not the best method), the reported improvements may be within run-to-run noise. Report mean ± standard deviation over seeds and run a paired significance test.","section":"Section V.C.1, Table II"}],"minor_comments":[{"comment":"In the paper-organization paragraph, “real-wi experiments and results” is a typo; it should read “real-world experiments and results.”","section":"Section I"},{"comment":"The sentence mentioning “images from the dataset and vehicle speed data” appears to be a leftover from another dataset description; the SafeGraph dataset used here contains mobility counts and contextual variables, not images or vehicle speed data.","section":"Section V.B"},{"comment":"The notation in Algorithm 2 is inconsistent: “min lags” and “w” are used interchangeably, and the conditioning set “Si” inside the loop is not explicitly defined for each i. Please clarify the algorithm steps and variable definitions.","section":"Algorithm 2"},{"comment":"The comma after the display equation is misplaced, and the time-series length T used in the correlation calculation should be defined explicitly.","section":"Section III.A, Eq. (2)"},{"comment":"The claim of being “the first to consider temporal dimensions” in STPG is too strong, since [12] and [13] operate on temporal chunks; please temper the claim to “first to consider within-temporal causal structure.”","section":"Sections I and II.B"},{"comment":"Please clarify what the color scale and the row/column aggregations represent (e.g., sum of causal weights versus number of causal edges).","section":"Section V.D, Fig. 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript has a promising idea, but the central claim that causal content drives the OOD improvement is not yet supported by the experiments. The missing controls, unreported hyperparameter M, and lack of error bars are all fixable within a revision, so I do not recommend rejection at this stage. Please also verify the citation pattern: several related-work and method references [14], [17], [18], [19] are from the same group, and the paper does not release code or data, which makes the missing M harder to audit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper does something sensible—use SyPI time-series causal selection to build an adjacency matrix for spatiotemporal prediction—but it doesn't show that the causal content is what drives the reported OOD gains. The experiments lack the controls needed to back the central claim.\n\nWhat's new: applying SyPI to adjacency learning with a correlation-based spatial pre-selection is a legitimate extension; the algorithm reduces the O(n^2) cost of CIT to O(Mn). The design choice of a simple GCN downstream is good; it isolates the effect of the adjacency matrix. The visualization of row/column aggregates is a nice touch and consistent with Manhattan being a mobility hub.\n\nSoft spots: The comparison is not controlled. The attention baseline is forced to use the same edges as the CAL matrix, so it isn't an independent attention graph. There's no random or non-causal sparsification control matched on edge count, so the RMSE improvements could come from pruning, not causality. Correlation threshold isn't matched to CAL's edge count. Hyperparameter M in the pre-selection isn't reported, so we can't see how edge counts vary. No error bars or significance tests on the RMSE comparisons. The stability assumption (causal relations fixed across the Omicron surge) is asserted, not tested. And the 'first to consider temporal dimensions' claim is contradicted by their own related-work discussion—[12] and [13] also use temporal data, just in chunks. Finally, the causal interpretation isn't validated against any ground-truth structure; SyPI's assumptions are simply imported.\n\nProportion: For a workshop paper, this is fine; for a claim as strong as 'captures causal relations,' it's not enough. The prediction improvement might be real, but the attribution to causality is unsupported. That said, the idea is worth pursuing, and the paper is clearly written and honest about its choices.\n\nWho should read it: researchers working on OOD generalization for graph-based spatiotemporal forecasting, especially traffic and mobility. It deserves a serious peer review, not a desk reject, but a reviewer should push for sparsity-matched baselines, error bars, reported hyperparameters, and ideally a synthetic ground-truth test. I'd suggest major revision, not acceptance.","headline":"A sensible SyPI-to-adjacency extension, but the experiments don't isolate causality from pruning, so the central claim is overreach.","tokens_in":41,"tokens_out":2049,"would_cite":false,"duration_ms":41800,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Learning causal adjacency improves out-of-distribution graph forecasting.","keywords":["causal adjacency learning","spatiotemporal prediction","graph neural networks","out-of-distribution generalization","conditional independence test","SyPI","human mobility forecasting","COVID-19"],"falsifier":"Re-estimate the causal adjacency matrix on the 16-week out-of-distribution test window alone and compare it with the matrix learned from the first 50 weeks. If the two matrices differ substantially, or if a GCN using the test-window matrix beats the fixed training matrix on the test data, then the assumption that causal relations are constant across the distribution shift is violated and the transferability claim fails.","tokens_in":8438,"feed_emoji":"📈","tokens_out":8657,"duration_ms":70734,"temperature":0.7,"pith_summary":"This paper argues that the adjacency matrix fed into a spatiotemporal graph predictor should encode causal relations among nodes rather than memorized similarities. It introduces Causal Adjacency Learning (CAL), a procedure that runs temporal conditional independence tests on the training time series and filters the results with the SyPI algorithm to build a causal adjacency matrix, which is then used in a downstream graph convolutional network. On real-world records of weekly visits to 172 ZIP codes during the COVID-19 pandemic, CAL's matrix improves mobility forecasts on a 16-week out-of-distribution test window that includes the February 2022 Omicron surge, compared with distance-, correlation-, and attention-based matrices. The finding matters because conventional adjacency matrices are fitted to the training distribution and tend to fail when the test distribution shifts; a transferable causal matrix would make graph forecasting robust to shocks without re-estimating relations at test time.","feed_headline":"Causal adjacency beats distance and correlation in shifting data","feed_subtitle":"A fixed causal matrix learned once cuts average RMSE by 24% on a COVID-surge test window.","key_machinery":"The load-bearing object is the causal adjacency matrix $A_{CAU}$, built by an upstream Causal Adjacency Learning module. For each node, CAL first restricts candidate causes to the $M$ neighbors with highest Pearson correlation (spatial pre-selection), then applies a kernel-based conditional independence test to the lagged time series of each candidate against the target, conditioning on all other candidates and the target's own past. The SyPI algorithm filters the resulting dependencies to keep only those that survive two thresholded tests, removing spurious relations. This matrix is normalized into a graph Laplacian and used in a $K$-order Chebyshev polynomial graph convolution in the downstream spatiotemporal GCN; the claim is that this causal structure, being invariant across distributions, transfers to out-of-distribution test data where correlation and attention structures do not.","core_discovery":"The central claim is that causal relations among nodes in a spatiotemporal graph can be discovered once, encoded in an adjacency matrix, and transferred to a downstream prediction task, where they improve out-of-distribution performance even though the downstream model never performs causal inference. CAL tests, for each target node, whether a candidate neighbor's lagged time series is conditionally independent of the target given all other candidate series, using a kernel-based conditional independence test; the SyPI algorithm then removes false positives and negatives, and a correlation-based pre-selection step keeps the computation at $O(Mn)$ instead of $O(n^2)$. When the learned matrix $A_{CAU}$ is used in a graph convolutional network with Chebyshev spectral filters and a temporal convolution layer, it yields lower RMSE and MAE than distance, correlation, and attention matrices on all prediction horizons from one to four weeks ahead, with an average RMSE reduction over the four-week horizon of 24.71% relative to the distance matrix. The paper also reports that the matrix is sparser than the correlation and attention baselines and that row and column sums give an interpretable picture of regional influence, with Manhattan exerting the largest outward influence and Staten Island the most influenced.","pith_inferences":["The paper leaves untested whether the learned causal structure remains stable over longer horizons or across further shocks; re-estimating $A_{CAU}$ on rolling windows and measuring drift would be a direct stress test.","The same upstream/downstream recipe could extend to other STPG domains with sudden shifts, such as traffic incidents, disease spread, or energy demand, but only the mobility case is demonstrated here.","Because Pearson correlation is used for spatial pre-selection, candidates are pre-filtered by a linear criterion; a nonlinear pre-selection step might admit different edges and change which causal relations survive SyPI.","The downstream model is a deliberately simple GCN to isolate the effect of the adjacency matrix; testing the same matrix in more expressive predictors would show whether the transferable benefit persists across model families."],"forward_implications":["A single upstream causal learning pass is enough: the learned adjacency matrix can be reused by any downstream STPG model without re-running causal discovery.","Forecasts on the out-of-distribution window improve across all horizons; average RMSE over the four-week horizon falls by 24.71% versus the distance matrix and by 10.98% versus the correlation matrix.","The sparser causal matrix (9.67% fewer edges than correlation, 15.98% fewer than attention) reduces computing cost while retaining predictive signal, suggesting many correlation-based edges are noise.","The method surfaces interpretable structure: regional influence patterns, such as Manhattan's dominant outgoing influence, can be read directly from the matrix."],"supporting_citations":[{"why":"Supplies the kernel-based conditional independence test that CAL uses to test temporal conditional independence.","marker":"[25]"},{"why":"Supplies the SyPI algorithm, which filters false positive and false negative causal dependencies from the CIT output.","marker":"[26]"},{"why":"Provides the distance-based adjacency baseline against which CAL's gains are measured.","marker":"[6]"},{"why":"Provides the correlation-based adjacency baseline.","marker":"[16]"},{"why":"Provides an attention-based adjacency baseline.","marker":"[8]"},{"why":"An existing causal latent-variable STPG method that CAL contrasts with for treating time as a chunk and lacking transferability.","marker":"[13]"},{"why":"An existing attention-mask causal method that CAL contrasts with for understudying within-temporal patterns.","marker":"[12]"}],"fun_headline_variants":["Causal adjacency cuts OOD RMSE by 24%","One causal matrix, 24% better OOD forecasts","Causal edges beat distance and correlation in shifting data","Causal matrix learned once, used for OOD gains","24% less RMSE with causal adjacency on OOD graphs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The causal relations among regions learned from the first 50 weeks of training data remain unchanged during the February 2022 Omicron surge, so the fixed adjacency matrix still describes how regions influence each other on the out-of-distribution test window.","fun_headline_variants_meta":{"raw":{"variants":["Causal adjacency cuts OOD RMSE by 24%","One causal matrix, 24% better OOD forecasts","Causal edges beat distance and correlation in shifting data","Causal matrix learned once, used for OOD gains","24% less RMSE with causal adjacency on OOD graphs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00067,"raw_usage":{"total_tokens":3060,"prompt_tokens":958,"completion_tokens":2102,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":2020}},"tokens_in":574,"tokens_out":2102,"duration_ms":14086,"temperature":1.0,"reasoning_tokens":2020,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:30:26.105029+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-estimate the causal adjacency matrix on the 16-week out-of-distribution test window alone and compare it with the matrix learned from the first 50 weeks. If the two matrices differ substantially, or if a GCN using the test-window matrix beats the fixed training matrix on the test data, then the assumption that causal relations are constant across the distribution shift is violated and the transferability claim fails.","supporting_citations":[{"cited_title":"Necessary and sufficient conditions for causal feature selection in time series with latent common causes,","cited_arxiv_id":null,"evidence_quote":"Supplies the SyPI algorithm, which filters false positive and false negative causal dependencies from the CIT output."},{"cited_title":"Trafficgan: Network-scale deep traffic prediction with generative adversarial nets,","cited_arxiv_id":null,"evidence_quote":"Provides the correlation-based adjacency baseline."}],"review_version":1}