{"id":"c2b4725f-020e-4bbc-8d89-fc5d049accec","arxiv_id":"2412.05534","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"MIP uses memory-bank prompts and variance-based invariant training to improve urban flow prediction under distribution shifts on METR-LA and NYCBike1.","lead":"This paper presents MIP, a spatial-temporal forecasting framework that adds a memory bank and an invariant-learning objective to graph neural networks, aiming to keep urban flow predictions accurate when data distributions shift over time. It is worth a look because reliable traffic and crowd forecasting matters for cities, and the paper claims consistent gains over strong baselines on two public benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (15) feeds variant prompts into the deployed prediction backbone, so the paper's central claim that predictions use only invariant features is not supported by its own architecture.","rationale":"The reader's weakest_assumption is Assumption 1, the existence of an invariant sufficient statistic. My concern is different and, I think, more decisive: regardless of whether Assumption 1 holds, the paper's own architecture in Eq. (15) injects variant prompts into the deployed prediction backbone, so the claimed mechanism cannot be true as stated. The reader's rationale did note this H_I/H_V discrepancy, so there is partial agreement, but the formal weakest_assumption field points to Assumption 1. I also agree with the reader that the invariance objective is not derived from the assumed conditional independence and that the empirical evaluation lacks error bars and released code; these are secondary. The architecture-level inconsistency alone is enough to reject the paper's central claim as written, so I would keep the reader's REJECT verdict. A corrected version that removes H_V from the deployed predictor, derives Linv from Assumption 1, and adds statistical rigor could be reconsidered.","tokens_in":20952,"tokens_out":4115,"duration_ms":40652,"concrete_test":"Reimplement or, if the authors release code, patch the backbone so the initial embeddings in Eq. (15) are G_t^0 = H_t^I instead of H_t^V, retrain on the same chronological split, and report MAE/RMSE/MAPE on test sets 0-2. Then run the mirror variant with G_t^0 = H_t^V but with Linv removed. If the H_I-initialized model matches MIP within repeated-run noise, the discrepancy is harmless; if it is worse, or if the H_V-without-Linv variant retains most of the gain, then H_V is an active input to the deployed predictor and the invariance claim should be withdrawn.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the deployed predictor f_theta1 consumes only invariant features, as stated in the abstract and in Eq. (4). The implementation does the opposite: Section IV-E sets the initial GNN node embeddings to G_t^0 = H_t^V (Eq. 15), and the output of these GNN/Transformer layers is mapped to the final prediction via Eq. (17). Thus H_V is a direct input to the deployed predictor. Moreover, the invariant loss Linv in Eq. (13) is computed from Eq. (11), which uses an auxiliary predictor pred(H_I || hat_H_V) that the paper explicitly says is not used for final predictions. The variance penalty therefore never constrains the influence of H_V in the deployed predictor. Unless the model happens to learn to ignore H_V despite receiving it as input, the advertised invariant-only mechanism is absent; the reported gains could equally come from H_V carrying environment-dependent predictive signal. This is a load-bearing inconsistency because it targets the mechanism promised in the title and abstract, not merely a notational detail.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MIP (Memory-enhanced Invariant Prompt Learning), a framework for urban-flow forecasting under distribution shift. MIP maintains a learnable memory bank of prototype vectors, derives a static semantic adjacency matrix from the bank, and extracts invariant and variant prompt tensors H_I and H_V for each time step by querying the bank with positive and negative softmax scores (Eqs. 8-9). During training, variant prompts are randomly swapped across nodes and time steps (Algorithm 1), and an invariance loss (Eq. 13), computed on a separate auxiliary predictor, is used to encourage H_I to be sufficient for the label. At inference, a GNN/Transformer backbone produces the forecasts. The paper evaluates MIP on METR-LA and NYCBike with three chronologically split OOD test sets, reporting state-of-the-art or near-state-of-the-art results in Table II, along with ablations, hyperparameter sensitivity studies, a case study, and efficiency measurements.","tokens_in":21181,"tokens_out":19201,"duration_ms":175104,"significance":"The problem is timely and the high-level idea, replacing explicit environment enumeration with a latent-space intervention on prompts, is novel and potentially valuable for spatio-temporal OOD learning. The evaluation is honest in one important respect: the OOD test sets are strictly held out and were not used for fitting, so the reported numbers are not circular. The paper also provides ablations and sensitivity analyses and is transparent about its hyperparameters. However, the contribution as written is not established: the deployed backbone is initialized with the variant prompts (Eq. 15), the invariance loss is applied to an auxiliary predictor that is explicitly discarded at inference, and both prompt types are convex combinations of the same memory-bank rows. The invariant-only mechanism promised in the title and abstract is therefore not what the implementation does, and without error bars the SOTA claim is not yet supported. The core idea deserves careful revision; the current manuscript does not substantiate it.","major_comments":[{"comment":"Section IV-E directly contradicts the paper's central claim. The deployed prediction backbone is described as being \"fed with ... the invariant prompts H_I,\" but two sentences later the same section sets the initial node embeddings to G_t^0 = H_t^V, and the propagation rule (15) then carries these variant prompts through the GNN and temporal Transformer to the final MLP (17). No step in Section IV-E feeds H_I into this backbone, so the inference-time predictor's node features are the variant prompts. This contradicts Eq. (4), which defines the prediction model as f_{theta1}(H_I, G) with Y independent of H_V given H_I, and the abstract's statement that \"the predictions are only made with invariant features.\" Furthermore, the invariance loss L_inv in Eq. (13) is computed through the auxiliary predictor pred(H_I || hat_H_V) of Eq. (11), which the paper explicitly says \"will not be used for computing the final predictions\"; the variance penalty therefore never constrains how the deployed predictor uses H_V. As written, the reported OOD gains cannot be attributed to the advertised invariant-only mechanism; they could equally come from H_V carrying environment-specific predictive signal. The authors must either correct Eq. (15) so that the deployed backbone consumes only H_I, or reformulate the central claim and supply an experiment showing that the final predictor's outputs are unchanged under re-sampling or intervention on H_V.","section":"IV-E (Eq. 15); III-B (Eq. 4)"},{"comment":"Section IV-A claims that the memory bank Phi \"is trained to contain only invariant patterns,\" yet Section IV-B obtains the variant prompt as H_V = softmax(-Q Phi^T) Phi (Eq. 9), a convex combination of the same bank rows. If Phi stores only invariant prototypes, then H_V is also composed of invariant features, and the negation merely down-weights prototypes that the query deems dissimilar; it does not, by construction, produce environment-dependent, label-independent features. The invariant/variant split therefore rests entirely on the model's own softmax scores, with no external signal (known environments, causal annotations, or distributional tests) verifying the conditional-independence premise of Assumption 1 in Section III-B. This matters because the paper's OOD argument depends on the split being semantically meaningful rather than a reparameterization. Please provide either a formal argument for why negative-softmax combinations of invariant prototypes are label-independent given H_I, or an empirical check (e.g., verify that H_V tracks known shift proxies such as time-of-day or week index while H_I does not).","section":"IV-A, IV-B (Eqs. 8-9)"},{"comment":"The empirical support for the SOTA claim needs strengthening. Table II reports a single run per method with no error bars or significance tests, and several reported differences are small (e.g., METR-LA overall MAE: GMSDR 3.42 vs. MIP 3.46; NYCBike(In) overall MAE: STNorm 4.88 vs. MIP 4.87), so the claimed margins may be within run-to-run variation. The sentence in Section V-C that \"MIP clearly outperforms all competing baselines over the three testing sets\" is also contradicted by the METR-LA MAE column of the very same table; the paper later acknowledges this in the \"sole exception\" clause, but the claim sentence should be corrected. In addition, STONE [49], a directly comparable spatio-temporal OOD method reviewed in Section II-B, is absent from the baseline list in Section V-A; either add it or justify its exclusion. Please report multiple seeds with means and standard deviations (or equivalent) and reconcile the claim text with Table II.","section":"V-C, Table II"},{"comment":"Section IV-C and Algorithm 1 introduce the intervention as a random swap of entries of H_V across nodes and time steps and assert that this \"mimics\" the effect of data distribution shifts, but no evidence or theoretical justification is given that such swaps correspond to plausible interventional distributions. The sensitivity analysis in Fig. 5 shows that the intervention rate r can vary from 0.05 to 0.95 with negligible effect on RMSE; the explanation offered (that propagation spreads variant patterns to all nodes) is plausible but unverified, and it is in tension with the claim that the spatial-temporal intervention is a key driver of invariance. Please clarify what the intervention is intended to achieve, why its rate is largely inconsequential for the final RMSE, and provide an experiment isolating the contribution of the intervention (e.g., comparing the swap against random noise or against no intervention while keeping the same loss).","section":"IV-C, Algorithm 1, Fig. 5"}],"minor_comments":[{"comment":"The objective is written as \"arg min_{theta1, theta1}\"; the second subscript should be theta2 so that the extractor f_{theta2} is actually optimized. The subsequent equations also switch between psi_{theta2} and f_{theta2} for the same object, which should be made consistent.","section":"III-B, Eq. (4)"},{"comment":"The decomposition P(Y|X^s) = sum_s P(Y|X, E=E_s) P(E=E_s) is said to follow from \"Bayesian rules,\" but as written the conditioning sets are inconsistent (X^s on the left, X and E on the right); rewrite or remove this equation and the surrounding argument.","section":"IV-G, Eq. (21)"},{"comment":"The sentence \"we number the tree test sets with 0, 1, and 2\" contains a typo: \"tree\" should be \"three.\"","section":"V-A"},{"comment":"Several typos appear in this section: \"the matrices of the final horizon\" should be \"the metrics of the final horizon,\" \"METE-LA\" should be \"METR-LA,\" and the TESTAM MAPE value \"43.63 %\" in the NYCBike(Out) block contains a stray space.","section":"V-C, Table II"},{"comment":"The phrase \"under constant distribution shifts\" is ambiguous; the intended meaning appears to be continuous or pervasive distribution shifts, and the wording should be clarified.","section":"Abstract"},{"comment":"Figure 1's legend contains \"invariment feature\" and Fig. 3's caption contains \"Interventioned prompts\"; both should be corrected to \"invariant\" and \"intervened,\" respectively.","section":"Figs. 1 and 3"},{"comment":"The efficiency measurements report sub-millisecond figures without stating the measurement protocol (e.g., warm-up, number of repeated runs, batch composition) or any variance; please add these details or soften the efficiency claim.","section":"V-G, Figs. 10-11"},{"comment":"Since STONE [49] is discussed in the related work as a graph-mask-based OOD method, the \"Difference with Other Invariant Learning Methods\" paragraph in Section IV-G should explicitly contrast MIP's latent prompt swap with STONE's mask-based environment simulation.","section":"II-B, IV-G"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the basic idea is worth pursuing, but the manuscript as submitted is not in a publishable state: the central mechanism described in Section IV-E is internally contradicted by Eq. (15), and the empirical claims lack error bars. If the authors can correct Eq. (15) to match the invariant-only description and add the missing validation, the paper could become a solid contribution. If, on the other hand, the deployed model really does consume H_V, then the paper's framing would need to change substantially (e.g., to a memory-augmented prompt-based STGNN rather than an invariant-learning method), and much of the novelty argument would need to be revisited. I also note that the omission of STONE [49] from the baselines is unfortunate given that it is the closest recent competitor and is cited in the paper itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The positive/negative softmax query over a shared memory bank to produce invariant and variant prompts, and then swapping variant prompts across nodes and time as a latent intervention, is a new combination; I don't see that exact setup in CaST, CauSTG, or STONE. The chronological three-test-set evaluation is a reasonable OOD protocol, and since the test sets are held out, the reported gains over baselines are not circular. If the mechanism worked as advertised, this would be a solid contribution.\n\nBut the mechanism is not as advertised. Section IV-E says the prediction backbone is fed the invariant prompts, then Eq. (15) initializes the GNN node embeddings with G^0_t = H^V_t, the variant prompts. Those GNN/Transformer outputs go into the final prediction MLP (Eq. 17), so the deployed predictor consumes H_V directly. This contradicts the abstract, Eq. (4), and the whole framing. It's not a notation nitpick; it targets the central claim. Worse, the invariant loss (Eq. 13) is computed through the auxiliary predictor pred(H_I || hat_H_V) in Eq. (11), which the paper says is not used for final predictions. So the variance penalty never constrains the deployed predictor's reliance on H_V. The gains could just as easily come from H_V carrying environment-dependent signal.\n\nThe empirical results look plausible but modest, with MIP second on METR-LA MAE and only two datasets tested. No error bars, no code released. The invariance assumption (Assumption 1) is also only loosely connected to the loss: the conditional independence y ⊥ F_V | F_I is not enforced by any term; the variance penalty is a proxy.\n\nWhat's genuinely good: the semantic adjacency matrix built from memory prototypes, the cheap latent intervention, and the clear writing about the intended design. The H_V/H_I mismatch may be a fixable typo, but as written the paper doesn't support its headline mechanism.\n\nFor a reader, this is a good discussion piece on invariant learning and internal consistency, but I wouldn't cite it as-is. I would send it to a serious referee rather than desk reject; the idea is novel and the error is specific enough that a corrected version could be a solid paper. Recommendation: reject in current form, invite resubmission with the discrepancy fixed, error bars added, and code released.","headline":"A genuinely new prompt-learning combination for OOD urban flow, but the paper's own equations feed variant prompts into the final predictor, and the invariant loss never constrains that path.","tokens_in":21736,"tokens_out":4295,"would_cite":false,"duration_ms":36455,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Urban flow forecasting can be made robust to distribution shifts by learning which patterns are causal and which are spurious.","keywords":["urban flow prediction","distribution shift","out-of-distribution generalization","invariant learning","prompt learning","memory bank","spatial-temporal graph neural networks","traffic forecasting"],"falsifier":"Take a period of urban flow data with a known external event (e.g., a holiday or a major accident) that changes the relationship between traffic at neighboring sensors and the future flow at a target sensor, so the label genuinely depends on a variant pattern; if MIP's test error on that period does not jump materially compared with baselines, the invariance loss has failed to block the variant signal; a cheaper check is to permute the variant prompts between two distant days at inference time and measure whether the model's predictions shift.","tokens_in":20727,"feed_emoji":"🚦","tokens_out":5155,"duration_ms":48019,"temperature":0.7,"pith_summary":"The paper tries to show that urban flow prediction can be made resilient to distribution shifts by separating causal, invariant patterns from environment-specific variant patterns, without modeling environments explicitly. It introduces MIP, which stores causal prototypes in a learnable memory bank, extracts invariant and variant prompts for every location and time step, and intervenes in latent space by swapping variant prompts across nodes and time. An invariance loss that penalizes the variance of prediction error forces the predictor to rely on invariant prompts only. Comparative experiments on METR-LA and NYCBike across three temporally shifted test sets show MIP outperforming the baselines overall, with gains concentrated in the more shifted test sets.","feed_headline":"Traffic forecasting survives distribution shifts via memory prompts","feed_subtitle":"Stable predictions when traffic data drifts, by learning which patterns are causal and which are spurious.","key_machinery":"The load-bearing object is the trainable memory bank $\\Phi \\in \\mathbb{R}^{M \\times d}$ of $M$ prototype vectors that store causal patterns. Invariant prompts are computed as $H_I^t = \\mathrm{softmax}(Q^t \\Phi^\\top)\\Phi$ and variant prompts as $H_V^t = \\mathrm{softmax}(-Q^t \\Phi^\\top)\\Phi$, so the negation flips the attention to pick patterns the memory bank does not encode. The intervention step randomly swaps entries of the variant prompt tensor across node and time pairs (Algorithm 1), and the invariant learning loss $\\mathcal{L}_{inv} = \\mathbb{E}[l] + \\lambda_1 \\mathrm{Var}[l]$ over per-location, per-time prediction errors enforces that predictions do not change when variant patterns change. A semantic adjacency matrix built purely from the memory-bank prototypes supplies a second graph alongside the geographic graph for information propagation.","core_discovery":"The central claim, stated in the paper's own terms, is that MIP clearly outperforms all competing baselines over the three testing sets for urban flow prediction under distribution shifts, on both the METR-LA traffic-speed dataset and the NYCBike bike-flow dataset. The mechanism is that a trainable memory bank memorizes causal features of the spatial-temporal graph; querying it yields invariant and variant prompts, and swapping a fraction of the variant prompts between nodes and time steps creates synthetic distribution shifts. Training with the invariant learning loss (mean plus variance of the prediction error) then makes predictions depend only on the invariant prompts, which is what gives the model OOD robustness. The paper further claims that this approach bypasses the need to parameterize or enumerate environments, which previous invariant learning methods for spatial-temporal data require.","pith_inferences":["The intervention's validity rests on the swap being confined to truly label-irrelevant patterns; if some variant prompts are partially label-relevant, the same swap will also corrupt the invariant signal, so the method's success is itself evidence for how cleanly the memory bank separates the two.","The same memory-bank-and-prompt design could be carried over to other spatial-temporal forecasting settings, such as crowd flow or energy demand, whenever a small set of causal prototypes plausibly summarizes the stable part of the dynamics.","A direct test of the disentanglement claim would be to freeze a trained MIP and feed it a deliberately 'wrong' variant prompt from a different time and place at inference; predictions that barely change would confirm that only invariant prompts drive the output."],"forward_implications":["Deployed traffic models could keep making accurate predictions for a stretch of time after real-world distribution shifts, without retraining or re-estimating environments.","The method removes the sensitive hyperparameter of the number of virtual environments, which earlier OOD methods for spatial-temporal graphs depend on.","Semantic connections based on causal prototypes allow information to flow between locations with similar underlying flow behavior, even when they are not geographic neighbors.","Because inference uses only the backbone model and drops the prompt extractor and intervention machinery, test-time latency remains low enough for near-real-time use."],"supporting_citations":[{"why":"Supplies the diffusion convolutional backbone, the METR-LA dataset, and a key baseline.","marker":"[6]"},{"why":"Graph WaveNet serves as the predictor pred(·) inside the invariant learning module and as a baseline.","marker":"[7]"},{"why":"Invariant risk minimization supplies the mean-plus-variance principle that MIP adapts for latent prompts.","marker":"[24]"},{"why":"Provides the structural causal model and the conditional independence assumption underlying the invariant/variant split.","marker":"[27]"},{"why":"DIDA is a graph invariant learning method under spatio-temporal distribution shift that MIP builds upon and compares against.","marker":"[29]"},{"why":"DIR's intervention on variant rationales is the conceptual basis for MIP's latent-space intervention on variant prompts.","marker":"[31]"},{"why":"CaST uses an environment codebook for causal treatment of spatial-temporal forecasting, the approach MIP bypasses.","marker":"[32]"},{"why":"CauSTG is a representative environment-based OOD spatial-temporal baseline whose reliance on preset environments MIP avoids.","marker":"[33]"},{"why":"MegaCRN introduces the memory-augmented architecture that motivates MIP's learnable memory bank.","marker":"[42]"}],"fun_headline_variants":["Memory prompts keep urban flow predictions stable under data shifts","Shift-proof traffic forecasting with memory-enhanced invariant prompts","Causal memory bank anchors traffic predictions against distribution drift","Invariant prompt learning via memory bank makes traffic flow OOD-safe","Traffic flow forecasting that learns causal patterns to beat data shifts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that every node's learned features split exactly into an invariant part that fully determines the label and a variant part that is conditionally independent of the label given the invariant part--if real urban flow data has no such clean split, the invariance loss cannot find a stable target and the claimed robustness has no foundation.","fun_headline_variants_meta":{"raw":{"variants":["Memory prompts keep urban flow predictions stable under data shifts","Shift-proof traffic forecasting with memory-enhanced invariant prompts","Causal memory bank anchors traffic predictions against distribution drift","Invariant prompt learning via memory bank makes traffic flow OOD-safe","Traffic flow forecasting that learns causal patterns to beat data shifts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000715,"raw_usage":{"total_tokens":3218,"prompt_tokens":956,"completion_tokens":2262,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":2182}},"tokens_in":572,"tokens_out":2262,"duration_ms":17324,"temperature":1.0,"reasoning_tokens":2182,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:37:27.597605+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a period of urban flow data with a known external event (e.g., a holiday or a major accident) that changes the relationship between traffic at neighboring sensors and the future flow at a target sensor, so the label genuinely depends on a variant pattern; if MIP's test error on that period does not jump materially compared with baselines, the invariance loss has failed to block the variant signal; a cheaper check is to permute the variant prompts between two distant days at inference time and measure whether the model's predictions shift.","supporting_citations":[{"cited_title":"Causality: models, reasoning, and inference, by judea pearl, cambridge university press, 2000,","cited_arxiv_id":null,"evidence_quote":"Provides the structural causal model and the conditional independence assumption underlying the invariant/variant split."},{"cited_title":"Dynamic graph neural networks under spatio-temporal distribution shift,","cited_arxiv_id":null,"evidence_quote":"DIDA is a graph invariant learning method under spatio-temporal distribution shift that MIP builds upon and compares against."},{"cited_title":"Maintaining the status quo: Capturing invariant relations for ood spatiotemporal learning,","cited_arxiv_id":null,"evidence_quote":"CauSTG is a representative environment-based OOD spatial-temporal baseline whose reliance on preset environments MIP avoids."},{"cited_title":"Spatio-temporal meta-graph learning for traffic forecasting,","cited_arxiv_id":null,"evidence_quote":"MegaCRN introduces the memory-augmented architecture that motivates MIP's learnable memory bank."}],"review_version":1}