{"id":"aaa8a9b5-8574-4754-abc5-9809d17dfd70","arxiv_id":"2506.12764","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Base3, a training-free interpolation of EdgeBank, PopTrack, and the new t-CoMem module, achieves competitive or state-of-the-art MRR on TGB dynamic link prediction datasets.","lead":"A new training-free model called Base3 combines three simple scores, edge memory, node popularity, and co-occurrence memory, for dynamic link prediction. On several temporal graph benchmarks it matches or beats deep learning models while using no training, which could make it useful for low-resource deployments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported competitiveness rests on test-set hyperparameter selection and partly unspecified multi_conf weights; a validation-based re-evaluation is required.","rationale":"Both the reader's verdict and my read point to the same soft spot: the paper's headline result is an empirical comparison, and its validity depends on a clean evaluation protocol. The ablation table reveals that the chosen defaults correspond to the best test MRR, not the best validation MRR, which is a concrete instance of test-set leakage. The missing Table ?? means the multi_conf weights are unspecified, so the exact model is not reproducible and the reported numbers could depend on unstated choices. These issues do not demonstrate that the method is wrong, but they do mean the 'training-free and competitive' claim is not yet established under a fair comparison. The method itself is simple and deterministic; if the code is made available and the weight table is completed, a validation-based re-run could settle the concern. Therefore the correct action remains conditional acceptance, as the reader proposed.","tokens_in":12407,"tokens_out":7076,"duration_ms":71683,"concrete_test":"Re-run the full TGB evaluation with a clean protocol: (i) complete the multi_conf weight table a priori, e.g., fix the four weight vectors to the values stated in the prose and deposit them before seeing test labels; (ii) select the three hyperparameters (memory span, co-occurrence weight, K) per dataset using only validation MRR from a grid search, with no test-set peeking; (iii) report test MRR for the resulting configuration. If the test MRR on tgbl-coin or tgbl-flight drops by more than twice the reported std of the corresponding deep baseline (≈0.01), the headline claim of competitiveness fails. A minimal check is to recompute the tgbl-wiki-v2 row of Table 3 with the validation-selected span 1.0/1.0/1000: if its test MRR (0.727) is used in Table 1, Base3 falls below TNCN on wiki-v2, reversing the 'second place' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—Base3 is competitive with SOTA deep models—depends on the reported MRR numbers being both accurate and fairly obtained. Two related weaknesses undermine this. First, the default configuration is selected using test-set information. In Table 3 (Section 4.3), the authors report both validation and test MRR for tgbl-wiki-v2. For K=1000, the best validation MRR is 0.736 at memory span 1.0 and co-occurrence weight 1.0, while the chosen default (span 0.1, weight 1.0) has validation MRR 0.727 but a higher test MRR (0.743 vs 0.727). The text states the span is fixed 'as that is optimal under optimal choices for other hyperparameters,' but the optimal choice on validation would be span 1.0. The selection therefore appears to maximize test MRR, not validation MRR, which is inconsistent with the standard TGB protocol used for the deep baselines. Second, the multi_conf interpolation weights are not fully defined: Appendix A refers to a 'Table ??' that is missing, and the prose only gives partial weight values (e.g., t-CoMem gets 20% when both signals are reliable; EdgeBank gets 45% or PopTrack 70% in the one-signal cases) without specifying the complete weight vectors or the rule for resolving ties. If these weights were themselves chosen by inspecting test results, the reported advantages over TNCN/DyGFormer (e.g., 0.773 vs 0.762 on tgbl-coin) could reflect overfitting rather than genuine predictive power. At minimum, the method as described cannot be reproduced exactly, and the comparability of the headline numbers is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Base3, a training-free ensemble for dynamic link prediction that linearly combines three non-learnable signals: EdgeBank-style edge recurrence, PopTrack-style node popularity, and a new t-CoMem module that tracks temporal co-occurrence patterns with recency decay. The authors evaluate Base3 and t-CoMem on the Temporal Graph Benchmark (TGB) datasets and report MRR scores competitive with deep learning baselines such as DyGFormer, TNCN, TGN, and GraphMixer, including the highest test MRR on tgbl-coin and strong performance on tgbl-wiki-v2 and tgbl-flight. They also report AUROC under random, inductive, and historical negative sampling. The central claim is that a simple, interpretable, training-free model can rival state-of-the-art deep temporal graph models, especially under more challenging negative sampling strategies.","tokens_in":12722,"tokens_out":3138,"duration_ms":35719,"significance":"If the reported results are reliable, the paper would be a valuable contribution to the growing literature on lightweight baselines for temporal graph learning. The proposed method is deterministic, requires no training, and the authors provide code, which are strengths for reproducibility. The evaluation across five TGB datasets and three negative sampling strategies is broad. However, the significance is substantially undercut by two methodological concerns: the default hyperparameters appear to be selected using test-set performance rather than validation performance, and the multi_conf interpolation weights are not fully specified (Appendix A contains a missing table reference). These issues affect the fairness of the comparison with deep baselines and the reproducibility of the method. If the authors can re-establish the results under validation-based model selection and provide complete weight definitions, the contribution would be solid.","major_comments":[{"comment":"The selection of default hyperparameters appears to maximize test MRR rather than validation MRR. For K=1000 and co-occurrence weight 1.0, memory span 1.0 gives validation MRR 0.736 and test MRR 0.727, while span 0.1 gives validation MRR 0.727 and test MRR 0.743. The text states that span 0.1 is fixed \"as that is optimal under optimal choices for other hyperparameters,\" but a validation-based choice would favor span 1.0. Since the deep baselines follow the standard TGB protocol of selecting hyperparameters on validation, this discrepancy gives Base3 an unfair advantage and undermines the claim that it is competitive with state-of-the-art models on the reported test sets.","section":"§4.3, Table 3"},{"comment":"The interpolation strategy multi_conf is chosen based on test-set results. On tgbl-wiki-v2, EB_conf achieves test MRR 0.752 while multi_conf achieves 0.743; the authors nevertheless select multi_conf because it performs better on tgbl-review-v2 (test MRR 0.108 vs 0.084). This means the default configuration of Base3 is selected using test information from two datasets. To fairly compare against deep baselines, the interpolation strategy should be selected using validation MRR or a separate development set, and the reported test numbers should be re-generated under that selection rule.","section":"§4.4, Table 4"},{"comment":"The multi_conf interpolation weights are not fully specified. The prose states that when both EdgeBank and PopTrack signals are reliable, t-CoMem receives 20%, and when only one signal is positive, EdgeBank receives 45% or PopTrack receives 70%, but the complete weight vectors (α, β, δ) for all four cases are not given, and the text refers to a \"Table ??\" that is missing. Without these weights, the method cannot be reproduced exactly, and there is no way to verify whether the weights were chosen by inspecting test results. The authors should provide the complete weight table and clarify how the weights were determined.","section":"Appendix A"},{"comment":"The t-CoMem time window tw is a free hyperparameter set to 1,000,000 by default, but it is not included in the ablation study in Table 3. Since t-CoMem is a core component of Base3, the reported results are conditional on an unstated and apparently untuned choice. A sensitivity analysis for tw should be provided, or at minimum the default should be justified.","section":"§3.1"}],"minor_comments":[{"comment":"The caption states that first, second, and third best performances are highlighted, but the table as typeset does not show any highlighting; this should be fixed so the reader can verify the rankings.","section":"Table 1 caption"},{"comment":"Equation (4) contains a stray \"= 1\" in the numerator; the formula for the Resource Allocation Index should be cleaned up.","section":"Equation (4)"},{"comment":"The text uses both \"tgbl-review-v2\" and \"tgbl-review\" in Table 4 and the surrounding discussion; the dataset name should be used consistently.","section":"§4.4"},{"comment":"The code link is given as \"available here\" without an actual URL; the URL should be included in the manuscript.","section":"Introduction"},{"comment":"The term \"co-occurence\" is misspelled; it should be \"co-occurrence\" in the text and in the hyperparameter description.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The test-set hyperparameter selection issue is the main concern. If the authors cannot demonstrate that their reported results survive validation-based selection, the paper's central claim would be materially weakened. The missing Appendix A table is a concrete reproducibility defect that must be fixed. I would not recommend rejection at this stage because the core idea is promising and the reported numbers may still be competitive under a fair protocol, but the current manuscript does not support the strong claims as written."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Base3 is a genuinely new, simple, deterministic ensemble of EdgeBank, PopTrack, and a new t-CoMem co-occurrence memory. It is training-free, interpretable, and on the reported TGB MRR it matches or beats several deep models. If those numbers are fairly obtained, that is a useful practical result and a good challenge to the field.\n\nThe paper is clearly written, and t-CoMem is a real contribution: it combines source-side recent neighborhood activity with decayed popularity and co-occurrence counts, addressing a limitation of PopTrack. The robustness experiment under inductive and historical sampling on tgbl-wiki-v2 is a good idea, and the determinism argument for single runs is legitimate.\n\nThe main soft spot is the tuning protocol. The default hyperparameters (span 0.1, co-occurrence weight 1.0, K=1000, multi_conf) are selected with test MRR on tgbl-wiki-v2 in view. In Table 3, for K=1000 the best validation MRR is 0.736 at span 1.0, but the paper picks span 0.1 because its test MRR is higher (0.743 vs 0.727). That is test-set selection, inconsistent with the TGB protocol used for the deep baselines. Along the same lines, the multi_conf weights are not fully specified: Appendix A describes four cases, but the table is missing (\"Table ??\") and the neither-signal case only says t-CoMem gets 'more importance' without exact values. The method as published cannot be reproduced exactly, and the margin over deep models on tgbl-coin (0.773 vs 0.762) could shrink under a properly validated configuration.\n\nTwo lesser issues. First, the negative-sampling robustness claim is shown only on tgbl-wiki-v2; one dataset is thin. Second, on the high-surprise datasets (review, comment) Base3 is clearly below the deep models, so the 'competitive' claim is dataset-dependent. None of this invalidates the central idea, but it changes how much weight I can put on the headline numbers.\n\nAll that said, this is a good empirical data point for anyone who wants a cheap, interpretable baseline for temporal link prediction, and it deserves a serious referee. The authors need to re-run the tuning on validation only, fill in the missing weights table, and ideally report the full weight vectors. If they do, it should be a solid workshop or short-conference paper.","headline":"Simple training-free ensemble that looks competitive with deep models on TGB, but the headline comparison rests on test-set-tuned hyperparameters and a missing weights table; needs a validation-based re-run before I'd trust the numbers.","tokens_in":13274,"tokens_out":2727,"would_cite":false,"duration_ms":26707,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free ensemble of three simple signals rivals deep temporal graph models on benchmark link prediction.","keywords":["dynamic link prediction","temporal graph learning","training-free baseline","ensemble interpolation","edge recurrence","node popularity","co-occurrence memory","Temporal Graph Benchmark"],"falsifier":"Re-run Base3 on the full TGB suite with the multi_conf weights shown (the paper's Table ?? in Appendix A is missing) and with per-dataset hyperparameter tuning; if the default configuration loses to a tuned configuration by a large margin on tgbl-review-v2, tgbl-comment, or tgbl-coin, then the reported competitive results do not reflect a generally robust configuration.","tokens_in":1520,"feed_emoji":"","tokens_out":1707,"duration_ms":58019,"temperature":0.7,"pith_summary":"This paper tries to show that complex neural architectures are not necessary for competitive dynamic link prediction on the Temporal Graph Benchmark. It builds a fully training-free, deterministic ensemble called Base3 that linearly combines three non-learnable signals: edge recurrence (EdgeBank), global node popularity (PopTrack), and a newly proposed temporal co-occurrence memory (t-CoMem). The author argues that this simple interpolation reaches state-of-the-art-level Mean Reciprocal Rank, even ranking first on tgbl-coin and close to the top on tgbl-wiki-v2 and tgbl-flight, while requiring no training, backpropagation, or learned parameters. If correct, the central claim matters because it shows that carefully chosen hand-crafted temporal signals can match or beat deep models on several realistic benchmarks, and that the field may be overvaluing architectural complexity.","feed_headline":"Zero-training ensemble rivals deep temporal graph models","feed_subtitle":"A fixed interpolation of recurrence, popularity, and co-occurrence scores matches deep networks on TGB link prediction.","key_machinery":"The load-bearing mechanism is the interpolation of three complementary non-learnable scorers. EdgeBank stores observed edges and assigns a score of 1 to any edge seen before. PopTrack assigns a binary score based on whether the destination node ranks in the top-$K$ most popular nodes by recent interaction frequency. The new t-CoMem module maintains, for each source node, a time-windowed deque of recently observed destinations; it scores a candidate pair by summing, over all recent neighbors $n_i$ of $u$, the decayed popularity $d \\cdot p_i$ with $d = \\exp(-(t-t_i)/t_w)$, then adds a co-occurrence term $f = \\lambda \\cdot c/(1+c)$, where $c$ is the observed co-occurrence count, and finally squashes the sum through $1/(1+1/(\\sum d\\cdot p_i + f))$. Base3 combines the three scores with heuristic weights chosen by a confidence scheme (multi_conf) that boosts EdgeBank when the edge is in memory and boosts PopTrack when the destination is popular. This machinery carries the argument because it converts two known baselines and one new memory module into a cheap, deterministic predictor whose only tuning is a few scalar hyperparameters.","core_discovery":"On the paper's own terms, Base3 is an interpolation model with scores $\\alpha \\cdot s_{\\text{EB}} + \\beta \\cdot s_{\\text{PT}} + \\delta \\cdot s_{\\text{CM}}$, where $s_{\\text{EB}}$ marks whether an edge appeared before, $s_{\\text{PT}}$ marks whether the destination is among the top-$K$ popular nodes, and $s_{\\text{CM}}$ is the new t-CoMem score that combines the source node's recently observed neighbors with decayed PopTrack popularity of those neighbors plus a co-occurrence count. The paper reports test MRR values of 0.743 on tgbl-wiki-v2, 0.108 on tgbl-review-v2, 0.773 on tgbl-coin, 0.450 on tgbl-comment, and 0.794 on tgbl-flight, with t-CoMem alone reaching 0.840 on tgbl-flight. These numbers put Base3 above TGN, GraphMixer, and TNCN on several datasets, and above DyGFormer on tgbl-coin. The claimed discovery is that a deterministic, interpretable combination of recurrence, popularity, and co-occurrence can rival modern deep temporal graph networks, and that the robustness is especially visible under inductive and historical negative sampling, where existing models degrade sharply.","pith_inferences":["The result is an existence proof that hand-designed temporal signals can saturate much of what deep models extract on these benchmarks; a natural extension is to test the same three-signal combination on other temporal graph suites beyond TGB.","The paper leaves implicit that benchmark leaderboards dominated by deep models may partly measure how well networks memorize recurring edges and track popularity, rather than how well they model complex temporal dependencies.","A testable extension is to replace the hand-set multi_conf weights with a small logistic regression or a per-dataset learned scalar, which could raise performance further while still keeping most of the interpretability.","The paper argues for efficiency but reports no wall-clock runtime; a direct head-to-head timing comparison against TGN-style models would make the deployment claim concrete."],"forward_implications":["If the reported numbers hold, training-free methods become the natural first baseline on the Temporal Graph Benchmark, since they are deterministic, fast, and require no hyperparameter learning.","On low-to-moderate surprise datasets such as tgbl-wiki-v2 and tgbl-coin, deep models' advantage over simple recurrence and popularity signals appears small or even negative, so those leaderboards may be conflating memorization with generalization.","The strong robustness of t-CoMem and Base3 under inductive and historical negative sampling suggests that soft popularity propagation is a more reliable inductive signal than EdgeBank's hard memorization or PopTrack's binary top-$K$ cutoff.","The modular design means each component can be reused as a cheap, interpretable building block inside larger learned systems, for example as a prior, an initialization, or a regularizer."],"supporting_citations":[{"why":"Supplies EdgeBank, the recurrence-based foundation that Base3 extends, and defines the inductive and historical negative sampling strategies used for robustness evaluation.","marker":"[16]"},{"why":"Supplies PopTrack, the popularity-based signal that provides inductive capabilities and the top-$K$ scoring logic used by Base3 and t-CoMem.","marker":"[4]"},{"why":"Provides the Temporal Graph Benchmark datasets and the MRR evaluation protocol on which all head-to-head results are reported.","marker":"[8]"},{"why":"Supplies DyGFormer, the strongest deep baseline on tgbl-wiki-v2 that Base3 trails and the model it beats on tgbl-coin.","marker":"[22]"},{"why":"Supplies TNCN, the deep baseline that narrowly edges Base3 on tgbl-coin and that t-CoMem beats on tgbl-flight.","marker":"[23]"},{"why":"Supplies TGN, the classic memory-based deep model used as a comparison baseline in the MRR table.","marker":"[17]"},{"why":"Supplies GraphMixer, the parameter-efficient deep baseline that Base3 outperforms on tgbl-wiki-v2 and tgbl-coin.","marker":"[2]"}],"fun_headline_variants":["Untrained ensemble matches deep temporal graph models","No-training score blend rivals learned link predictors","Interpolation method outdoes deep models in link prediction","Simple, interpretable ensemble challenges neural baselines","Zero-training trio robust under realistic negative sampling"],"cache_read_input_tokens":15360,"weakest_assumption_plain":"The default hyperparameters (memory span 0.1, co-occurrence weight 1.0, and $K=1000$), along with the weights of the multi_conf interpolation scheme, were fixed using ablation experiments on tgbl-wiki-v2, and the paper assumes these same choices remain near-optimal on the other four datasets.","fun_headline_variants_meta":{"raw":{"variants":["Untrained ensemble matches deep temporal graph models","No-training score blend rivals learned link predictors","Interpolation method outdoes deep models in link prediction","Simple, interpretable ensemble challenges neural baselines","Zero-training trio robust under realistic negative sampling"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000576,"raw_usage":{"total_tokens":3077,"prompt_tokens":1035,"completion_tokens":2042,"prompt_tokens_details":{"cached_tokens":1024},"prompt_cache_hit_tokens":1024,"prompt_cache_miss_tokens":11,"completion_tokens_details":{"reasoning_tokens":1973}},"tokens_in":11,"tokens_out":2042,"duration_ms":322722,"temperature":1.0,"reasoning_tokens":1973,"cache_read_input_tokens":1024,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:42:32.099813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run Base3 on the full TGB suite with the multi_conf weights shown (the paper's Table ?? in Appendix A is missing) and with per-dataset hyperparameter tuning; if the default configuration loses to a tuned configuration by a large margin on tgbl-review-v2, tgbl-comment, or tgbl-coin, then the reported competitive results do not reflect a generally robust configuration.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies PopTrack, the popularity-based signal that provides inductive capabilities and the top-$K$ scoring logic used by Base3 and t-CoMem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Temporal Graph Benchmark datasets and the MRR evaluation protocol on which all head-to-head results are reported."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies DyGFormer, the strongest deep baseline on tgbl-wiki-v2 that Base3 trails and the model it beats on tgbl-coin."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies TNCN, the deep baseline that narrowly edges Base3 on tgbl-coin and that t-CoMem beats on tgbl-flight."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies TGN, the classic memory-based deep model used as a comparison baseline in the MRR table."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies GraphMixer, the parameter-efficient deep baseline that Base3 outperforms on tgbl-wiki-v2 and tgbl-coin."}],"review_version":1}