{"id":"a20b0958-f0f2-4ba3-ab7b-ece66ccf7b01","arxiv_id":"2505.05568","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A single pretrained graph model for relational databases, Griffin unifies encoders and decoders and reports the best average rank across 24 tasks in its evaluation.","lead":"Griffin is a graph-based neural network trained once on many tabular and relational datasets, then fine-tuned for new database tasks. It reports better average rankings than task-specific models on two relational database benchmarks, especially when training data is scarce.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Regression comparisons in RelBench are computed on quantile-normalized targets while some baselines come from original raw-scale reports, so the headline average-rank claim rests on an incomparable metric.","rationale":"The reader's weakest_assumption identifies the same confound, and I agree. The paper does provide raw tables and a code link, which is real evidence, but the central comparison's validity hinges on a single testable condition: all models must be scored on the same target scale under the same pipeline. The paper's own Appendix B shows this condition is not met, both by normalizing RelBench targets and by sourcing SAGE numbers from the original report rather than re-evaluating them. My concrete check would settle whether the regression gains survive an apples-to-apples comparison. I do not see an internally inconsistent derivation or a formal proof flaw beyond this empirical confound; the architecture story is plausible and the raw tables are useful. Therefore I do not move the verdict: CONDITIONAL remains appropriate until the aligned-pipeline numbers are re-reported on raw targets and SAGE is re-run in the same pipeline.","tokens_in":27238,"tokens_out":7670,"duration_ms":80236,"concrete_test":"For the five RelBench regression tasks in Table 3, re-run every model (Griffin-unpretrained, Griffin-pretrained, SAGE, GAT, PNA, HGT, DFS variants) under one identical pipeline with raw-scale targets: either apply the inverse quantile transform to Griffin's normalized predictions and compare against original RelBench raw-scale metrics, or retrain all models directly on raw targets with the same preprocessing, fanout, and hyperparameters. Also re-run SAGE through the same 4DBInfer pipeline instead of taking its numbers from the original report. If Griffin-unpretrained no longer has the best average rank on regression tasks, or if the Table 3 shift is not explained as target normalization, the headline claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that Griffin-unpretrained and every baseline are scored on the same target scale under the same preprocessing pipeline. Section B.2 breaks this condition: RelBench regression tasks are redefined by quantile normalization, with Table 3 showing 'RelBench Original Results' versus 'Aligned to Griffin Results' shifting rel-trial/study-adverse from 44.473 to 2.199. Section B.3 then says SAGE results are taken from the original RelBench report while other baselines were re-evaluated through the 4DBInfer pipeline. If SAGE's raw-scale values are compared with Griffin's normalized-scale values, Table 4's regression wins (e.g., study-adverse 1.17 vs 1.5251) are scale artifacts, not evidence of superiority. If Table 4's SAGE row is itself normalized, then the text contradicts the table and the reader cannot verify which numbers are comparable. Because these regression tasks contribute to the average rank in Figure 2, this single confound threatens the headline 'outperforms all other models in average rank.' The absence of error bars or seed variance in Table 4 makes the rank comparison additionally fragile.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Griffin, a graph-centric foundation model for relational databases. An RDB is converted into a heterogeneous temporal graph with rows as nodes and PK-FK relationships as edges. Griffin encodes cells with a frozen text encoder (Nomic) for categorical/text values and a pretrained float encoder for numerical values, augments nodes and edges with metadata embeddings, injects a task embedding derived from the target column name, and processes the sampled rooted subgraph with an MPNN that combines a cross-attention module and hierarchical (per-relation mean, cross-relation max) aggregation. A shared decoder handles classification via inner products with label text embeddings and regression via a pretrained number decoder. Training is multi-stage: completion pretraining on roughly 200 single-table datasets, joint supervised fine-tuning on single-table and/or RDB datasets, and per-task fine-tuning. Experiments span 24 tasks from 4DBInfer and RelBench (over 150 million nodes), and the paper reports three main findings: (Q1) Griffin-unpretrained achieves the best average rank against four GNN and four DFS-based baselines; (Q2) single-table-only pretraining improves average rank; (Q3) transfer benefits correlate with the similarity and diversity of the SFT domain. Code is released at github.com/yanxwb/Griffin.","tokens_in":27406,"tokens_out":11994,"duration_ms":106730,"significance":"If the main claims held as stated, the paper would constitute a meaningful step toward relational-database foundation models: a single pretrained architecture with unified encoders and decoders could replace task-specific GNNs on RDB prediction tasks, and the single-table-to-RDB multi-stage pretraining recipe would be a reusable template. Strengths of the manuscript include the release of code; a large pretraining corpus (over 150 million nodes) assembled from public benchmarks; a self-contained, testable architecture consisting of cross-attention over cells and hierarchical aggregation; the coherence of the pretraining objective in Eq. (6), which uses a fixed text encoder as a target rather than a fitted circular target; and the transferability study, which averages over five random split selections. Two load-bearing issues prevent acceptance of the strongest claims at present: the regression metrics in Table 4 appear to mix incompatible target scales (quantile-normalized versus original raw scale), and the 'only architecture' comparison is confounded by Griffin's different input encoders.","major_comments":[{"comment":"The regression comparisons that feed the headline average-rank claim are computed on incompatible target scales. Section B.2 states that RelBench regression tasks were redefined via quantile normalization, and Table 3 shows the effect: the rel-trial/study-adverse MAE changes from 44.473 (RelBench original) to 2.199 (aligned), a change of roughly an order of magnitude. Section B.3 then states that for the RelBench datasets, 'the Sage results are taken from the original report' while all other baselines were re-evaluated through the 4DBInfer pipeline. Consequently, in Table 4's regression block Griffin's MAE values (normalized scale) are compared against SAGE values that, if the text is literal, are on the original raw scale; the regression wins (e.g., rel-trial/study-adverse 1.1700 vs. 1.5251 for SAGE) would then be scale artifacts rather than evidence of superiority. The numbers are also internally inconsistent: Table 3 reports both 44.473 and 2.199 for the same task, while Table 4 reports 1.5251 for SAGE, matching neither. Because the MAE/RMSE tasks contribute to the average rank in Figure 2 and to the Section 6.2 claim that 'Griffin-unpretrained outperforms all other models in average rank,' the comparability of the regression metrics must be established before the main claim can be accepted. Please re-evaluate every baseline, including SAGE, under a single explicitly stated target scale (or report both scales for both Griffin and all baselines).","section":"§B.2–B.3, Tables 3–4, Figure 2"},{"comment":"The claim that Griffin-unpretrained 'differs from other GNN baselines only in its architectural design' is confounded by the input encoders. Griffin encodes categorical and text cells with Nomic sentence embeddings, encodes numerical cells with a pretrained float encoder, and additionally injects metadata embeddings (table names, column names, edge types) and a task embedding; the GNN baselines SAGE/GAT/PNA/HGT use one-hot categorical encoding and GloVe text embeddings (Table 2). The Q1 comparison therefore varies architecture and input representation simultaneously, so the superior average rank in Figure 2 cannot be attributed to the cross-attention and hierarchical-aggregation design alone. To support the 'only architecture' claim, add an ablation in which the standard MPNN backbones are given Griffin's encoders (or, equivalently, Griffin's MPNN is given the baselines' one-hot inputs), and qualify the claim in Section 4.3 accordingly.","section":"§4.3, §B.1 (Table 2)"},{"comment":"The main results and the ablation are reported without variance or the number of seeds, despite several very small margins: for example, Seznam/charge accuracy is 0.7998 for Griffin-unpretrained versus 0.8053 for GAT, Outbrain/ctr ROC-AUC is 0.6246 versus 0.6239 for SAGE, and in the ablation Virus/wnv is 0.6680 for Griffin versus 0.6648 for Griffin-mean-GNN. The statements in Section 6.2 that Griffin 'demonstrates significant improvements' and that removing the components 'results in a significant performance drop' are not substantiated without repeated runs, error bars, or a significance test; the transfer study in Figure 4 uses five seeds for split selection, so the infrastructure for this exists. Please report means and standard deviations (or per-seed values) over at least three seeds for the main tables, or weaken the 'significant' claims proportionately.","section":"§6.2, Tables 4–5"},{"comment":"The Q2 claim that 'Griffin-Pretrained outperforms its non-pretrained counterpart, validating the universal benefits of pretraining' is contradicted by Table 4 on roughly one third of the tasks: pretraining degrades results on Rel-avito/user-clicks (0.6330 vs. 0.6639), Rel-f1/DNF (0.7091 vs. 0.7052), Rel-f1/top3 (0.7795 vs. 0.7855), Rel-hm/user-churn (0.6804 vs. 0.6847), Rel-f1/position (0.5694 vs. 0.5586), Rel-hm/item-sales (0.8962 vs. 0.8879), Rel-trial/site-success (0.7945 vs. 0.7926), and Rel-trial/study-adverse (1.2148 vs. 1.1700). The average-rank improvement (3.042 vs. 3.708) is real but is not 'universal'; the text should be reworded to claim a net average improvement, and the tasks where pretraining hurts should be discussed, particularly since four of the eight are regression tasks affected by the scale-alignment issue raised above.","section":"§6.3, Table 4"}],"minor_comments":[{"comment":"The phrase 'foundation model attemptation' appears in the abstract and again as 'Graph-centric RelatIonal databaseFoundatIoNmodel attemptation' in Section 1; this is awkward and should be replaced with a standard term such as 'effort' or 'approach.'","section":"Abstract and §1"},{"comment":"The caption's final sentence, 'All values are positive; higher values indicate better performance for Accuracy and ROC-AUC, while lower values are better for left ones,' is garbled and should state the direction of better performance for each panel explicitly.","section":"Figure 2 caption"},{"comment":"Table 5 reports MAE and Logloss values with negative signs (e.g., -0.6593, -0.5684) without explanation; the sign convention (apparently negating losses so that higher is better uniformly) should be stated in the caption.","section":"Table 5"},{"comment":"The row label 'Aligned to Griffin Results' is ambiguous about which model produced those numbers; specify the model and state explicitly that the alignment changes the target scale, not merely preprocessing details, so that readers can track which Table 4 entries are comparable.","section":"§B.2, Table 3"},{"comment":"The sentence 'For transfering between Others-1 and Others-2' contains a typo, and the one-directional transfer benefit (Others-1 to Others-2 but not vice versa) is asserted from figures without reported variance; either add error bars to the transfer figures or soften the asymmetry claim.","section":"§6.4"},{"comment":"The selection of the 50 labeled single-table datasets used for joint SFT and the assignment of tasks into the Commerce-1/2 and Others-1/2 groups are described qualitatively; provide the full task-to-group mapping and the dataset selection criterion in the appendix to support replication.","section":"§6.1 and §A"}],"recommendation":"major_revision","confidential_remarks":"The key request for the revision is to resolve the provenance of the regression numbers: Table 3 reports 44.473 and 2.199 for rel-trial/study-adverse while Table 4 reports 1.5251 for SAGE on the same task, and the text says SAGE results were taken from the original report; these numbers cannot all be on the same scale. I would ask the authors to re-run all baselines, including SAGE, under the aligned pipeline and to report per-seed results for the main tables. In addition, the abstract's 'first foundation model attemptation' phrasing and the 'universal' pretraining claim should be toned down to match the evidence in Table 4. The architectural contributions are plausible and the ablation is directionally supportive, so I see this as a fixable revision rather than a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, this is the first serious attempt at a graph-centric foundation model for relational databases, and it ships code plus a large appendix of raw numbers. Second, the headline claim that Griffin outperforms all baselines in average rank is undercut by a metric alignment problem that the paper itself documents.\n\nWhat is actually new and good: the unified encoder using pretrained text and float encoders, cross-attention over row cells, hierarchical relation aggregation, and multi-stage pretraining with a single checkpoint across tasks. That combination is, as far as I know, new for RDBs. The architecture ablation showing the benefit of cross-attention and max aggregation is clean and credible. The pretraining objective uses a fixed text encoder as both target and classification head, which is coherent rather than circular, and no fitted constants are fed back into evaluation. Appendix F provides raw results for every figure, which is real and useful.\n\nThe soft spots are real. Appendix B.2 redefines RelBench regression targets by quantile normalization; Table 3 shows rel-trial/study-adverse original MAE of 44.473 becoming 2.199. Then B.3 says SAGE results are taken from the original RelBench report while other baselines were re-evaluated through the 4DBInfer pipeline. But Table 4 lists SAGE study-adverse as 1.5251, which is neither the original raw value nor the aligned value of 2.199 shown in Table 3. That is an internal contradiction. If SAGE was actually re-evaluated, the text is wrong; if not, the comparison mixes raw-scale and normalized-scale numbers. Either way, the regression rows that feed the average rank in Figure 2 are not verifiable as reported. No error bars or seed variance are given for the main results, and the transfer experiments use five seeds only for the split, not for training runs. Griffin's encoders also differ from the baselines (Nomic text embeddings and a pretrained float encoder versus one-hot and GloVe), so the 'only architecture' interpretation of Griffin-unpretrained is confounded with input encoding.\n\nThese are fixable problems, not fatal ones. The transfer study between commerce and non-commerce domains is suggestive but secondary, and the few-shot comparison to TabPFNv2 is a useful addition.\n\nWho is this for: researchers working on relational deep learning and tabular foundation models. The architecture and benchmark integration are worth refereeing carefully. A serious referee should ask for the same-encoder baselines, same-normalization comparisons, standard deviations across seeds, and a clear statement of which scale every regression number is on. If those come back, the central claim may well survive; as it stands, it is conditional.","headline":"A credible and substantial first RDB foundation model attempt whose headline average-rank claim is undercut by an internal regression-metric alignment inconsistency and missing error bars.","tokens_in":27974,"tokens_out":2726,"would_cite":true,"duration_ms":32205,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Griffin, a graph-centric foundation model, unifies encoders and decoders so one pretrained checkpoint can handle diverse relational database tasks.","keywords":["relational databases","graph neural networks","foundation models","tabular data","pretraining","transfer learning","heterogeneous graphs","few-shot learning"],"falsifier":"Rerun all RelBench regression tasks with the original raw-scale targets and the original preprocessing for every baseline, denormalizing Griffin's outputs before computing MAE; if Griffin no longer holds the best average rank, the 'superior or comparable' claim depends on the normalized metric.","tokens_in":27011,"feed_emoji":"🗄️","tokens_out":7680,"duration_ms":70961,"temperature":0.7,"pith_summary":"This paper aims to establish that one pretrained architecture can serve as a foundation model for relational databases (RDBs), replacing the current practice of training a separate graph neural network for each database task. Griffin converts each RDB into a heterogeneous graph whose nodes are rows and whose edges are primary-key/foreign-key links, then uses a unified text encoder for categorical and metadata cells, a pretrained float encoder for numbers, and a task embedding derived from the target column name. The paper evaluates Griffin on 24 tasks from two benchmarks covering over 150 million nodes. It reports that the architecture alone beats task-specific GNN and deep-feature-synthesis baselines in average rank, that pretraining on single tables alone improves performance further, and that supervised fine-tuning on similar or diverse relational datasets gives the largest gains in low-data settings. If these claims hold, a single checkpoint could generalize across schemas, tasks, and domains that currently demand bespoke models.","feed_headline":"One pretrained model bests task-specific GNNs on database tasks","feed_subtitle":"A single checkpoint transfers across schemas and tasks, and helps most when labeled data is scarce.","key_machinery":"The mechanism is Griffin's trio of unified components. A unified feature encoder sends every categorical, textual, and metadata value through the same pretrained text encoder, applies quantile normalization plus a pretrained float encoder/decoder pair to numerical values, and creates a task embedding by encoding the target column's name. The MPNN then alternates cross-attention, in which the node state queries column-name keys and cell-value values, with hierarchical aggregation that averages neighbors within each relation type and takes the max across types. Finally, a unified task decoder predicts classification labels by dot-product similarity to text-embedded label names, and regression targets through the pretrained number decoder.","core_discovery":"The central discovery, stated on the paper's own terms, is that relational database prediction can be unified under one graph-centric model without losing to task-specific models. The paper's key evidence is that Griffin-unpretrained, the architecture without any pretraining, already has the best average rank across all compared models, and that Griffin-pretrained improves on it. Pretraining is done in two stages: completion pretraining on over 200 single-table datasets with a masked-cell cosine objective, and joint supervised fine-tuning on selected single-table and RDB tasks. Downstream, Griffin shows strong transferability when the supervised fine-tuning domain is similar to the target or more diverse than it, with the largest benefits in 512-sample few-shot settings.","pith_inferences":["Editorial inference: because completion pretraining needs no labels, scaling the pretraining corpus with additional unlabeled relational databases should improve generalization further, following the same logic as masked language modeling.","Editorial inference: the observed collapse of first-layer cross-attention to mean aggregation suggests a general design rule for column-graph models: run self-attention over cells before conditioning queries on the task.","Editorial inference: before deploying Griffin on regression tasks, a practitioner should calibrate the denormalizer per column, because the reported normalized-scale errors are not directly comparable to raw-scale business metrics."],"forward_implications":["A single Griffin checkpoint pretrained only on single-table datasets can transfer to relational database tasks, so foundation-model pretraining does not require a matching target RDB.","The architecture alone, without any pretraining, already improves average rank over task-specific GNN and deep-feature-synthesis baselines, so the reported gains are not solely an artifact of pretraining.","Joint supervised fine-tuning on relational datasets that are either similar to the downstream domain or more diverse than it produces the largest transfer gains, with the strongest effect in low-data settings with only hundreds of training samples.","Because classification uses text-embedded label names and regression shares a single float decoder, the same model can be pointed at a new task by changing only the task embedding and the label embeddings."],"supporting_citations":[{"why":"Supplies the 4DBInfer benchmark, its RDB datasets, and the quantile-normalization pipeline that Griffin aligns to.","marker":"(Wang et al., 2024)"},{"why":"Supplies the RelBench benchmark and RDB datasets, including the original regression numbers that the paper re-aligns.","marker":"(Robinson et al., 2024)"},{"why":"Provides the quantile normalization method used on numerical features and regression targets.","marker":"(Bolstad et al., 2003)"},{"why":"Provides the pretrained text encoder that maps categorical, textual, metadata, and task-embedding inputs into one space.","marker":"(Nussbaum et al., 2024)"},{"why":"Defines the DFS method used to construct the single-table baseline pipelines the paper compares against.","marker":"(Kanter & Veeramachaneni, 2015)"},{"why":"One source of the single-table pretraining datasets, contributing tens of thousands of rows to the completion pretraining corpus.","marker":"(Yan et al., 2024)"},{"why":"The other source of single-table pretraining datasets used during completion pretraining.","marker":"(Kim et al., 2024)"},{"why":"Provides TabPFNv2, the few-shot single-table foundation model that Griffin is compared against in low-data settings.","marker":"(Hollmann et al., 2025)"},{"why":"Supplies the similarity and diversity transferability hypotheses that the RDB supervised fine-tuning experiments test.","marker":"(Ehrig et al., 2024)"}],"fun_headline_variants":["Graph-centric foundation model unifies relational DB tasks","One pretrained model transfers across database schemas and tasks","First foundation model for relational databases also helps in few-shot","Database pretraining that boosts low-data prediction on graphs","Griffin: a single model for diverse relational database tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central comparison assumes that predicting quantile-normalized target values and reporting those normalized errors is the right yardstick for regression, even though the paper's own alignment table shows original RelBench regression errors on a very different scale, for example study-adverse MAE of 44.473 versus 2.199 after alignment.","fun_headline_variants_meta":{"raw":{"variants":["Graph-centric foundation model unifies relational DB tasks","One pretrained model transfers across database schemas and tasks","First foundation model for relational databases also helps in few-shot","Database pretraining that boosts low-data prediction on graphs","Griffin: a single model for diverse relational database tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000764,"raw_usage":{"total_tokens":3352,"prompt_tokens":872,"completion_tokens":2480,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":2401}},"tokens_in":488,"tokens_out":2480,"duration_ms":16016,"temperature":1.0,"reasoning_tokens":2401,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:02:16.395601+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun all RelBench regression tasks with the original raw-scale targets and the original preprocessing for every baseline, denormalizing Griffin's outputs before computing MAE; if Griffin no longer holds the best average rank, the 'superior or comparable' claim depends on the normalized metric.","supporting_citations":[{"cited_title":"The detailed information is shown in Table","cited_arxiv_id":null,"evidence_quote":"Supplies the RelBench benchmark and RDB datasets, including the original regression numbers that the paper re-aligns."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the DFS method used to construct the single-table baseline pipelines the paper compares against."}],"review_version":1}