{"id":"f3b97fec-8550-425f-a3bf-94542cac9d74","arxiv_id":"2411.11375","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Training a GNN by querying a graph database for neighbor samples and features reduces memory use enough to train on small machines, but is much slower than in-memory training.","lead":"This paper trains graph neural networks by keeping the whole graph in a graph database and pulling only small random samples of neighbors into memory during training. It shows this uses far less memory than the usual approach, at the cost of slower training, which matters for very large graphs that cannot fit in RAM.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 8 GB single-machine result does not account for the Neo4j server's configured 32 GB heap and 32 GB page cache, so the headline memory advantage is unverified; the Figure 4 global LIMIT also changes GraphSAGE sampling semantics.","rationale":"I read the paper as a preliminary systems contribution: it demonstrates that a graph database query engine can serve as the sampling and feature-retrieval backend for a PyG-style training loop, and it reports execution-plan evidence (Table 1) and low-RAM rows (Table 2) that are suggestive. The query-plan analysis and the running code are real supporting evidence. However, the strongest claim in the abstract and Section 5.2 is the resource advantage: training ogbn-papers100M on a machine with 8 GB RAM. That claim depends on how 'RAM' is counted. The paper's own Appendix C specifies 32 GB heap and 32 GB page cache for Neo4j; if these are on the same machine, the 8 GB row is impossible, and if they are on a different machine, the single-machine framing is misleading. This is not a stylistic issue but a missing accounting of the system's total memory footprint. The reader's identified weakness about the global LIMIT sampler is also real and independently important: Figure 4 uses one ORDER BY rand() LIMIT over all seed nodes rather than per-node, per-layer sampling, and no accuracy comparison against the standard sampler is provided. I therefore partially agree with the reader, but I would elevate the memory-accounting issue because it directly undermines the headline result. Since both issues are addressable by additional experiments, I would not reject the paper outright, but I would mark the central claim as unverified until the full-system memory measurement and the standard-sampler accuracy comparison are supplied.","tokens_in":11996,"tokens_out":7858,"duration_ms":98405,"concrete_test":"Rerun the Table 2 'Disk-based' 8 GB row on a single machine while monitoring total system memory with a cgroup or /usr/bin/time -v peak RSS that includes the Neo4j JVM, the Neo4j page cache, and the PyG training process. If the peak total exceeds 8 GB, or if the machine cannot start Neo4j with the Appendix C heap and page cache settings, the single-machine memory claim fails. Separately, run the same 2-layer GraphSAGE configuration on ogbn-products for 10 epochs with the standard PyG NeighborSampler and with the Figure 4 query sampler, using identical fanout, and compare test accuracy and the fraction of seed nodes receiving zero sampled neighbors.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim is that training can proceed while materializing only minimal data into memory, enabling ogbn-papers100M training on a machine with 8 GB RAM. The load-bearing condition is that the reported RAM figure covers the whole training system, including the graph database. Table 2 reports a 'Disk-based' backend running at 8 GB, but Appendix C configures the Neo4j server with dbms.memory.heap.initial_size = 32GB, dbms.memory.heap.max_size = 32GB, and dbms.memory.pagecache.size = 32GB. If Neo4j runs on the same machine as the training process, the 8 GB row is physically inconsistent with a 32 GB JVM heap plus a 32 GB page cache. If instead Neo4j runs on a separate server, the experiment is not single-machine and the memory of the database node is excluded from the RAM column. Either way, the paper's signature resource claim is not established by the reported experiments. A second, independent weakness is the sampler in Figure 4: ORDER BY rand() LIMIT $MAX_NEIGHBOURS is applied once to the combined result set across all seed nodes, which is not the per-node, per-layer sampling defined by GraphSAGE in Equation 2. The only validation, Appendix D, checks the marginal degree distribution of sampled nodes, not per-seed coverage or downstream model accuracy, and Section 5.3 reports only an unbaselined 58% accuracy after 10 epochs. The memory accounting is the more direct threat to the headline claim, but both need to be resolved before the paper can be read as demonstrating what it claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes training GNNs by offloading neighbourhood sampling and feature retrieval to a graph database query engine, so that only minimal metadata and per-batch query results are materialized in the training process. The method is implemented with PyG and Neo4j, and evaluated on ogbn-papers100M and ogbn-products in single-machine and distributed settings. The main reported results are that single-machine training can proceed with as little as 8 GB RAM for ogbn-papers100M, and that distributed training scales roughly linearly up to 8 processes while keeping the database as a central store. The paper also includes a query-plan analysis and an empirical degree-distribution check for the sampling query.","tokens_in":12348,"tokens_out":2086,"duration_ms":22305,"significance":"If the memory-scaling claim is valid, the paper opens a practical new direction for training GNNs on graphs that do not fit in RAM, and it builds a bridge between graph database systems and graph ML. The work is accompanied by released code, a query-plan verification in Section 5.1, and distributed experiments that go beyond a simple feasibility study. However, the significance is currently constrained by the unresolved memory accounting and by the lack of a model-accuracy baseline, both of which are central to the paper's claims.","major_comments":[{"comment":"The headline claim that training ogbn-papers100M is possible on a machine with 8 GB RAM is not established, because the memory column appears to exclude the graph database server. Appendix C states that Neo4j is configured with dbms.memory.heap.initial_size = 32GB, dbms.memory.heap.max_size = 32GB, and dbms.memory.pagecache.size = 32GB. If Neo4j runs on the same machine as the training process, the 8 GB figure is incompatible with a 32 GB JVM heap plus a 32 GB page cache; if Neo4j runs on a separate server, the experiment is not single-machine and the database node's memory is omitted from the RAM column. The paper must either report actual end-to-end peak memory for the whole system or clearly state that the RAM column refers only to the training process, not the database server.","section":"Section 5.2 / Table 2 / Appendix C"},{"comment":"The sampling query in Figure 4 applies ORDER BY rand() LIMIT $MAX_NEIGHBOURS to the combined result set across all seed nodes, which is not the per-node, per-layer uniform sampling defined by GraphSAGE in Equation (2). This changes the number of neighbours per seed node and the layer-wise sampling semantics, since the global limit may exhaust the budget on a few high-degree seeds and leave others with fewer or no sampled neighbours. The paper acknowledges this optimization in Section 4 but validates it only with the marginal degree-distribution check in Appendix D, which does not test per-seed coverage or downstream model behaviour. To support the claim that the method trains the stated model, the authors should compare the sampling distribution (or, more directly, model accuracy) against the standard per-node GraphSAGE sampler, or explicitly reframe the method as training a different, database-sampled model.","section":"Section 4 / Figure 4 / Equation (2)"},{"comment":"The reported accuracy of 58% after 10 epochs is not compared against any baseline, so it cannot support the claim that the proposed pipeline trains a functioning GraphSAGE model. The text says 'our model gives an average of 58% accuracy' but gives no standard GraphSAGE accuracy on the same task, data split, and hyperparameters, and no indication of whether 58% is typical for this dataset after 10 epochs with the same architecture. Without such a comparison, the reader cannot distinguish between a correct implementation with expected accuracy and a pipeline whose sampling or feature retrieval corrupts the training signal.","section":"Section 5.3"}],"minor_comments":[{"comment":"There are small notation errors in Equation (1): the initial condition refers to 'x(0)_v in R^{1*n} for u in V' and the neighbourhood is described as 'some neighbourhood of u', where the intended variable is v. Please correct these.","section":"Section 2.1 / Equation (1)"},{"comment":"The statement that the approach 'should enable graph ML workloads for extremely large graphs, such as Graph500-scale34' is speculative and not supported by any experiment in the paper. It should be marked as future work or removed.","section":"Section 5.2"},{"comment":"The empirical sampling-distribution check is performed on ogbn-products, not on ogbn-papers100M, and it reports only the marginal frequency of sampled nodes. It would be more informative to also report per-seed neighbourhood sizes and compare them with the intended per-node limit, especially since the global LIMIT is the main source of divergence from GraphSAGE.","section":"Appendix D"},{"comment":"The query returns node_1.id, node_1.features, node_2.id, node_2.features, but the text does not explain how the returned two-hop neighbours are associated with their seed nodes in the training loop. Clarify the mapping from src_id to each hop's neighbours.","section":"Section 4 / Figure 4"},{"comment":"The table does not state which dataset is used for the single-machine results; from context it appears to be ogbn-papers100M, but this should be explicit in the caption or text.","section":"Section 5 / Table 2"},{"comment":"The configuration lists Neo4j, Kùzu, PyTorch, and PyG versions, but does not give the exact dataset split, evaluation metric, or model hyperparameters (e.g., hidden dimension, dropout, learning rate) used for the 58% accuracy figure in Section 5.3. Please add these details for reproducibility.","section":"Appendix C"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and useful integration of graph DBs with GNN training, and the code release is a positive feature. The two main issues in my report — the ambiguous memory accounting and the unvalidated sampling-semantics change — are both fixable with additional experiments or clarifications, but they currently block the paper's central claims. I would also suggest that the authors consider whether the '8 GB RAM' row should be reframed as 'training-process memory' if the database runs on a separate machine, since the current presentation overstates the single-machine memory advantage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely new systems idea — train a GNN by issuing neighbour sampling and feature retrieval as Cypher queries against a server-side graph database, never materialising the full graph in the training process. The paper is worth engaging with, but the headline memory claim needs a serious fix before the results can be taken at face value.\n\nThe core contribution is real. No prior work I know executes sampling inside a graph DB query engine; Kùzu loads the graph into an embedded DB and exposes it as PyG stores, but doesn't push sampling into the query layer. The query template in Figure 4 and the accompanying PROFILE analysis are a reasonable first engineering step, and the code is on GitHub. The observation that graph DBs already solve concurrent-read and partitioning problems is a fair one.\n\nThe soft spots are substantial and the stress-test note is right on both counts. First, the 8GB single-machine claim in Table 2 is inconsistent with Appendix C, which sets Neo4j's heap to 32GB and page cache to 32GB. If Neo4j is on the same box, 8GB of RAM cannot be the whole system; if it's on another server, then the experiment isn't single-machine and the database node's memory is excluded. Either way, the paper's signature resource claim — training a GNN on a machine with 8GB RAM — is not established by the reported setup. This needs to be clarified and re-measured with the database's memory footprint counted.\n\nSecond, the sampling in Figure 4 applies ORDER BY rand() LIMIT $MAX_NEIGHBOURS once to the combined multi-hop result, not per node per layer as GraphSAGE defines. The paper acknowledges this as an optimisation, but then uses it in the experiments and reports 58% accuracy after 10 epochs with no comparison to a standard GraphSAGE baseline. The degree-distribution check in Appendix D does not show that the model still learns the intended objective. So the method as evaluated is not exactly GraphSAGE, and we have no accuracy baseline to tell whether the approximation hurts.\n\nA minor point: the distributed experiment simulates multiple processes on one machine rather than real distributed nodes, so the 'almost free' distributed-training claim is only about concurrent reads, not network-scale scaling.\n\nWho this is for: people working on scaling GNNs with database backends, or on integrating graph ML with DBMS. It's a preliminary systems paper, not a definitive study. I'd send it to a serious referee — it deserves a careful look at the memory accounting and sampling semantics — but it should not be accepted as is. The idea is worth airing.","headline":"A genuinely new systems idea for GNN training via graph-DB queries, but the headline memory claim is undercut by the Neo4j configuration in the paper's own appendix.","tokens_in":12858,"tokens_out":3214,"would_cite":false,"duration_ms":30050,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A graph neural network can be trained by letting the graph database do neighbour sampling and feature retrieval, so the graph never has to be loaded into memory.","keywords":["graph neural networks","graph databases","neighbourhood sampling","Cypher queries","memory-efficient training","GraphSAGE","distributed training","graph partitioning"],"falsifier":"Run the same GraphSAGE model, with identical hyperparameters and seeds, on ogbn-products using the paper's database sampler and using the standard in-memory per-layer sampler; if test accuracy differs by more than run-to-run noise, the final-hop-only random LIMIT has changed the model and the claim that database queries reproduce GraphSAGE sampling fails.","tokens_in":11816,"feed_emoji":"🗄️","tokens_out":7428,"duration_ms":68174,"temperature":0.7,"pith_summary":"Training a graph neural network usually begins by loading the whole graph into memory, which caps model size by RAM. This paper tries to remove that cap: it proposes expressing neighbour sampling and feature retrieval as graph-database queries, so the training process materialises only the sampled batch of nodes and features. If the approach is right, GNN training becomes possible on machines with far less memory than the graph, and graph databases take on the role of graph store, feature store, and sampler in one. The authors support this with experiments on two large benchmarks, including successful training on a machine with 8 GB of RAM and roughly linear speedup in distributed training up to CPU saturation.","feed_headline":"Database-side sampling trains large-graph GNNs with just 8 GB of RAM","feed_subtitle":"Moving neighbour sampling and feature retrieval into graph queries keeps the graph out of RAM.","key_machinery":"The central mechanism is a graph-query template: given a batch of seed nodes, it performs a multi-hop pattern match, applies ORDER BY rand() followed by LIMIT $MAX_NEIGHBOURS to cap only the final sampled neighbourhood, and returns sampled node IDs together with their features in a single round-trip. The paper adapts standard GNN library interfaces so the database supplies both graph structure and features, replacing the usual in-memory graph representation and separate feature store. Limiting only the final hop, rather than each hop, is the optimisation that keeps intermediate materialisation small; it changes the sampling distribution relative to GraphSAGE's per-layer node-wise sampling, and the paper checks only the empirical degree distribution of sampled nodes as validation.","core_discovery":"The claim, stated in the authors' own terms, is that the data-access steps of mini-batch GNN training — initial metadata loading, multi-hop neighbour sampling, and feature retrieval — can be offloaded to a graph database, leaving only the minimal per-batch data in memory. The paper implements this with a query template that chains two-hop pattern matches, orders the result by a random value, and applies LIMIT to cap the final neighbourhood, while fetching node features in the same query. Empirically the setup trains GraphSAGE on ogbn-papers100M with 8 GB of RAM and under 1 KB of RAM at initialisation instead of 48 GB, and scales across multiple training processes reading from one database. The authors present this as a new way to scale GNNs and a new workload for graph databases.","pith_inferences":["The paper validates the sampler only by the shape of the degree distribution, not by comparing model accuracy against the standard GraphSAGE sampler; the strongest version of the claim would require an accuracy-parity test.","Because the random cap applies only to the final hop, the joint distribution over sampled multi-hop neighbourhoods differs from per-layer node-wise sampling, which could matter on graphs with highly skewed degrees.","A natural extension is to push the mean aggregation of the GraphSAGE update into the query engine as well, since mean is a basic aggregation operator in graph databases and the paper notes this direction.","The same query formulation could make single-node inference cheap, because classifying a new node would require only a local neighbourhood query rather than reconstructing the full graph."],"forward_implications":["GNN training becomes possible on machines whose RAM is far smaller than the graph, because memory use is bounded by the sampled neighbourhood of one batch plus model weights rather than by the full graph.","The separate graph-partitioning step for distributed training can be replaced by the database's horizontal scaling and concurrent read support, removing a preprocessing cost and a reproducibility concern.","Feature storage can live with the graph in the same database, eliminating a separate feature store and the consistency maintenance between stores.","Training throughput scales smoothly with hardware: more RAM or more processes reduces per-epoch time until the CPUs are saturated, as observed with eight processes in the distributed setup.","Graph databases gain a concrete machine-learning workload, since sampling-as-a-query is a read pattern distinct from traditional transactional and analytical database workloads."],"supporting_citations":[{"why":"Defines GraphSAGE and the node-wise per-layer neighbourhood sampling scheme that the paper's query must reproduce.","marker":"[2]"},{"why":"Provides the GraphStore and FeatureStore interfaces that the paper adapts so a graph database can serve as both graph structure and feature source.","marker":"[7]"},{"why":"Supplies the ogbn-papers100M and ogbn-products datasets that anchor the memory and scaling experiments.","marker":"[18]"},{"why":"Documents native graph storage with node attributes co-located, supporting the paper's claim that features can be retrieved efficiently with node IDs.","marker":"[24]"},{"why":"Describes an earlier integration of a graph database with PyG that loads the graph into memory, used as the comparison baseline for embedded databases.","marker":"[30]"},{"why":"Identifies the embedded graph database system used as the 'Embedded' backend in the single-machine experiments.","marker":"[47]"},{"why":"Describes the distributed data-parallel training setup whose architecture the paper's multi-process graph-database training mimics.","marker":"[6]"}],"fun_headline_variants":["Train large-graph GNNs with 8GB RAM using graph queries","Offload neighbor sampling to the database to train GNNs on huge graphs","Graph DB as the sampling engine: train GNNs on 100M-node graphs with 8GB RAM","Query engine does the heavy lifting: GNN training on 8GB RAM for huge graphs","Minimal RAM GNN training by pushing sampling into graph queries"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes that a database drawing a random cap of final-hop neighbours produces the same kind of neighbourhood sample as GraphSAGE's per-layer sampling, so that the model learned is the one the paper claims to train.","fun_headline_variants_meta":{"raw":{"variants":["Train large-graph GNNs with 8GB RAM using graph queries","Offload neighbor sampling to the database to train GNNs on huge graphs","Graph DB as the sampling engine: train GNNs on 100M-node graphs with 8GB RAM","Query engine does the heavy lifting: GNN training on 8GB RAM for huge graphs","Minimal RAM GNN training by pushing sampling into graph queries"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001024,"raw_usage":{"total_tokens":4247,"prompt_tokens":803,"completion_tokens":3444,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":419,"completion_tokens_details":{"reasoning_tokens":3346}},"tokens_in":419,"tokens_out":3444,"duration_ms":22034,"temperature":1.0,"reasoning_tokens":3346,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:34:23.965699+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same GraphSAGE model, with identical hyperparameters and seeds, on ogbn-products using the paper's database sampler and using the standard in-memory per-layer sampler; if test accuracy differs by more than run-to-run noise, the final-hop-only random LIMIT has changed the model and the claim that database queries reproduce GraphSAGE sampling fails.","supporting_citations":[{"cited_title":"Hamilton, Zhitao Ying, and Jure Leskovec","cited_arxiv_id":null,"evidence_quote":"Defines GraphSAGE and the node-wise per-layer neighbourhood sampling scheme that the paper's query must reproduce."},{"cited_title":"Open graph benchmark: Datasets for machine learning on graphs","cited_arxiv_id":null,"evidence_quote":"Supplies the ogbn-papers100M and ogbn-products datasets that anchor the memory and scaling experiments."},{"cited_title":"https://neo4j.com/","cited_arxiv_id":null,"evidence_quote":"Documents native graph storage with node attributes co-located, supporting the paper's claim that features can be retrieved efficiently with node IDs."},{"cited_title":"Kùzu: Graph learning applications need a modern graph DBMS","cited_arxiv_id":null,"evidence_quote":"Describes an earlier integration of a graph database with PyG that loads the graph into memory, used as the comparison baseline for embedded databases."},{"cited_title":"Kùzu graph database management system","cited_arxiv_id":null,"evidence_quote":"Identifies the embedded graph database system used as the 'Embedded' backend in the single-machine experiments."}],"review_version":1}