{"id":"c2774c93-51d5-4046-b169-29940c9b5fa5","arxiv_id":"2501.08998","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A Riemannian diffusion model with an equivariant graph network generates crystal structures that sit close to DFT ground states and can be steered toward specified point groups.","lead":"CrystalGRW is a generative model that creates new crystal structures by running a trained diffusion process on the natural geometric spaces for atomic positions, atom types, and lattice shapes. It can target properties like crystal symmetry, and its best candidates sit close to density-functional-theory ground states, which could speed up materials discovery.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Novelty criterion may label near-duplicates of training structures as novel; the low RMSD for non-novel samples and 48% novelty rate suggest the near-ground-state claim for 'novel' structures is not yet separated from memorization.","rationale":"The reader's weakest assumption is that the learned score functions generalize outside the ALEX-MP-20 training distribution, so that novel structures are not partial reconstructions of training data. My stress-test identifies the same load-bearing point and sharpens it: the paper never reports how far 'novel' generated structures are from the nearest training structure, and the very low RMSD for non-novel structures is a red flag that the generation process may be interpolating near training motifs. The S.U.N. subset issue and the exclusion of certain elements are secondary but reinforce the need for a more careful evaluation. This does not refute the paper's central contribution; it means the strongest claim should be accepted only conditionally, pending a direct test of whether 'novel' structures are truly outside the training distribution. Since the reader already issued a CONDITIONAL verdict, my read does not change that verdict.","tokens_in":30995,"tokens_out":4994,"duration_ms":50081,"concrete_test":"Re-run generation from a model trained on ALEX-MP-20 with a held-out composition split: hold out 10% of compositions, train on the remaining 90%, generate structures, and for every generated 'novel' structure compute the minimum RMSD to the held-out training set and to the held-out compositions' DFT-relaxed structures. If the novel-set minimum-RMSD distribution is not clearly separated from the non-novel distribution (e.g., median below 0.05 Å), or if held-out-composition generation fails, the near-ground-state claim for novel structures is not supported. A simpler alternative for the existing 10^4 generated samples: report the histogram of minimum StructureMatcher RMSD from each 'novel' sample to ALEX-MP-20; if most novel samples lie within 0.05 Å of a training structure, the novelty label is too permissive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that CrystalGRW generates 'novel' structures close to DFT ground states rests on the assumption that the 48% of samples not matched to ALEX-MP by StructureMatcher are genuinely outside the training distribution. The paper's own numbers weaken this: non-novel samples relax to an average RMSD of 0.0014 Å, while 'unique and novel' samples average 0.011 Å (§II C–D, Fig. 4(b)). StructureMatcher's default tolerances (ltol=0.2, stol=0.3, angle_tol=5.0; Appendix G) are loose enough that a generated structure can differ from a training entry by several tenths of an Å in lattice parameters and still count as a match; conversely, the same tolerances may fail to detect near-duplicates obtained by interpolating between training structures. Since the model is trained on ALEX-MP-20 and novelty is checked against the superset ALEX-MP, a 'novel' label only means no training entry is within the loose match threshold. No distribution of distance to the nearest training structure is reported for the novel set. The very low memorized-sample RMSD and the drop in uniqueness and novelty at 10^4 samples indicate that the model may be reproducing training motifs with small perturbations. If so, the 0.011 Å RMSD for 'novel' structures is not evidence of discovering new ground states, only of interpolating near known ones. Additionally, the 1,436-structure subset used for the S.U.N. rate is not described as randomly selected and explicitly excludes structures containing Mo, Pd, Ir, and Pt, so the 37.72% rate is not established as representative of the full generation distribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript introduces CrystalGRW, a score-based diffusion generative model for crystal structures that operates on a product of Riemannian manifolds: fractional coordinates on a 3-torus, atomic types on a hypercube/simplex via a uniform-spacing map, and lattice matrices in Euclidean space. The denoiser is an EquiformerV2 equivariant graph neural network, with a modified atomic-type estimation head and classifier-free guidance for point-group conditioning. The authors train on ALEX-MP-20, generate 10^4 structures, and report that all are structurally valid, 90.74% are compositionally valid, the average RMSD to DFT-relaxed structures is 0.0053 Å, the S.U.N. rate on a 1,436-structure DFT subset is 37.72%, and point-group conditioning is successful for the most frequent symmetry classes. The paper also provides algorithmic listings, hyperparameters, code/data availability, and DFT evaluation details.","tokens_in":31442,"tokens_out":11865,"duration_ms":122170,"significance":"If the evaluation gaps are closed, this is a useful contribution. The paper extends Riemannian score-based generative modeling to full crystal generation with an equivariant architecture, provides a detailed and reproducible experimental setup, compares against MatterGen and other baselines, and gives concrete generated structures with DFT-computed Ehull values that are checkable predictions. The very low reported RMSD values and the S.U.N. example table are promising scientific outputs. The main risk is not circularity in the training/evaluation design, but rather that the 'novel, close-to-ground-state' claim is not yet separated from memorization of training motifs and from possible favorable filtering in the DFT-evaluated subset.","major_comments":[{"comment":"The central S.U.N. comparison is not supported by the reported evaluation protocol. The 37.72% S.U.N. rate is computed on 1,436 structures whose selection from the 10^4 generated set is not described; if this subset is not a random or otherwise predefined sample, the comparison with MatterGen's 38.57% is invalid. In addition, §IV D states that structures containing Mo, Pd, Ir, or Pt were excluded from the Ehull analysis after observing anomalously low values; the number of affected structures and the stability statistics before exclusion are not reported. Please specify the subset construction, report the stability rate on the full generated set or on a clearly random subset, and give with/without-exclusion results so the reader can assess the bias introduced by these choices.","section":"§II C and §IV D"},{"comment":"The novelty claim is not yet separated from memorization. StructureMatcher is used with ltol=0.2, stol=0.3, angle_tol=5.0 against ALEX-MP, and since the training set ALEX-MP-20 is contained in ALEX-MP, a 'novel' label only means that no training entry is within a loose match threshold. The very low non-novel RMSD (0.0014 Å), the higher novel RMSD (0.011 Å), and the drop in uniqueness/novelty at 10^4 samples make it plausible that many 'novel' samples are small perturbations of training motifs that happen to fall outside the tolerance window. No distance-to-nearest-training-structure histogram is reported for the novel set. Please provide this histogram, report novelty under tighter StructureMatcher tolerances, and, if possible, evaluate against ALEX-MP-ICSD so the reader can judge whether the novel structures are genuinely outside the training distribution.","section":"§II C and Appendix G"},{"comment":"The uniform-spacing map is not a bijection on the full hypercube, and the paper's use of it in the diffusion loop may break species identity. H maps Δ^d to the sorted subset of C^d, while H^{-1} sorts an arbitrary vector A before differencing; Algorithms 1, 4, and 5 apply H^{-1} to A_t that has become unsorted after the geodesic random walk, so the result is a projection onto sorted order statistics rather than the inverse of H. This projection can decouple the d+1 species labels from the components of the simplex vector, and at large noise levels the sorted spacings are exchangeable over species, making the denoising task ill-posed. Please clarify how species identity is preserved through this projection, or modify the construction so that the forward and inverse maps are true inverses on the space actually used for the random walk.","section":"Appendix A 4 and Algorithms 4-5"},{"comment":"Equations (A5) and (A6) define the exponential and inverse exponential maps on T3 with cos/sin expressions that are the geodesic formulas for a sphere, not for a flat torus; on T3 the correct maps are componentwise addition/subtraction modulo 1. Since Eq. (3) and Algorithms 1-3 rely on these maps, the mathematical presentation is internally inconsistent. Please correct the formulas or explicitly state that the analytic form is illustrative and that the implementation uses modular wrapping, with the corresponding inverse map. As written, the appendix does not provide a reproducible definition of the torus score-matching objective.","section":"§IV A and Appendix A 3"}],"minor_comments":[{"comment":"The row for point group 432 in Table II lists labels (2,2,1,0,1,1,1), which do not match the verbal description in Appendix D that 432 has labels (3,2,1,0,0,0,0); please reconcile the table and the text.","section":"Table II and Appendix D"},{"comment":"The footnote to Table I says RMSD is averaged over 10^3 structures sampled from the generation set, but the paper does not state whether this is the same 10^3 used for Fig. 4(b) nor how the 10^3 are sampled; please specify this to make the comparison with the 10^4-generation claims precise.","section":"Table I and Fig. 4"},{"comment":"There are several typographical errors, including 'acheiving', 'CryslGR W' in the code availability line, 'F orward', 'V ariation', 'pricipal', and 'ubutes'; a careful proofreading pass is needed.","section":"Throughout"},{"comment":"The text states that the forward–inverse pair in Eqs. (A7)-(A10) is 'a bijection, so no information is lost'; this is only true between the simplex and the sorted subset of the hypercube, and the wording should be corrected even if the algorithmic concern in the major comments is resolved.","section":"Appendix A 4"}],"recommendation":"major_revision","confidential_remarks":"The paper reports promising results, but the S.U.N. and novelty claims depend on evaluation choices that are not fully disclosed. I would ask the authors for the per-structure DFT-evaluated subset, the nearest-training-structure distances for the novel set, and the with/without-exclusion Ehull statistics; making these available would allow the claims to be independently verified. The atomic-type map concern in Appendix A 4 is the one that most affects reproducibility and may require a re-examination of the method rather than just additional experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: CrystalGRW is a solid engineering paper that combines RSGM with EquiformerV2 to generate near-ground-state crystal structures with point-group conditioning. The headline numbers (RMSD 0.0053 Å on ALEX-MP-20, S.U.N. 37.72%) are impressive and worth a serious referee. The novelty is moderate—the framework is borrowed from RSGM/FlowMM, and the contributions are the EquiformerV2 denoiser, the learned atomic-type estimator, and the 7-label point-group conditioning scheme.\n\nWhat's genuinely good: the paper is transparent about its choices and limitations. It shows the atomic-type estimator dramatically improves perovskite composition generation (Appendix C), it reports the symmetry bias and its effect on uniqueness, and it openly discusses the exclusion of Mo/Pd/Ir/Pt from stability analysis. The RMSD numbers, if reproducible, beat MatterGen on the same dataset. The code and dataset links are provided.\n\nThe soft spots are in the evaluation protocol. The novelty check uses StructureMatcher with loose tolerances against ALEX-MP, not ALEX-MP-20, and the paper doesn't report the distance-to-nearest-training-structure distribution for the 'novel' set. Given that non-novel samples have RMSD 0.0014 Å and novelty drops at 10^4 samples, the 48% novelty figure is likely inflated by near-duplicates that don't trigger the matcher. That doesn't kill the paper—even interpolating near known structures is useful for inverse design—but the phrase 'novel' should be softened and the metric tightened.\n\nSecond, the S.U.N. rate is computed on an unspecified 1436-structure subset. The paper doesn't say how it was selected, and it excludes four elements. That makes the 37.72% hard to compare with MatterGen's 38.57% on 10^4 samples. This is fixable with a clear sampling protocol.\n\nThird, the point-group conditioning works well for frequent symmetries but macro F1 scores are 0.13–0.22. The authors acknowledge this, so it's a limitation, not a flaw.\n\nOverall: the core method is sound, the math is standard RSGM, and the implementation is detailed. I'd send this to review, and recommend the authors tighten the novelty definition and the S.U.N. selection before publication. It's a useful reference point for anyone working on generative models for crystals.","headline":"Solid engineering contribution: RSGM + EquiformerV2 generates near-ground-state crystals with strong RMSD numbers, but the novelty and S.U.N. evaluation protocols need tightening before the claims fully hold.","tokens_in":31973,"tokens_out":2352,"would_cite":true,"duration_ms":22679,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CrystalGRW generates crystals 0.0053 Å from their DFT ground states and can steer the point group on request.","keywords":["crystal structure generation","diffusion models","Riemannian manifolds","geodesic random walks","equivariant graph neural networks","inverse design","point group conditioning","DFT ground states"],"falsifier":"Hold out a random 10–20% of ALEX-MP-20 during training, generate a fresh batch, and check each claimed novel structure against the held-out set with the same StructureMatcher tolerances; if held-out structures reappear among 'novel' generations at well above chance, the novelty and RMSD claims are largely memorization. An even more direct check is to relax 100 claimed novel, unique structures with DFT and verify that their computed energies and forces genuinely sit near the convex hull rather than echoing training data.","tokens_in":30790,"feed_emoji":"⚛️","tokens_out":7613,"duration_ms":73633,"temperature":0.7,"pith_summary":"CrystalGRW is a generative model that produces new crystalline materials whose structures are already close to their density-functional-theory ground states, so candidates can be trusted before expensive relaxation. It does this by diffusing each crystal property on the geometric space it actually lives in: fractional coordinates on a 3D torus, atomic types on a probability simplex, and lattice matrices in Euclidean space, with an equivariant graph network denoising all three together. On the ALEX-MP-20 dataset the average displacement between generated and DFT-relaxed structures is 0.0053 Å, compared with 0.021 Å for the leading prior model, and 95% of the 1,436 structures checked by DFT relax below the 100 meV/atom stability cutoff. The model also accepts a condition, demonstrated for crystallographic point groups, letting a user steer generation toward a target symmetry. If these claims hold, CrystalGRW offers a route from composition to near-relaxed, symmetry-consistent candidate structures without running DFT on every candidate.","feed_headline":"Crystal diffusion lands 0.005 Å from DFT ground states","feed_subtitle":"It also steers crystal symmetry, so users can request a target point group for inverse materials design.","key_machinery":"The carrying object is the geodesic random walk on a product of Riemannian manifolds, meaning a random walk that follows shortest paths on curved spaces, one manifold per crystal property. Fractional coordinates diffuse on the 3D torus T3, atomic-type probabilities diffuse on a d-simplex that is first mapped by a uniform-spacing bijection to a d-hypercube with reflecting boundaries, and the lattice matrix diffuses in Euclidean R3×3; reverse time is driven by learned manifold-specific scores. Denoising is performed by EquiformerV2, an SO(3)-equivariant graph neural network whose l=1 output head predicts the coordinate score as an equivariant vector and whose l=0 heads predict atomic-type and lattice scores. The score-matching training objective uses a small-time asymptotic that replaces the true score by (1/t) times the inverse exponential map from the noised point toward the original data, and an adaptive timestep with a variance-exploding no-drift forward process is used in practice.","core_discovery":"On its own terms, the paper's discovery is that a generative model built from geodesic random walks on manifolds can propose crystal structures that sit essentially on top of their DFT-relaxed geometries while remaining mostly valid, unique, and novel. Of $10^{4}$ generated samples, all pass a 0.5 Å interatomic-distance check and 90.74% are charge neutral; across the 1,436 structures relaxed with DFT, 95.0% have energy above hull below 100 meV/atom and the stable-unique-novel (S.U.N.) rate is 37.72%, comparable to MatterGen's 38.57%. The average RMSD between generated and relaxed structures is 0.0053 Å on ALEX-MP-20, with the novel subset at 0.011 Å, and 42.60% of generated structures already have DFT forces below the 0.02 eV/Å relaxation convergence threshold. Conditional generation guided by seven symmetry labels (three rotation axes, three mirror-plane types, and inversion) reproduces frequent point groups such as m-3m, 4/mmm, and 2/m with high success rates, while rare point groups tend to fall back to higher-symmetry relatives.","pith_inferences":["A natural extension the paper does not run: couple CrystalGRW to a machine-learned force field to pre-relax every generated candidate, then DFT only the survivors; the force predictions shown in the paper suggest this would be cheap.","The seven-label point-group decomposition could generalize to space-group-level control, which would let users target specific Wyckoff settings rather than just the point-group class.","The drop in uniqueness from 90% at 10^3 samples to 76% at 10^4, plus the 0.0014 Å RMSD for non-novel samples, hints that some 'novel' structures may be recombinations of training fragments; checking generation on a held-out subset of ALEX-MP-20 would separate interpolation from memorization."],"forward_implications":["Generated structures arrive close enough to their DFT ground states (0.0053 Å average RMSD) that many can skip most of the relaxation search, cutting the cost of downstream DFT validation.","Point-group conditioning gives a handle for inverse design: a user can ask for cubic or hexagonal symmetry and receive candidates whose space group reflects that input.","Because the S.U.N. rate is in line with MatterGen's while compositional validity is higher, the approach is a viable alternative for screening stable, unique, novel candidates from a large materials database.","The same manifold decomposition applies to other periodic systems, since coordinates, occupancies, and lattice metrics are universal descriptors of any crystal."],"supporting_citations":[{"why":"Supplies the Riemannian score-based generative modeling framework and the score-matching loss that CrystalGRW adapts to crystal properties.","marker":"[47]"},{"why":"Provides the EquiformerV2 equivariant graph network used as the denoiser backbone.","marker":"[14]"},{"why":"Supplies the ALEX-MP-20 training set, the stability and novelty evaluation protocol, and the S.U.N. baseline the paper compares against.","marker":"[28]"},{"why":"Defines the structural and compositional validity checks (0.5 Å distance and charge neutrality) and the Perov-5 benchmark used for the atomic-type head ablation.","marker":"[22]"},{"why":"Provides classifier-free guidance, the mechanism CrystalGRW uses for point-group conditioning.","marker":"[49]"},{"why":"Gives the uniform-spacing bijection between simplex and hypercube used to walk on atomic-type space.","marker":"[50]"},{"why":"Provides the small-time asymptotic that justifies replacing the true manifold score by (1/t) times the inverse exponential map.","marker":"[63]"},{"why":"Supplies the variance-exploding no-drift SDE formulation adopted for the Euclidean lattice walk.","marker":"[32]"}],"fun_headline_variants":["Geodesic diffusion generates crystals 0.005 Å from DFT truth","CrystalGRW: diffuse on manifolds to propose near-ground-state crystals","Symmetry-aware generative model nails crystal structures to 0.005 Å","Target point groups on demand: diffusion model for inverse crystal design","Stable, unique, novel: CrystalGRW matches MatterGen on structure generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the score functions learned on ALEX-MP-20 generalize to compositions, cell sizes, and geometries outside the training distribution, so the roughly 48% of generated structures counted as novel are newly invented crystals rather than reassembled pieces of training data.","fun_headline_variants_meta":{"raw":{"variants":["Geodesic diffusion generates crystals 0.005 Å from DFT truth","CrystalGRW: diffuse on manifolds to propose near-ground-state crystals","Symmetry-aware generative model nails crystal structures to 0.005 Å","Target point groups on demand: diffusion model for inverse crystal design","Stable, unique, novel: CrystalGRW matches MatterGen on structure generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1442,"prompt_tokens":963,"completion_tokens":479,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":382}},"tokens_in":579,"tokens_out":479,"duration_ms":5421,"temperature":1.0,"reasoning_tokens":382,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:11:55.790918+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out a random 10–20% of ALEX-MP-20 during training, generate a fresh batch, and check each claimed novel structure against the held-out set with the same StructureMatcher tolerances; if held-out structures reappear among 'novel' generations at well above chance, the novelty and RMSD claims are largely memorization. An even more direct check is to relax 100 claimed novel, unique structures with DFT and verify that their computed energies and forces genuinely sit near the convex hull rather than echoing training data.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Riemannian score-based generative modeling framework and the score-matching loss that CrystalGRW adapts to crystal properties."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the structural and compositional validity checks (0.5 Å distance and charge neutrality) and the Perov-5 benchmark used for the atomic-type head ablation."},{"cited_title":"Ho and T","cited_arxiv_id":null,"evidence_quote":"Provides classifier-free guidance, the mechanism CrystalGRW uses for point-group conditioning."},{"cited_title":"Devroye, Non-uniform random variate generation (Springer, New York, NY, 2013) pp","cited_arxiv_id":null,"evidence_quote":"Gives the uniform-spacing bijection between simplex and hypercube used to walk on atomic-type space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the small-time asymptotic that justifies replacing the true manifold score by (1/t) times the inverse exponential map."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the variance-exploding no-drift SDE formulation adopted for the Euclidean lattice walk."}],"review_version":1}