{"id":"15c7ae30-4107-45ed-8d4e-3d8f88af81b5","arxiv_id":"2608.07786","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Weight-space spectral statistics and subspace geometry separate independent, same-family, and shared-base LLMs, and track fine-grained post-training differences.","lead":"Researchers show that large language models carry a weight-space fingerprint: simple statistics of the model's internal matrices can reveal whether two models were trained independently, come from the same family, or share a common base. This could help auditors and regulators trace the origin of an open-weight model using only its weights, without training data or outputs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The S1-vs-S2 AUC 0.850 rests on only 37 distinct model pairs and an unvalidated layer-alignment interpolation (Eq. 4); the reported 110+ pairs are near-duplicates, so the robustness claim is not yet established.","rationale":"The reader correctly flags the unvalidated interpolation in Eq. (4), and I agree that the normalized-layer-index assumption is a real weakness: normalized depth need not correspond to functional similarity across different architectures. However, I would place the most load-bearing weight on the effective sample size and the Gaussian-augmentation design. Only 57 distinct pairs exist in the model zoo, and only 37 of them are used for the S1-vs-S2 comparison. Adding sigma=1e-4 noise to each pair produces near-identical fingerprints, so the claimed 114-pair evaluation is effectively still 57 independent pairs. With no confidence intervals and no code, an AUC of 0.850 could easily shift under bootstrapping or a slightly different pair selection. The same-series definition is also internally inconsistent: Mixtral-8x7B is a sparse Mixture-of-Experts model paired with dense Mistral-7B, and Falcon-7B vs Falcon-11B likely differ in architecture; this makes the S2 label itself a mixture of 'same training pipeline' and 'related architecture', and the interpolation assumption can behave differently across those cases. On the positive side, the paper uses standard SVD/principal-angle machinery, provides full algorithmic pseudocode, and its controlled Alpaca data-scale ablation is a good experimental design even though it uses only n=4 points with no seed replicates. The fine-grained subspace-alignment findings are plausible and could survive additional scrutiny, but the coarse-grained Trace claim is the one that must hold for the hierarchy narrative, and it is the least statistically supported. A conditional acceptance requiring the bootstrap/equal-depth analysis, released code, and a larger distinct-pair evaluation is the appropriate outcome; this matches the reader's conditional verdict, so no verdict change is needed.","tokens_in":25135,"tokens_out":10561,"duration_ms":98849,"concrete_test":"Release the fingerprint/evaluation code and recompute the Fig. 6 AUC using only the 37 original distinct pairs (no Gaussian duplicates), with a 95% bootstrap confidence interval and a permutation test against shuffled S1/S2 labels; also stratify by equal-depth pairs, where Eq. (4) is the identity. If the CI lower bound is below 0.70, the permutation p is above 0.05, or the equal-depth AUC is substantially below 0.850, the coarse-grained separation is not robust to the interpolation assumption and the headline claim needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing gap is statistical, not conceptual. The headline S1-vs-S2 result (AUC 0.850, Fig. 6) is computed on only 27 independent-origin and 10 same-series distinct pairs; the reported '110+ pairs' is reached by adding Gaussian noise with sigma=1e-4 to every model, which changes Trace scores by less than 1e-3 in most rows and by only a few hundredths even in low-correlation rows, so the augmented pairs are not independent replicates. There are no confidence intervals, no released code, and no preset decision threshold. The regime labels are also noisy: Mistral-7B-v0.3 vs Mixtral-8x7B-v0.1 and Falcon-7B vs Falcon-11B are listed as same-series despite MoE/dense and possibly differing architectures, so the S2 definition is not consistently 'same architecture, different scale'. Eq. (4) adds a second unvalidated layer: linear interpolation over a normalized layer index assumes functional correspondence across depths, and no alternative alignment is tested. Because the coarse-grained half of the central claim rests entirely on this AUC, either small-sample inflation or a bad alignment assumption would collapse it.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a white-box, weight-only method for tracing LLM lineage. It defines three regimes of increasing shared information—independent-origin (S1), same-series (S2), and shared-base (S3)—and introduces two complementary geometric fingerprints: a spectral trace fingerprint (Eqs. 2–5) derived from layer-wise sums of squared singular values, and a subspace-alignment fingerprint (Eqs. 6–8) based on the smallest principal angles between top-k singular-vector subspaces. The spectral trace is claimed to separate S1 from S2 (AUC 0.850, Fig. 6), while subspace alignment is claimed to distinguish shared-base variants differing in data scale and post-training algorithm (Table 2, Fig. 9). The paper reports experiments on 'over 110' model pairs, including Gaussian-noise-perturbed duplicates, and concludes that weight-space geometry provides a robust and interpretable lineage signal.","tokens_in":25315,"tokens_out":4617,"duration_ms":43215,"significance":"The conceptual framing—treating spectral magnitude and subspace direction as complementary lineage signals—is a useful step beyond existing static fingerprints such as AWM, HuReF, and PDF, and the proposed metrics are simple, computationally cheap, and parameter-light (k=256, J=3, K_layer=3). The controlled Alpaca data-scale ablation and the RLHFlow/Tulu variant comparisons are well designed and, if validated, would make the fine-grained results genuinely useful for model governance and provenance auditing. The main limitation is statistical: the headline coarse-grained result rests on 37 distinct pairs, the noise-augmented '110+ pairs' are not independent replicates, and the layer-alignment assumption in Eq. (4) is unvalidated. These issues are fixable with additional analysis and reporting, so the underlying approach remains promising.","major_comments":[{"comment":"The S1-versus-S2 AUC of 0.850 is computed on only 27 independent-origin and 10 same-series distinct model pairs; the claimed 'over 110 pairs' is reached by adding Gaussian noise (sigma=1e-4) to every model, which nearly doubles the dataset with perturbed duplicates. These augmented pairs are not independent replicates, so the AUC and the robustness claim do not follow from the reported sample. Moreover, the perturbation is not harmless in all rows: Table A2 shows Trace for Gemma-2B vs Gemma-7B changing from 0.6169 to 0.0298 and for Mistral-7B-v0.3 vs Mixtral-8x7B from 0.8679 to 0.4265 under sigma=1e-4. This contradicts the statement in Appendix D that Trace 'remains stable under perturbation.' Please report the AUC with confidence intervals on the 37 distinct pairs, and present the Gaussian perturbation as a separate stability analysis rather than as additional samples.","section":"§4, Fig. 6, Appendix B/D"},{"comment":"The layer-alignment step in Eq. (4) linearly interpolates trace fingerprints over a normalized layer index, implicitly assuming that layer position as a fraction of total depth corresponds to functional similarity across architectures of different depths. This assumption is load-bearing for the coarse-grained separation (Finding 1), since many S2 pairs (e.g., Pythia-1B vs 1.4B, LLaMA-2-7B vs 13B) have different depths and the S1 set includes models with 28, 32, and 40 layers. No validation of this alignment is provided, and no alternative alignment (e.g., comparing only common-depth layers, or aligning by block type and distance from input/output) is tested. Please add an ablation or justification for why normalized-layer interpolation is appropriate, or show that the S1-vs-S2 result is insensitive to the alignment choice.","section":"Eq. (4), §4"},{"comment":"The same-series regime is defined in §3 as pairs that 'share architectures and training pipelines, but differ in scale,' yet the S2 set in Appendix B.2 includes Mistral-7B-v0.3 vs Mixtral-8x7B-v0.1, which is a dense-versus-MoE pair, and Falcon-7B vs Falcon-11B, whose architectural differences are not documented. This inconsistency directly affects the measured S1-vs-S2 separation, because the MoE pair has the highest same-series Trace score (0.8679 in Table A2) and may inflate or distort the distribution. Please either exclude these pairs from the main S2 evaluation or analyze them as a separate intermediate category, as is already done for Qwen2.5-vs-Qwen3 in Appendix F.","section":"Appendix B.2, §3 definition of S2"},{"comment":"The paper's reproducibility claim is not fully met: the GitHub link in the header is malformed ('/githubGitHub/gl⌢beProject') and no working code repository is provided. Given that the paper's contribution is an empirical fingerprinting method with specific aggregation parameters (k=256, J=3, K_layer=3) and a particular interpolation rule, a public implementation is important for the community to verify the reported AUC and the subspace-alignment scores and to apply the method to new models.","section":"Reproducibility statement, header"},{"comment":"The fine-grained shared-base results, while suggestive, are reported without any uncertainty quantification. For example, Table A7 lists a single subspace-alignment score per pair, and the observed differences (e.g., Qwen3-4B vs Qwen3-4B-Thinking at 0.184 versus LLaMA-3.1-8B vs Alpaca-10% at 0.976) are presented as exact. Since these scores are computed from a single model checkpoint per pair, there is no estimate of variability from weight initialization or sampling. Please provide repeated-measure or bootstrap-based error bars, or at least report the per-component ranges, so the reader can judge whether the claimed data-scale and algorithm-specific effects are larger than the noise in the measurement procedure.","section":"§5, Table A7, Appendix K"}],"minor_comments":[{"comment":"The abstract and contribution list state 'over 110 diverse open-weight LLM pairs,' but the actual number of distinct pairs in the three regimes is 57, with the remainder being Gaussian-perturbed copies. Please state the distinct-pair count explicitly in the main text to avoid overstating the diversity of the evaluation.","section":"Abstract and §4"},{"comment":"The caption says the shaded regions have 'boundaries defined by the mean trace scores,' but it is not clear whether these boundaries are the mean of the scenario distribution or of something else. Please define the shading rule precisely in the caption or in Appendix D.","section":"Fig. 5"},{"comment":"The main text describes aggregating 'the three lowest-similarity layers' for each component, and Algorithm A2 uses K_layer for the same purpose; please define K_layer=3 explicitly in the text near Eq. (8) rather than only in the algorithm pseudocode.","section":"§5, Algorithm A2"},{"comment":"The sensitivity analysis for J shows that scores increase monotonically with J, and the paper states that relative differences remain consistent, but no threshold or decision rule is derived from these scores. Since the paper does not provide a calibrated classifier for the shared-base regime, please clarify whether the fine-grained claims are intended as descriptive findings or as an operational discrimination method.","section":"Appendix I, Table A6"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and the main experiments are, on their face, carefully designed. The load-bearing concerns are statistical: the AUC headline relies on 37 distinct pairs plus perturbed duplicates, and some perturbed rows are not stable, which undercuts the robustness narrative. The layer-alignment assumption and the S2 label inconsistency are also fixable. I believe a major revision that adds proper uncertainty quantification, reports distinct-pair results, validates Eq. (4), and cleans up the S2 label set would make the paper suitable for publication. The malformed code link should be corrected before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper is worth reading for its fine-grained shared-base analysis, but the headline coarse-grained AUC is weaker than the abstract suggests.\n\nWhat's genuinely good: the subspace alignment metric (principal angles between top singular subspaces) applied to shared-base pairs gives a clean, interpretable signal. The controlled Alpaca-SFT data-scale ablation — same base, same algorithm, only the amount of fine-tuning data changes — shows a monotonic decrease in alignment from 0.976 at 10% to 0.889 at 100% of the dataset. That is the strongest result in the paper and holds up as a real empirical finding. The algorithm-variant experiments (DPO/PPO/RAFT) also show component-specific directional changes that look plausible. The metrics have no fitted parameters: k=256, J=3, K_layer=3 are fixed and there is sensitivity analysis for J. That is honest practice.\n\nThe soft spots are on the coarse-grained side. The S1-vs-S2 AUC of 0.850 is computed on only 27 independent-origin and 10 same-series distinct pairs. The '110+ pairs' in the abstract is reached by adding Gaussian noise with sigma=1e-4 to every model, which just re-measures the same pairs; some of those scores shift a lot (pairs involving Gemma move by more than 0.1 in Trace), so the noise is neither harmless nor a source of independent replicates. No confidence intervals, no repeated runs, no released code. With n=37, an AUC of 0.850 is not a clean separation, and the reported stability is not convincing.\n\nTwo more issues compound that. The S2 label set is inconsistent: Mistral-7B (dense) vs Mixtral-8x7B (MoE) and Falcon-7B vs Falcon-11B are listed as same-series despite differing architectures and pipelines, which violates the paper's own definition. And Eq. (4) aligns trace fingerprints of models with different depths by linear interpolation over a normalized layer index; that assumes a functional correspondence between layers that is never validated or compared against alternatives like dynamic time warping or simple truncation.\n\nThese problems are real but not fatal to the central idea. The fine-grained subspace results are interesting enough to justify a revised version. The paper should release code, report confidence intervals, use only distinct pairs, clean the S2 labels, and test the interpolation assumption.\n\nWho's this for? Anyone working on model provenance, licensing audits, or weight-space geometry. It deserves a serious referee, but with the expectation of major revision before it can be trusted as a tool.","headline":"Promising fine-grained subspace-alignment results, but the headline coarse-grained AUC rests on 37 distinct pairs padded to '110+' by near-duplicate noise, and an unvalidated layer-interpolation assumption.","tokens_in":25909,"tokens_out":5670,"would_cite":false,"duration_ms":44681,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that the lineage of open-weight LLMs — independent, same-series, or shared-base — can be read off from weight geometry alone, using spectral energy for coarse separation and subspace alignment for fine discrimination.","keywords":["LLM lineage","weight-space fingerprints","spectral energy","singular value decomposition","subspace alignment","principal angles","model provenance","shared-base models"],"falsifier":"Train two models with identical architecture, tokenizer, and training data but independent random initializations, then compute their spectral trace correlation. If the score lands above the same-series threshold, spectral energy is encoding generic architecture rather than shared lineage; if it stays near independent-origin levels, the coarse lineage reading is confirmed.","tokens_in":1424,"feed_emoji":"🧬","tokens_out":1423,"duration_ms":52145,"temperature":0.7,"pith_summary":"This paper tries to establish that open-weight LLMs carry intrinsic fingerprints in their weight matrices, so that two models' relationship can be inferred without any training data or generated outputs. It defines a three-level hierarchy: independent-origin, same-series, and shared-base. It shows that spectral energy, computed cheaply as per-layer trace-based singular-value sums, separates the coarse levels, while subspace alignment, measured through principal angles between top singular-vector subspaces, separates fine variations inside the shared-base level, including dataset scale and post-training algorithm. If correct, model provenance, governance, and supply-chain auditing can be done passively from released weights.","feed_headline":"Spectral fingerprints in weights trace LLM lineage","feed_subtitle":"Singular-value energy separates model families; subspace angles separate post-trained twins.","key_machinery":"The machinery is a two-signal geometric fingerprint built from singular value decomposition of each weight matrix $W_c^{(l)}$. The first signal, spectral energy, is the Frobenius-norm-like trace $\\sqrt{\\mathrm{tr}((W_c^{(l)})^\\top W_c^{(l)})}$, summarized as a per-layer vector and compared by correlation after interpolation; it encodes global magnitude patterns that separate families and scales. The second signal, subspace alignment, compares top-$k$ left singular vector subspaces through principal angles, aggregated over the least-aligned layers and components; it captures the directional rotations that post-training induces without changing the spectrum. Together they form the unified fingerprint that the paper uses for hierarchical lineage tracing.","core_discovery":"The central claim is that weight-space geometry provides a reliable and interpretable signal for LLM lineage, enabling both coarse-grained regime separation and fine-grained discrimination within shared-base models. The paper proposes two complementary fingerprints: the spectral trace fingerprint, $\\tau_c(\\theta) = (t(W_c^{(1)}), \\ldots, t(W_c^{(L)}))$ with $t(W_c^{(l)}) = \\sqrt{\\mathrm{tr}((W_c^{(l)})^\\top W_c^{(l)})} = \\sqrt{\\sum_i \\sigma_i^2}$, which is aligned across different depths by linear interpolation over a normalized layer index and compared via Pearson correlation; and subspace alignment, which takes the singular values of the cross-subspace matrix $C_c^{(l)} = (U_{c,k}^{(l)}(\\theta_a))^\\top U_{c,k}^{(l)}(\\theta_b)$ as cosines of principal angles and averages the $J$ smallest ones. On over 110 open-weight LLM pairs, spectral trace separates independent-origin from same-series pairs with AUC 0.850, versus 0.256 for AWM and 0.655 for PDF, and perfectly separates same-series from shared-base pairs. Within the shared-base regime, subspace alignment orders Alpaca-SFT variants by data scale (0.976 at 10% down to 0.889 at 100%) and gives distinct per-component signatures to DPO, PPO, and RAFT post-training.","pith_inferences":["The three regimes are likely anchors of a continuous relatedness spectrum: the paper's cross-series Qwen2.5/Qwen3 mean of 0.479 sits between independent-origin and same-series means, suggesting any pair of open-weight models could be placed on the same graded scale.","Because subspace alignment is computed per component, it could serve as a diagnostic for which parts of a network a post-training method rewrites, potentially linking alignment drops to functional changes.","A natural testable extension is to separate data-sharing from code-sharing lineage: same architecture and tokenizer but different training data should produce different spectral-trace behavior than same data with different architecture.","The cheap trace fingerprint could be used as a screening stage, reserving the more expensive subspace-alignment computation for pairs that pass a coarse similarity threshold."],"forward_implications":["Lineage categories can be inferred from weights alone, without accessing training data, prompts, or model outputs.","Spectral trace gives a cheap prefilter: independent-origin versus same-series pairs are separable at AUC 0.850, while standard baselines fail.","Subspace alignment makes data-scale effects visible inside the shared-base regime, so one can rank post-training variants by how much weight geometry moved.","Different post-training algorithms leave distinctive per-component signatures, with the Q projection showing the largest method-dependent variation.","The coarse shared-base signal survives quantization, pruning, merging, and distillation, while subspace alignment tracks transformation severity."],"supporting_citations":[{"why":"Supplies the AWM baseline that the paper shows cannot separate independent-origin from same-series pairs.","marker":"(Zeng et al., 2025)"},{"why":"Supplies the PDF baseline whose scores overlap across coarse lineage regimes.","marker":"(Yoon et al., 2025)"},{"why":"Supplies the HuReF fingerprint that is inapplicable to same-series pairs with mismatched transformer depths.","marker":"(Zeng et al., 2024)"},{"why":"Supplies the principal-angle machinery used to define subspace alignment.","marker":"(Knyazev & Argentati, 2002)"},{"why":"Supplies the result that singular values of the cross-subspace matrix equal cosines of principal angles.","marker":"(Miao & Ben-Israel, 1992)"}],"fun_headline_variants":["Weight space fingerprints decipher LLM lineage","Spectral energy and subspace angles trace model roots","LLM ancestry decoded via weight matrix geometry","Singular values separate families, angles separate twins","Spectral trace and subspace angles reveal LLM lineage"],"cache_read_input_tokens":28032,"weakest_assumption_plain":"The load-bearing assumption is that linear interpolation over a normalized layer index makes layers at the same relative depth functionally comparable across different architectures and depths; if that mapping fails, the Trace separation of independent-origin from same-series pairs loses its grounding.","fun_headline_variants_meta":{"raw":{"variants":["Weight space fingerprints decipher LLM lineage","Spectral energy and subspace angles trace model roots","LLM ancestry decoded via weight matrix geometry","Singular values separate families, angles separate twins","Spectral trace and subspace angles reveal LLM lineage"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000248,"raw_usage":{"total_tokens":1613,"prompt_tokens":1081,"completion_tokens":532,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":697,"completion_tokens_details":{"reasoning_tokens":477}},"tokens_in":697,"tokens_out":532,"duration_ms":5666,"temperature":1.0,"reasoning_tokens":477,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:12:41.224074+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two models with identical architecture, tokenizer, and training data but independent random initializations, then compute their spectral trace correlation. If the score lands above the same-series threshold, spectral energy is encoding generic architecture rather than shared lineage; if it stays near independent-origin levels, the coarse lineage reading is confirmed.","supporting_citations":[],"review_version":1}