{"id":"ea8fca29-42e8-4e94-9531-c0c7179ed3a5","arxiv_id":"2508.10732","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A closed-form dual-stream least-squares method for personalized federated learning that is provably invariant to how data are distributed among clients.","lead":"APFL applies closed-form least-squares equations to personalized federated learning, using a shared global stream plus a per-client refinement stream atop a frozen foundation model. It claims a 'heterogeneity invariance' property and large accuracy gains over gradient-based baselines on non-IID benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The primary-stream aggregation is valid only if all clients share the same random projection R_P and activation σ_P; the paper leaves this protocol detail unstated, making Theorems 1 and 3 conditional on an unenforced assumption.","rationale":"I re-derived the recursive aggregation in §3.2 and found no algebraic flaw when all clients use the same random projection and activation: the recurrence (8)-(10) is consistent with computing (ΣΦ_k^TΦ_k + γI)^{-1} ΣΦ_k^TY_k. The central theoretical claim therefore appears sound under the natural reading that R_P and σ_P are global. However, the paper never explicitly says how clients are forced to use the same projection, and the main text's 'F_1 = \\hat P_1' is a concrete internal inconsistency that needs correction. The reader's weakest assumption is close to this concern, but I would not state that sharing is entirely unmentioned, because the equations use unsubscripted global symbols. The deeper reason for a conditional verdict is the combination of this unstated protocol detail, the F_1 initialization issue, and the missing appendices containing the proofs of Theorems 1-4. None of these is fatal if addressed, so I would keep the reader's CONDITIONAL verdict rather than moving to accept or reject.","tokens_in":12575,"tokens_out":10634,"duration_ms":118431,"concrete_test":"Read the omitted Algorithm 1 / Appendix A and verify (i) whether R_P and σ_P are constructed once on the server or from a shared seed and reused by all clients, and (ii) whether F_1 is initialized as \\hat G_1 rather than \\hat P_1. Then run a small numerical check: for K=2, generate X_1, X_2, choose two different R_P^{(1)} ≠ R_P^{(2)}, compute the left side of Eq. (10) using the paper's recursion, and compare it to the closed-form G = (Σ_k Φ_k^T Φ_k + γI)^{-1} Σ_k Φ_k^T Y_k from Eq. (2). If the two disagree while agreeing when R_P is shared, the unstated sharing requirement is load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations (7)-(10) aggregate client auto-correlation matrices A_k = Φ_k^T Φ_k + γI. This summation is defined only because all Φ_k live in a common feature space: Φ_k = σ_P(Backbone(X_k, Θ)R_P) with a single R_P and σ_P. The global notation suggests sharing, but the main text never states that R_P/σ_P are generated once on the server or from a common seed and distributed to every client. If clients sample independent projections, each Φ_k lies in a different random subspace; A_k and the Fused Knowledge recursion (8)-(10) are no longer additive, and Theorem 1's claimed equality with the centralized least-squares solution over D_{1:K} does not follow. Theorem 3 inherits the same dependency. Additionally, §3.2 writes the Fused Knowledge Matrix initialization as F_1 = \\hat P_1, which cannot be right: the refinement stream is not available at aggregation time, and the update (8) uses \\hat G_k. Unless this is a typo for \\hat G_1, the recursion is broken independently of the projection-sharing issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes APFL, an analytic personalized federated learning method. Each client extracts features from a frozen foundation model, applies a random projection and nonlinear activation to form two feature spaces: a shared primary stream Φ_k and a local refinement stream Ψ_k. The primary stream is trained locally as ridge regression (Eqs. (4)-(5)); clients upload the auto-correlation matrix A_k and local model G_k. The server aggregates them via recursive equations (7)-(10) to obtain a global primary stream G that is claimed to coincide with the centralized ridge regression solution over all clients' data (Eq. (2)). Each client then solves a local residual regression (Eqs. (12)-(13)) to obtain a refinement stream P_k, and inference is a weighted combination (Eq. (14)). The paper states three theorems: equivalence of primary and refinement streams and a 'heterogeneity invariance' property, plus a privacy theorem and complexity analysis. Experiments on CIFAR-100 and ImageNet-R with 50/100 clients under Dirichlet non-IID splits show accuracy improvements over baselines and lower communication/computation overhead.","tokens_in":12928,"tokens_out":8918,"duration_ms":99284,"significance":"If the equivalence theorems are proven, APFL is a significant contribution: it gives a closed-form single-round aggregation for PFL that is exactly the centralized ERM solution, avoiding iterative gradient updates and the associated non-IID drift. The heterogeneity invariance property is a direct consequence of defining G via the pooled objective (Eq. (2)); it is not a learned property, but the analytic aggregation does realize it exactly. The empirical gains are large and consistent. However, the paper currently ships without the appendices containing the proofs, and the aggregation equivalence depends on an unstated assumption of a shared random projection/activation across clients. These issues must be addressed before the claims can be accepted.","major_comments":[{"comment":"The proofs of Theorems 1-4 are deferred to 'Appendix A/B', but the manuscript as submitted contains no appendices. Theorem 1 is the central claim that the recursive aggregation (7)-(10) equals the centralized least-squares solution (2); Theorem 4 is a privacy guarantee. Without these proofs, the reader cannot verify the load-bearing claims. The complexity derivation in §3.4 (Appendix C) is likewise absent. Please include the full proofs and derivations in the revision.","section":"§3.4 Theorems 1-4"},{"comment":"The aggregation step sums A_k = Φ_k^T Φ_k + γI across clients. This is well-defined and equivalent to centralized ERM only if all Φ_k live in the same feature space, i.e., if R_P and σ_P in Eq. (3) are common across clients. The protocol never states that R_P/σ_P are generated once on the server (or from a common seed) and shared with all clients. If each client samples an independent random projection, the Φ_k lie in different random subspaces, and Theorems 1 and 3 no longer follow. Please state the sharing requirement explicitly and add it as a condition in the theorem statements and Algorithm 1.","section":"§3.2, Eq. (3), (6)-(10)"},{"comment":"The privacy claim is both unsupported and, as stated, suspect. From the uploaded quantities the server can compute A_k G_k = Φ_k^T Y_k, which directly reveals class-level feature sums. Whether this allows reconstruction of Φ_k or Y_k depends on the adversary's background knowledge and on N_k relative to d_P. The theorem asserts an unconditional impossibility. A formal model of the adversary (e.g., what is known about the backbone, R_P, label space) and a precise notion of impossibility are required. The proof is currently absent, so the claim cannot be evaluated.","section":"§3.4 Theorem 4"}],"minor_comments":[{"comment":"The Fused Knowledge Matrix is initialized as F_1 = P_hat_1, which appears to be a typo. The refinement stream is not available at aggregation time; the update (8) uses G_k. This should presumably be F_1 = G_hat_1. If it is not a typo, the recursion is broken.","section":"§3.2, Eq. (8)"},{"comment":"No error bars, confidence intervals, or significance tests are reported, and the asterisks in the 'Advance' row are not explained. Given the large claimed margins, at least a brief statistical summary would strengthen the empirical claim.","section":"§4.1 / Table 1"},{"comment":"The exact values of γ, β, d_P, d_R used in the main experiments are not stated in the main text; they are only said to be in Appendix D, which is missing. Please include these training details in the main text or provide the appendix.","section":"§3.4 / §4"},{"comment":"The notation X_k ∈ R^{N_k × l × w × h} is nonstandard for image data; typically one writes N_k × C × H × W. Please align with the common convention.","section":"§3.2, Eq. (3)"},{"comment":"The phrasing 'depending solely on the complete dataset D_1:K and its local dataset D_k' is slightly ambiguous, since D_1:K includes D_k. It may be clearer to say 'depending on the union of all data and on D_k, but not on the partition.'","section":"§3.4, Theorem 3"}],"recommendation":"major_revision","confidential_remarks":"The central idea is elegant and the empirical results are strong, but the manuscript is incomplete as submitted: the proofs of all theorems are in absent appendices, and the key aggregation equivalence rests on an unstated shared-random-projection assumption. The privacy theorem is also likely overstated. These are fixable in a revision, but they are load-bearing and currently prevent acceptance. I would be willing to re-review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: APFL is a real step forward for closed-form PFL, but the version on arXiv is not complete enough to verify the central claims. The dual-stream least-squares formulation — a shared primary stream solved globally plus per-client residual refinement — is genuinely new for this area, and the derivation at the level of equations (5)-(10) and (13) is internally consistent. The scalar case checks out. The authors also deserve credit for giving the aggregation recursion a clean interpretation: the server accumulates auto-correlation matrices and fuses local solutions without multiple rounds.\n\nThe soft spots are real but mostly fixable. First, nothing in the main text states that all clients must use the same random projection R_P and activation σ_P for the primary stream. If each client samples its own projection, the Φ_k live in different feature spaces, the sums in (7) are not well-defined, and Theorem 1 collapses. I'm fairly convinced this is just an unstated protocol detail — the notation with a single R_P suggests sharing — but it's load-bearing, so it has to be explicit.\n\nSecond, §3.2 initializes the Fused Knowledge Matrix as F_1 = \\hat P_1. That cannot be right: the refinement stream doesn't exist at aggregation time. It's almost certainly a typo for \\hat G_1, and the recursion in (8) uses \\hat G_k. Still, as printed, the algorithm is broken at step one.\n\nThird, the main proofs are all in Appendix A/B/C, which are missing from the arXiv text. Theorems 1-3 are the heart of the paper, so deferring them entirely is not acceptable for a claim like \"heterogeneity invariance.\" Also, the heterogeneity invariance itself is not deep: it follows from the pooled least-squares objective (2), which is by construction independent of how the data are partitioned. Presenting it as a discovery is overselling, though it's not wrong.\n\nEmpirically, the results are suggestive but underreported: no error bars, no significance tests, and no clean same-protocol baseline that isolates the analytic advantage from the frozen ViT-MAE backbone. The efficiency comparisons are informative, but single runs on two datasets are thin. The self-citations are heavy but appropriate; the aggregation math is built on prior work by the same group, and they cite it.\n\nWho this is for: researchers working on analytic learning and PFL will want to read it. I'd send it to review — there's enough substance — but I'd insist on the appendices, the shared-projection statement, a typo fix, and proper repeated runs.","headline":"APFL is a genuinely new closed-form PFL scheme with a dual-stream least-squares design, but the arXiv version lacks its main proofs and has two load-bearing protocol gaps that make the central claims conditional.","tokens_in":13375,"tokens_out":2290,"would_cite":true,"duration_ms":27689,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"APFL claims that replacing gradient updates with two closed-form least-squares streams makes each client’s personalized federated model independent of how the other clients’ data are distributed.","keywords":["personalized federated learning","analytic learning","least squares","non-IID data","heterogeneity invariance","closed-form solution","dual-stream model","ridge regression"],"falsifier":"Take one pooled dataset, split it into two different partitions among clients, and run APFL under the paper’s protocol: if the final $\\{\\hat{G}, \\hat{P}_k\\}$ differ between partitions, Theorem 3 fails. Equivalently, compare the server’s aggregated $\\hat{G}$ from equations (7)–(10) with a direct centralized computation $(\\Phi_{1:K}^\\top \\Phi_{1:K} + \\gamma I)^{-1} \\Phi_{1:K}^\\top Y_{1:K}$; any mismatch falsifies Theorem 1. A second check: run clients with different $R_P$ matrices and see whether the aggregation still matches the centralized solution.","tokens_in":12525,"feed_emoji":"🧮","tokens_out":6999,"duration_ms":69241,"temperature":0.7,"pith_summary":"The paper tries to establish that personalized federated learning can be done without gradient updates at all. It builds two closed-form linear models on top of a frozen foundation model: a shared primary stream that exactly reproduces the central ridge-regression solution over all clients’ data, and a per-client refinement stream that fits the residual left by the primary stream on that client’s local data. Because both streams are solved in closed form and aggregation is algebraic, the final personalized model is claimed to be invariant to how data are distributed across other clients. The authors report that APFL outperforms existing federated and personalized baselines by 1.10% to 15.45% accuracy across CIFAR-100 and ImageNet-R settings, with a single aggregation round.","feed_headline":"No gradients, one round: federated personalization via least squares","feed_subtitle":"APFL’s closed-form dual streams make each client’s model independent of how unevenly other data are split.","key_machinery":"The key mechanism is the recursive aggregation of the aggregated auto-correlation matrix $\\tilde{A}_k = \\tilde{A}_{k-1} + A_k$ with $A_k = \\Phi_k^\\top \\Phi_k + \\gamma I$, and the Fused Knowledge Matrix $F_k = \\Lambda_k F_{k-1} + \\Delta_k \\hat{G}_k$, with the coefficient matrices defined in equation (9). These recurrences are constructed so that the server output $\\hat{G} = [\\tilde{A}_K - (K-1)\\gamma I]^{-1} \\tilde{A}_K F_K$ equals the centralized least-squares solution defined in (2). The refinement stream uses a separate random projection $\\Psi_k$ and solves $\\hat{P}_k = (\\Psi_k^\\top \\Psi_k + \\beta I)^{-1}(\\Psi_k^\\top Y_k - \\Psi_k^\\top \\Phi_k \\hat{G})$, a local ridge regression on the prima","core_discovery":"The central claim is that the dual-stream construction exactly solves the PFL objective (1)–(2) without iterative optimization. The primary stream $\\hat{G}$ is not an approximation or a federation heuristic: Theorem 1 states that the server’s recursive aggregation over per-client matrices $A_k$ and $\\hat{G}_k$ equals the minimizer of the centralized ridge regression over all clients’ activated features. The refinement streams $\\hat{P}_k$ are the closed-form least-squares fits to the residuals $Y_k - \\Phi_k \\hat{G}$ (Theorem 2). Theorem 3 then states heterogeneity invariance: if a client’s local data and the pooled data are fixed, the final model $\\{\\hat{G}, \\hat{P}_k\\}$ is unchanged no matte","pith_inferences":[],"forward_implications":["APFL completes global aggregation and local personalization in a single communication round; the experiments show accuracy above 200-round gradient baselines.","The final personalized model is invariant to the non-IID partition of other clients’ data, so tuning a federated system to the particular heterogeneity pattern becomes unnecessary for the given backbone features.","The primary stream is exactly the centralized ridge-regression solution, so the aggregation step loses no information relative to pooling all features and labels.","The refinement stream’s closed form means personalization is computed locally with no extra server communication.","Under the stated assumptions, raw features and labels cannot be fully reconstructed from the uploaded $A_k$ and $\\hat{G}_k$.","The same recursive aggregation generalizes directly to other ridge-regression objectives, such as weighted least squares, while preserving the closed form.","Because the backbone is frozen, the invariance is invariance in feature space; a testable sequel is whether a lightly trained adapter before the analytic streams preserves the closed-form aggregation.","A natural stress test is to let different clients use different random projection matrices $R_P$ for the primary stream; if the aggregation then diverges from the centralized solution, the common-projection requirement is exactly the load-bearing assumption."],"supporting_citations":[{"why":"Supplies the blockwise recursive Moore–Penrose inverse technique that underpins the analytic aggregation recurrences.","marker":"Zhuang, Lin, and Toh 2021"},{"why":"Prior analytic federated learning method (AFL) that APFL extends from a single global model to personalized dual streams; also a baseline.","marker":"He et al. 2025"},{"why":"Provides the frozen ViT-MAE backbone used to extract all features in both streams.","marker":"He et al. 2022"},{"why":"Justifies using mean squared error loss instead of cross-entropy in analytic classification.","marker":"Hui and Belkin 2021"},{"why":"Supplies the CIFAR-100 benchmark used in the main experiments.","marker":"Krizhevsky and Hinton 2009"},{"why":"Supplies the ImageNet-R benchmark used in the main experiments.","marker":"Hendrycks et al. 2021"},{"why":"Provides the Dirichlet partition procedure used to generate non-IID client data splits.","marker":"Lin et al. 2020"},{"why":"Defines FedAvg, the standard federated baseline APFL compares against and outperforms.","marker":"McMahan et al. 2017"}],"fun_headline_variants":["Analytic federated learning: closed-form dual streams defeat non-IID","Zero-gradient PFL: least squares personalization stays robust to skew","Heterogeneity-invariant PFL via dual-stream least squares","One-shot federation? APFL solves personalization without iteration","No iterative fl: dual-stream least squares makes PFL exact"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The proof that the per-client auto-correlation matrices can be summed assumes every client uses the same random projection matrix $R_P$ and activation $\\sigma_P$ for the primary stream, so all $A_k$ live in one common feature space; the paper does not state this sharing requirement explicitly.","fun_headline_variants_meta":{"raw":{"variants":["Analytic federated learning: closed-form dual streams defeat non-IID","Zero-gradient PFL: least squares personalization stays robust to skew","Heterogeneity-invariant PFL via dual-stream least squares","One-shot federation? APFL solves personalization without iteration","No iterative fl: dual-stream least squares makes PFL exact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000374,"raw_usage":{"total_tokens":1839,"prompt_tokens":756,"completion_tokens":1083,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":995}},"tokens_in":500,"tokens_out":1083,"duration_ms":7816,"temperature":1.0,"reasoning_tokens":995,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:14:09.612590+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one pooled dataset, split it into two different partitions among clients, and run APFL under the paper’s protocol: if the final $\\{\\hat{G}, \\hat{P}_k\\}$ differ between partitions, Theorem 3 fails. Equivalently, compare the server’s aggregated $\\hat{G}$ from equations (7)–(10) with a direct centralized computation $(\\Phi_{1:K}^\\top \\Phi_{1:K} + \\gamma I)^{-1} \\Phi_{1:K}^\\top Y_{1:K}$; any mismatch falsifies Theorem 1. A second check: run clients with different $R_P$ matrices and see whether the aggregation still matches the centralized solution.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the blockwise recursive Moore–Penrose inverse technique that underpins the analytic aggregation recurrences."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Justifies using mean squared error loss instead of cross-entropy in analytic classification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the CIFAR-100 benchmark used in the main experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the ImageNet-R benchmark used in the main experiments."},{"cited_title":"U.; and Jaggi, M","cited_arxiv_id":null,"evidence_quote":"Provides the Dirichlet partition procedure used to generate non-IID client data splits."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines FedAvg, the standard federated baseline APFL compares against and outperforms."}],"review_version":1}