{"id":"8571f47a-4a5e-40dc-8a2e-e9a064dbf4aa","arxiv_id":"2502.01145","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Sheaf-FMTL learns projection maps between heterogeneous client models in decentralized federated learning, saving communication, but its convergence proof is incomplete and its objective degenerates to independent training.","lead":"The paper introduces Sheaf-FMTL, a decentralized federated multi-task learning algorithm that models client relationships as cellular sheaves with learned projection maps. It claims O(1/K) convergence and large communication savings, but the main convergence proof has a gap and the optimization objective has a trivial solution at zero projections.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 is unproven: the descent bound contains ∇PΨ at (θ^{k+1},P^k), but the conclusion bounds ∇Ψ at (θ^k,P^k); no argument connects them.","rationale":"The reader's rationale explicitly identifies the same proof gap: 'the convergence proof's final step compares gradients at inconsistent iterates, so the O(1/K) theorem is not established.' This is the most load-bearing concern because it directly invalidates the paper's central theoretical claim, independent of any other issues. The reader's formal weakest_assumption field focused on the trivial zero-projection solution, which is a separate and serious problem about the meaningfulness of the objective; however, even if that were addressed (e.g., by regularizing P), the proof gap would still block the claimed convergence guarantee. The empirical results show useful communication savings and the paper is clearly written, but the unproven Theorem 1 is the cornerstone of the contribution. Therefore the REJECT verdict stands unchanged, and I partially agree with the reader: the same concern appears in their rationale but is not their stated weakest assumption. A careful re-derivation of the final telescoping step would settle whether the theorem can be repaired.","tokens_in":34218,"tokens_out":6849,"duration_ms":66683,"concrete_test":"Independently re-derive the proof of Theorem 1, tracking the evaluation points in the telescoping sum. Specifically, attempt to prove Σ_{k=0}^{K-1} ∥∇Ψ(θ^k,P^k)∥² ≤ C(Ψ(θ^0,P^0)-Ψ⋆) using only Assumptions 1–2 and the stated step-size conditions. Show whether ∥∇PΨ(θ^k,P^k)∥² can be bounded in terms of ∥∇PΨ(θ^{k+1},P^k)∥² plus terms that telescope; if no such bound follows, the theorem is false. As a numerical check, run Sheaf-FMTL on a small graph (e.g., N=2, di=2, dij=1) and compute ∥∇PΨ(θ^1,P^0)∥² vs ∥∇PΨ(θ^0,P^0)∥² after the first θ-update; if the inequality used in (46) fails, the proof's final step is invalid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the O(1/K) convergence bound in Theorem 1, Eq. (20). The proof derives two descent inequalities: Eq. (37) bounds Ψ(θ^{k+1},P^k) using ∥∇θΨ(θ^k,P^k)∥², and Eq. (43) bounds Ψ(θ^{k+1},P^{k+1}) using ∥∇PΨ(θ^{k+1},P^k)∥². Summing gives Eq. (44)/(45), which contains a sum of ∥∇θΨ(θ^k,P^k)∥² and ∥∇PΨ(θ^{k+1},P^k)∥². In the final step, Eq. (46) claims a bound on (1/K)Σ∥∇Ψ(θ^k,P^k)∥², invoking the identity ∥∇Ψ(θ^k,P^k)∥² = ∥∇θΨ(θ^k,P^k)∥² + ∥∇PΨ(θ^k,P^k)∥². But the sum from (45) contains the P-gradient at (θ^{k+1},P^k), not (θ^k,P^k). No argument is provided to replace ∥∇PΨ(θ^{k+1},P^k)∥² by ∥∇PΨ(θ^k,P^k)∥² or to control the difference. Under Assumptions 1–2 alone, ∇PΨ is not shown to be Lipschitz in θ, so the mismatch is not a harmless technicality. The telescoping argument therefore does not establish a bound on the averaged squared norm of the gradient at the iterate (θ^k,P^k). The convergence theorem, which the paper identifies as the key theoretical contribution, is not supported by the proof.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Sheaf-FMTL, an alternating-gradient method for decentralized federated multi-task learning. Clients' models are coupled through learned restriction maps that project models into shared edge spaces, and the objective (17) adds a sheaf-Laplacian regularizer to the sum of local losses. The main theoretical claim is Theorem 1, an O(1/K) bound on the averaged squared gradient norm of the collaborative objective under L-smoothness of the losses and boundedness of the model parameters. The paper also shows that several existing FL/FMTL formulations are special cases (Section 3.3), and it reports experiments on rotated MNIST, heterogeneous CIFAR-10, and four additional datasets in which Sheaf-FMTL matches or approaches baseline accuracy while transmitting substantially fewer bits.","tokens_in":34591,"tokens_out":7596,"duration_ms":84770,"significance":"The unification of several existing FL/FMTL methods in Section 3.3 is correct and useful, and the empirical study is extensive: the communication-savings claim is supported by the transmitted-bit comparisons in Tables 4 and 5 and in Figures 2 and 6, and the paper honestly quantifies the extra storage and compute. If Theorem 1 were correctly proved and if the learned-map mechanism were guaranteed to avoid the decoupled solution, the paper would be a valuable theoretical contribution to decentralized FMTL. As it stands, the proof of the main theorem has a genuine gap, and the objective admits the trivial P=0 solution, so the theoretical and conceptual center of the paper is not established.","major_comments":[{"comment":"The descent bound (44) contains the term ||nabla_P Psi(theta^{k+1}, P^k)||^2, not ||nabla_P Psi(theta^k, P^k)||^2. After summation, Eq. (45) bounds the k-sum of the former quantity. Eq. (46) then asserts a bound on (1/K) sum_k ||nabla Psi(theta^k, P^k)||^2, using the identity ||nabla Psi(theta^k, P^k)||^2 = ||nabla_theta Psi(theta^k, P^k)||^2 + ||nabla_P Psi(theta^k, P^k)||^2. No step connects the two P-gradient terms. Under Assumptions 1 and 2 alone, nabla_P Psi is not Lipschitz in theta with a constant that is independent of P, and no a priori bound on ||P^k|| is shown, so the mismatch cannot be dismissed as a minor technicality. This is a genuine gap in the proof of Theorem 1, which is the paper's main theoretical claim.","section":"Appendix E, Eqs. (44)–(46)"},{"comment":"The objective in (17) is minimized, or at least has stationary points, at P_ij = 0 for all edges with each theta_i at a local optimum of f_i, because the sheaf regularizer is nonnegative and vanishes for zero restriction maps. Thus Theorem 1's stationary-point guarantee is compatible with the algorithm learning no interaction at all and reducing to independent local training. Remark 3's nonzero initialization only avoids the exact zero fixed point at initialization; it does not prevent the iterates from converging to the decoupled solution. To support the claim that Sheaf-FMTL learns task relationships, the formulation needs a constraint or regularizer that excludes the zero maps, or the paper must explicitly analyze and state the conditions under which the iterates remain in a non-degenerate regime.","section":"Section 3.4, Eq. (17) and Remark 3"}],"minor_comments":[{"comment":"The norm of the matrix-valued gradient nabla_P Psi is not specified in the theorem statement; it should be identified as the Frobenius norm.","section":"Theorem 1 and Eq. (20)"},{"comment":"The text says 'about 100x lower than dFedU (3230.9KB)', but Table 4 reports transmitted bits in MB, and dFedU's value is 3230.9 MB, not 3230.9 KB; the factor relative to 38.2 MB is about 85, not 100.","section":"Table 4 and Section 4.2"},{"comment":"The sentence 'It surpasses the local baseline accuracy by approximately62.5% around 60 rounds' is missing a percent sign or spacing; it should read 'approximately 62.5%.'","section":"Section 4.3, Figure 3"},{"comment":"Appendix E restates Theorem 1 as Theorem 2 with identical content; this duplication is confusing and should be removed or clearly labeled as a proof of the earlier theorem.","section":"Appendix E, Theorem 2"},{"comment":"Lemma 1 in the main text and Lemma 3 in Appendix D are identical; the paper should state the lemma once and reference it from the appendix.","section":"Lemma 1 and Lemma 3"},{"comment":"The 'Non-IID Metric' column lists qualitative descriptions rather than numerical metrics; reporting actual numerical values would make the heterogeneity quantification more reproducible.","section":"Table 9"}],"recommendation":"reject","confidential_remarks":"This paper has a useful empirical component and correctly identifies several existing methods as special cases, but the main theoretical result is not proved as written, and the optimization formulation admits a trivial decoupled stationary point. These are load-bearing issues for the paper's central claims. If the authors can close the proof gap, possibly with an additional bounded-P assumption, and either modify the formulation or substantially revise the claims to address the zero-map solution, a resubmission could be reconsidered. In its current form, I would not recommend acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere is my take on the sheaf-FMTL paper. The core idea is genuinely novel: representing client relationships as cellular sheaves with learned restriction maps, and using sheaf Laplacian regularization to handle heterogeneous model sizes in decentralized multi-task learning. The special-case unification (FedU, personalized FL, hybrid FL, and conventional FL as instances of the same objective) is a useful and correct observation. The experimental work is also solid: multiple datasets, heterogeneous architectures, honest reporting of the storage and compute overhead, and a clear demonstration that communicating low-dimensional projections saves bits without hurting accuracy much.\n\nThe problem is the theory. Theorem 1 as stated is not established. The stress-test note is right: the descent bound in (44) controls the P-gradient at (θ^{k+1}, P^k), but the conclusion in (46) is about (1/K)Σ||∇Ψ(θ^k, P^k)||². That final step silently replaces one iterate with the other. Under Assumptions 1 and 2, ∇_PΨ is not Lipschitz in θ, so the mismatch cannot be pushed through without an additional argument. This is the central theoretical contribution, and it fails on its own terms.\n\nThe second concern is about the objective itself. The regularizer is nonnegative and vanishes when all P_ij = 0, so the trivial solution — independent local optima with zero projection matrices — is a global minimizer of (17). Nothing in the convergence theorem rules out landing there. Random non-zero initialization gets the algorithm into a coupled regime in practice, and the experiments show it stays there and learns useful maps, but the theory does not explain why. So the formal guarantees are much weaker than the paper claims.\n\nThere are minor issues too: some of the reported savings (“100×”, “100% less”) do not match the tables, and the stochastic mini-batch version used in experiments is not covered by the deterministic proof.\n\nWho is this for? Researchers in federated multi-task learning and communication-efficient personalization. The framework and experiments are interesting enough to warrant a serious referee, despite the broken theorem. I would send it to review with the expectation of a major revision: either fix the descent proof, add the missing smoothness between θ and P, or state a weaker guarantee that honestly reflects what is proved. The empirical communication-savings claim stands on its own.\n\nRecommendation: accept for peer review, require substantial revision.","headline":"Novel sheaf-based framework with solid experiments, but the main convergence theorem has a genuine proof gap and the objective admits a trivial independent-training solution.","tokens_in":35122,"tokens_out":2450,"would_cite":false,"duration_ms":26253,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","68W15","90C26"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that modeling client relationships as a cellular sheaf, with learned projection maps per edge, unifies decentralized federated multi-task learning and yields an O(1/K)-convergent algorithm that communicates…","keywords":["federated multi-task learning","cellular sheaves","sheaf Laplacian","decentralized optimization","model heterogeneity","communication efficiency","convergence analysis"],"falsifier":"The paper's own Remark 3 shows that zero-initialized interaction matrices freeze the coupling and reduce the method to independent training; so a decisive check is to run with nonzero random initialization on a task pair known to share a subspace and test whether the learned maps stay nonzero and improve accuracy over local training. If the maps collapse to zero or the accuracy matches local training, the central practical claim fails.","tokens_in":34004,"feed_emoji":"🕸️","tokens_out":9500,"duration_ms":91070,"temperature":0.7,"pith_summary":"This paper claims that client relationships in decentralized federated multi-task learning can be modeled by a cellular sheaf, with each client's model living in its own vector space and each communication link carrying a shared projection space; learned linear maps project heterogeneous models into those spaces so they can be compared and aligned. The resulting objective, local losses plus a sheaf-Laplacian regularizer, is optimized by alternating gradient descent on the models and the projection maps, and clients exchange only the low-dimensional projected vectors. The paper proves an $O(1/K)$ convergence rate to a stationary point of the collaborative objective under smoothness and bounded-domain assumptions. It also shows the formulation specializes to graph-Laplacian FMTL, personalized federated learning, hybrid federated learning, and, in the $\\lambda\\to\\infty$ limit, conventional federated learning. The motivating payoff is that one algorithm handles clients with different model sizes and data distributions while transmitting far fewer bits, with experiments reporting accuracy close to baselines at roughly a hundredfold communication saving.","feed_headline":"Sheaf model cuts federated communication ~100x at same accuracy","feed_subtitle":"New algorithm learns projection maps between heterogeneous client models, proving O(1/K) convergence while cutting transmitted bits.","key_machinery":"The load-bearing object is the cellular sheaf on the client graph: vertex stalks $\\mathbb{R}^{d_i}$ hold each client's model, edge stalks $\\mathbb{R}^{d_{ij}}$ are the shared disclose spaces, and restriction maps $P_{ij}\\in\\mathbb{R}^{d_{ij}\\times d_i}$ project models into those spaces. The sheaf Laplacian $L_F=\\delta^*\\delta$ turns these maps into a regularizer, since $\\theta^T L_F\\theta$ equals the sum of squared projected disagreements over all edges. The algorithm alternates a gradient step on each $\\theta_i$ with a gradient step on each $P_{ij}$, requiring clients to exchange only the vectors $P_{ij}\\theta_i$ and $P_{ji}\\theta_j$; assigning $d_{ij}=\\lfloor\\gamma\\min(d_i,d_j)\\rfloor$ makes those exchanged vectors a tunable fraction of the model size. This machinery does two jobs at once: it lets models of different sizes interact through projections, and it makes the coupling strength itself learned rather than fixed in advance.","core_discovery":"On the paper's own terms, the central discovery is that the sheaf Laplacian quadratic form $\\theta^T L_F(P)\\theta = \\sum_{(i,j)\\in E}\\|P_{ij}\\theta_i - P_{ji}\\theta_j\\|^2$ provides a learnable measure of disagreement between heterogeneous local models. Because each edge has its own low-dimensional space and its own pair of restriction maps, models of different dimensions can be compared without ever requiring $\\theta_i$ and $\\theta_j$ to live in the same space. Theorem 1 states that, under $L$-smooth losses and a bounded-domain assumption, Sheaf-FMTL's alternating updates satisfy $\\frac{1}{K}\\sum_{k=0}^{K-1}\\|\\nabla\\Psi(\\theta^k,P^k)\\|^2 \\le \\frac{\\Psi(\\theta^0,P^0)-\\Psi^\\star}{\\rho K}$, an $O(1/K)$ rate to a stationary point of the collaborative objective $\\Psi$. The same objective is shown to recover dFedU-style Laplacian FMTL, personalized FL, hybrid FL, and conventional FL as special cases, which is what makes the method a unified framework rather than a standalone heuristic.","pith_inferences":["Editorial inference: treating the learned norms $\\|P_{ij}\\|_F$ as a task-similarity matrix suggests the communication graph could be pruned or rewired during training, which the paper does not explore.","Editorial inference: because $P_{ij}=0$ is a stationary point of the regularizer, the practical value of the method rests on initialization; adding a norm or rank penalty on $P$ would make collaboration robust and is a direct testable modification.","Editorial inference: the paper measures communication in transmitted bits but does not analyze privacy; a concrete next experiment is to quantify how much a local dataset can be inferred from the disclosed vectors $P_{ij}\\theta_i$.","Editorial inference: the framework is stated for a fixed, connected graph; extending the sheaf to time-varying or asynchronous topologies would test whether the convergence argument survives topology changes."],"forward_implications":["With $d_{ij}=\\lfloor\\gamma\\min(d_i,d_j)\\rfloor$, each client sends vectors of dimension $d_{ij}$ instead of its full model, so small $\\gamma$ yields the reported communication savings of roughly $100\\times$ at comparable accuracy.","The unified formulation means analysis and code for Sheaf-FMTL can be specialized to recover dFedU, personalized FL, and hybrid FL objectives, giving those methods a common convergence proof.","Clients with completely different architectures, where direct parameter averaging is undefined, can still collaborate: the heterogeneous-architecture experiments show Sheaf-FMTL exceeding the local-training baseline.","The method targets cross-silo settings with ample client compute, since storing and updating the $d_{ij}\\times d_i$ restriction maps adds memory and FLOPs even though it removes communication."],"supporting_citations":[{"why":"Supplies the Laplacian-regularized FMTL objective and dFedU baseline that Sheaf-FMTL generalizes and is compared against.","marker":"(Dinh et al., 2022)"},{"why":"Introduces FMTL with a learned task-relationship matrix, the server-based precursor whose convergence rate Sheaf-FMTL matches.","marker":"(Smith et al., 2017)"},{"why":"Provides the personalized-FL objective that the sheaf formulation recovers as a special case.","marker":"(Hanzely & Richtárik, 2020)"},{"why":"Provides the hybrid-FL selection-matrix formulation recovered as a special case.","marker":"(Zhang et al., 2024)"},{"why":"Brings sheaf Laplacian regularization with constraints into distributed optimization, the theoretical basis for the regularizer.","marker":"(Hansen & Ghrist, 2019)"},{"why":"Establishes sheaf theory as a way to organize local observations and model global consistency, motivating the sheaf representation of client interactions.","marker":"(Robinson, 2014; 2013; Riess & Ghrist, 2022)"},{"why":"Supplies the D-PSGD decentralized baseline used in experiments for communication and accuracy comparisons.","marker":"(Lian et al., 2017)"}],"fun_headline_variants":["Sheaves unify heterogeneous federated learning, cut communication","Sheaf-based method saves bits in multi-task federated learning","Cut communication in heterogeneous federated learning with sheaf projections","Sheaf-FMTL: unified framework for heterogeneous federated learning, saves bits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that random nonzero initialization of the interaction matrices keeps them from collapsing to the trivial solution $P_{ij}=0$, where every client trains independently and the collaboration term contributes nothing; the paper's convergence theorem includes that trivial stationary point, so the claimed benefits depend on the learned maps actually staying alive.","fun_headline_variants_meta":{"raw":{"variants":["Sheaves unify heterogeneous federated learning, cut communication","Sheaf-based method saves bits in multi-task federated learning","Cut communication in heterogeneous federated learning with sheaf projections","Sheaf-FMTL: unified framework for heterogeneous federated learning, saves bits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00153,"raw_usage":{"total_tokens":6177,"prompt_tokens":1046,"completion_tokens":5131,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":5058}},"tokens_in":662,"tokens_out":5131,"duration_ms":35911,"temperature":1.0,"reasoning_tokens":5058,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T16:25:04.553037+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"The paper's own Remark 3 shows that zero-initialized interaction matrices freeze the coupling and reduce the method to independent training; so a decisive check is to run with nonzero random initialization on a task pair known to share a subspace and test whether the learned maps stay nonzero and improve accuracy over local training. If the maps collapse to zero or the accuracy matches local training, the central practical claim fails.","supporting_citations":[],"review_version":1}