{"id":"1434c09e-5727-401d-bf76-a56b0631ecb5","arxiv_id":"2606.13180","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"JiRAIYA introduces a hierarchical FL framework on Web3 using delegated managers, novelty detection, consensus, and reputation scores to improve transparency and attack resilience.","lead":"The paper proposes JiRAIYA, a reputation-based hierarchical federated learning framework that integrates Web3 technologies for greater transparency, trust, and auditability in model training. A smart generalist might read it to understand how decentralized ledger systems could extend collaborative AI training beyond closed corporate environments.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Consensus among managers may not mitigate poisoning if malicious majority or weak novelty detection; no formal threat model or attack simulations provided.","rationale":"Reader's weakest assumption directly identifies the same gap; full text does not appear to close it with formal analysis or targeted attack results, so the UNVERDICTED verdict should move to CONDITIONAL pending the concrete test.","tokens_in":1731,"tokens_out":336,"duration_ms":8485,"concrete_test":"Re-run the attack experiments from §5 with an explicit 30% malicious manager fraction and two standard poisoning strategies (label-flipping and model-replacement); report the fraction of poisoned updates accepted into the global model. If acceptance rate exceeds 5% while maintaining comparable accuracy on clean data, the mitigation claim does not hold.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that broadcasting encoded updates to managers, combined with independent novelty detection + consensus, reliably filters adversarial/poisoning updates (and that reputation backup ensures continuity). Section 3 describes the mechanism at high level but provides neither a threat model (e.g., fraction of Byzantine managers, attack types like model replacement or backdoor) nor quantitative evaluation of detection rates under those attacks. Experiments are claimed to be \"under real world scenarios\" yet the description gives only aggregate effectiveness/resilience metrics without reporting attack success rates, false-positive rates on benign updates, or ablation on the novelty detector. If the consensus threshold or novelty threshold is set too loosely, or if managers can collude, poisoned updates can still be incorporated; the reputation mechanism does not address this at the validation step.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes JiRAIYA, a hierarchical federated learning framework that uses Web3 technologies (including blockchain for transparency) to address opacity in traditional FL aggregation. Delegated managers orchestrate training within federations; model updates are encoded and broadcast to all managers, who apply novelty detection and reach consensus to accept or reject updates before incorporation into the global model. A reputation-score-based backup mechanism ensures continuity. The authors claim that extensive experiments under real-world scenarios demonstrate the framework's effectiveness and resilience against adversarial and poisoning attacks, enabling transparent FL beyond enterprise settings.","tokens_in":1905,"tokens_out":540,"duration_ms":10542,"significance":"If the central claims on attack mitigation and real-world performance hold, the work would offer a concrete path to auditable, decentralized FL by repurposing existing Web3 primitives rather than introducing new external validators. The hierarchical manager design and reputation backup are pragmatic engineering choices that could reduce single points of failure, but the absence of quantitative attack evaluations limits the immediate impact on the FL security literature.","major_comments":[{"comment":"Section 3 (framework description): the consensus-plus-novelty-detection mechanism is presented at a high level without a formal threat model (e.g., Byzantine fraction among managers, model-replacement or backdoor attack definitions, or assumptions on manager collusion). This is load-bearing because the central claim that broadcasting updates plus independent evaluation reliably filters poisoning attacks rests on unstated security assumptions.","section":"Section 3"},{"comment":"Experiments section (and abstract): the manuscript asserts 'extensive experiments conducted under real world scenarios' demonstrate resilience, yet reports only aggregate effectiveness metrics. No attack success rates, false-positive rates on benign updates, ablation on the novelty detector, or comparison against baselines under explicit poisoning are provided. This undermines the resilience claim that is central to the contribution.","section":"Experiments section"}],"minor_comments":[{"comment":"Abstract: 'broad casted' should be 'broadcast'; 'the effectiveness, resilience' is missing 'and'.","section":"Abstract"},{"comment":"Notation for reputation scores and consensus thresholds is introduced without a clear table or equation reference, making it difficult to reproduce the exact validation logic.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper reads primarily as a system proposal; the lack of any quantitative security evaluation or threat-model section makes it a borderline fit for a venue expecting rigorous empirical validation of security claims. Citation of prior Web3-FL works appears thin."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address the two major comments point by point below and commit to revisions that directly strengthen the security analysis and experimental evidence.","responses":[{"response":"We agree that a formal threat model is necessary to rigorously ground the security claims. In the revised manuscript we will insert a dedicated threat-model subsection in Section 3 that explicitly states (i) the assumed Byzantine fraction among managers, (ii) definitions of model-replacement and backdoor attacks, and (iii) collusion assumptions. This will clarify how the broadcast-plus-consensus design is intended to filter poisoning under those assumptions.","revision_made":"yes","referee_comment":"[Section 3] Section 3 (framework description): the consensus-plus-novelty-detection mechanism is presented at a high level without a formal threat model (e.g., Byzantine fraction among managers, model-replacement or backdoor attack definitions, or assumptions on manager collusion). This is load-bearing because the central claim that broadcasting updates plus independent evaluation reliably filters poisoning attacks rests on unstated security assumptions."},{"response":"We accept that the current experimental presentation does not supply the quantitative attack metrics needed to substantiate the resilience claims. In the revision we will augment the experiments section with (i) attack success rates under poisoning and backdoor scenarios, (ii) false-positive rates on benign updates, (iii) ablation results isolating the novelty detector, and (iv) direct comparisons against standard FL baselines under the same attack models. These additions will be reported alongside the existing aggregate metrics.","revision_made":"yes","referee_comment":"[Experiments section] Experiments section (and abstract): the manuscript asserts 'extensive experiments conducted under real world scenarios' demonstrate resilience, yet reports only aggregate effectiveness metrics. No attack success rates, false-positive rates on benign updates, ablation on the novelty detector, or comparison against baselines under explicit poisoning are provided. This undermines the resilience claim that is central to the contribution."}],"tokens_in":1364,"tokens_out":430,"duration_ms":13783,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper outlines a new architecture called JiRAIYA for running federated learning on Web3. Delegated managers handle local federations, model updates get encoded and broadcast for independent novelty detection, consensus among managers decides acceptance, and a reputation score acts as backup to keep training going. The goal is more transparency and auditability without extra external validators.\n\nWhat stands out as new is the specific layering: Web3 broadcast plus manager-level consensus to replace opaque central aggregation. It builds on known pieces like novelty detection and reputation but ties them to a hierarchical Web3 structure aimed at non-enterprise settings.\n\nThe idea addresses a genuine pain point in current FL systems around trust and visibility. Using existing Web3 mechanisms for encoding and agreement is a reasonable direction if the overhead stays manageable.\n\nThe soft spots are substantial and sit right at the center. The abstract states that extensive real-world experiments show effectiveness and resilience against adversarial and poisoning attacks, yet the text gives no methods, datasets, metrics, error bars, or attack success rates. There is also no threat model describing the fraction of malicious managers, attack types like model replacement, or how the novelty threshold and consensus rules perform under those conditions. Without those, it is impossible to judge whether the consensus step actually filters bad updates or just adds communication cost.\n\nThe reputation backup helps with continuity but does not strengthen the validation step itself. If the paper has more in the full text, that would change the picture, but based on what is here the claims rest on assertion rather than shown results.\n\nThis is for people already working at the intersection of blockchain and distributed ML who want architecture sketches. It does not yet have the grounding for a serious referee process. I would not send it to peer review until the experimental evaluation and threat analysis are added and the results are reported in detail.","headline":"JiRAIYA sketches a Web3 hierarchical FL setup with manager consensus and novelty checks but supplies no data, threat model, or attack results to support its resilience claims.","tokens_in":2390,"tokens_out":457,"would_cite":false,"duration_ms":15261,"reading_group":"no","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A Web3 hierarchy lets delegated managers reach consensus on encoded model updates to keep federated learning transparent and resistant to poisoning.","keywords":["federated learning","web3","hierarchical architecture","consensus mechanism","reputation system","novelty detection","model poisoning","transparency"],"falsifier":"An experiment in which a poisoning attack succeeds in altering the global model even after the consensus and novelty checks, or where the measured communication volume exceeds that of validator-based alternatives.","tokens_in":2629,"feed_emoji":"","tokens_out":544,"duration_ms":13778,"temperature":0.7,"pith_summary":"The paper introduces a federated learning system built on Web3 tools to address opaque aggregation and limited auditability in enterprise FL. It organizes participants into federations run by delegated managers who broadcast encoded updates, evaluate them independently with novelty detection, and accept only those that pass consensus. A reputation-based backup keeps model generation running if primary paths fail. Real-world experiments test resilience against adversarial attacks. The design aims to move reliable FL into open, decentralized settings without relying on external validators.","feed_headline":"Managers reach consensus on model updates via Web3 broadcast","feed_subtitle":"Novelty detection and reputation backup replace external validators in this hierarchical FL design.","key_machinery":"Hierarchical architecture of delegated managers that broadcast encoded updates for independent consensus evaluation combined with novelty detection and reputation-based backup.","core_discovery":"Model updates are encoded and broadcast to all managers, who independently evaluate their validity using novelty detection; updates approved by consensus are incorporated into the global model, while a reputation score mechanism provides backup to ensure continued model generation.","pith_inferences":["The same broadcast-and-consensus pattern could be extended to record an immutable training log on a blockchain for post-hoc audits.","Open participant pools might grow larger than in traditional FL because evaluation is distributed rather than centralized.","If novelty detection parameters are tuned per federation, the approach could adapt to domain-specific data distributions without global retraining."],"forward_implications":["Every accepted model update becomes visible and auditable by all managers in the hierarchy.","Training can continue without external validators because managers handle evaluation internally.","Reputation scores allow the system to fall back to reliable participants when others drop out.","The framework supports FL participation beyond closed enterprise networks by leveraging Web3 transparency."],"fun_headline_variants":["Web3 broadcast enables FL consensus without validators","Reputation scores ensure FL model continuity on Web3","Managers independently validate Web3 FL updates via consensus","Novelty detection secures hierarchical FL on Web3"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Broadcasting updates to multiple managers for consensus evaluation plus novelty detection can filter out bad updates effectively while keeping communication costs low.","fun_headline_variants_meta":{"raw":{"variants":["Web3 broadcast enables FL consensus without validators","Reputation scores ensure FL model continuity on Web3","Managers independently validate Web3 FL updates via consensus","Novelty detection secures hierarchical FL on Web3"]},"model":"grok-4.3","cost_usd":0.003666,"raw_usage":{"total_tokens":1870,"prompt_tokens":592,"num_sources_used":0,"completion_tokens":56,"cost_in_usd_ticks":36662000,"prompt_tokens_details":{"text_tokens":592,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1222,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":592,"tokens_out":56,"duration_ms":7061,"temperature":1.0,"reasoning_tokens":1222,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T05:58:45.274766+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment in which a poisoning attack succeeds in altering the global model even after the consensus and novelty checks, or where the measured communication volume exceeds that of validator-based alternatives.","supporting_citations":[],"review_version":1}