{"id":"fed2f697-3209-431b-badc-091f0fca87e9","arxiv_id":"2607.27859","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"ZAPs attributes DeFi rewards via percentile-normalized score and a four-layer adversarial defense, reporting 0.923 ensemble ROC-AUC and 30–90% cuts in simulated adversarial reward capture.","lead":"This paper presents ZAPs, a deployed scoring system that decides which crypto wallets get token rewards, combining volume-based scoring with an anomaly-detection stack to cut automated sybil farming. A smart generalist would read it to see how billions of dollars in crypto airdrops are being protected from bots — and how hard it is to verify such claims.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Isolation forest is trained on the same benign wallets used as evaluation negatives, so the +0.032 ensemble gain may reflect transductive leakage rather than real improvement.","rationale":"The reader's weakest assumption focuses on the activity skew of the benign comparison set (median 499 vs 21 transactions), which is a valid and disclosed concern. My read identifies a more fundamental methodological issue: the isolation forest is trained on the exact benign wallets that form part of the evaluation set, so its reported AUC and the ensemble improvement over the reconstruction model are not an unbiased estimate of performance on unseen benign wallets. This is not disclosed in the limitations section. The central claim — that the four-layer adversarial stack, and specifically the parallel anomaly ensemble, improves robustness — depends directly on Table 4 and the ablation in §5.2. If the +0.032 gain is an artifact of transductive leakage, the empirical support for the ensemble advantage collapses, although the single-detector reconstruction model and Proposition 1 remain valid. The paper's other evidence (simulations, production results) is suggestive but lacks the rigor needed to independently sustain the headline claim. This is a fixable flaw, so the appropriate verdict remains conditional: the authors should re-run the evaluation with proper train/test separation of the benign set and with λ selected by cross-validation rather than on the test set. The reader's verdict category is unchanged, but the condition should be sharpened to require this re-evaluation.","tokens_in":12924,"tokens_out":5195,"duration_ms":49732,"concrete_test":"Conduct a leave-benign-out evaluation: split the 383 benign wallets into, say, 70% train / 30% held-out. Fit the isolation forest on the 70% benign split (and the reconstruction model on 70% of malicious as before). Evaluate both detectors and the λ-sweep ensemble on the held-out 30% benign plus the held-out 30% malicious, repeated over 5 seeds. Report the benign-fit isolation forest AUC, the pooled-fit AUC, and the ensemble gain at the λ chosen by inner cross-validation on the training fold (not on the test set). If the benign-fit forest no longer exceeds chance or the ensemble gain falls outside the ±0.013 CI, the headline comparison in Table 4 is invalidated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 / Table 4: the evaluation set is 'the held-out 30% (322 wallets) plus the full benign set' of 383 wallets. The isolation forest's 'benign population' training regime fits on that same benign population, i.e., the 383 benign wallets are in the training set of the isolation forest and in the evaluation set. Thus the reported 0.638 AUC for the benign-fit forest and the +0.032 ensemble gain at λ*=0.90 measure the detector on its own training negatives. This is transductive leakage: the forest has seen the very benign wallets it is scored against. Malicious wallets are held out from the reconstruction model, but benign wallets are not held out from the isolation forest. The paper's limitation section candidly flags the activity skew of the benign set but not this overlap. Under a proper split, the benign-fit isolation forest's standalone AUC and the ensemble gain could shrink or vanish, undermining the claim that the ensemble beats its strongest member and that pooled fitting 'inverts polarity' — the inversion may just reflect the skewed in-sample test set. This is the most load-bearing weakness in the empirical support for adversarial robustness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ZAPs, a reward-attribution framework for DeFi that combines a percentile-normalized composite activity score, a two-layer cross-domain volume weighting, and a four-layer adversarial detection stack with graduated penalties. The central claims are (i) a structural bound on extractable rewards at any protocol (Proposition 1), (ii) an anomaly ensemble that reaches 0.923 ROC-AUC versus 0.891 for the reconstruction model alone when the isolation forest is fit on benign wallets, (iii) a discovered polarity inversion of the isolation forest under pooled fitting, (iv) simulated adversarial reward-capture reductions of 30–90% with 1–8% legitimate-user impact, and (v) live production improvements. The paper is candid about several limitations, including a small activity-skewed benign set, a static quality dimension, and unquantified post-distribution memory, but the central empirical claims are weakened by evaluation-design issues described below.","tokens_in":13274,"tokens_out":3661,"duration_ms":41351,"significance":"If the claims hold, ZAPs would be a useful deployed artifact for a practically important problem: sybil-resistant and adversarial-robust reward attribution in DeFi. The paper's strengths are its explicit architectural description, the honest statement of several limitations in Section 6, and the identification of an interesting empirical phenomenon—the polarity inversion of an isolation forest under pooled fitting (§4.3, §5.2). However, the headline ensemble gain is not currently supported because of a training/evaluation overlap and test-set-based selection of λ. The structural contribution (Proposition 1) is correct but reduces to the definition of the telescoping weight. The work is therefore more of an engineering report with a promising but incompletely validated detection result than a fully established scientific claim at this stage.","major_comments":[{"comment":"The isolation forest is trained on the benign population and evaluated on the same benign set. The evaluation set is 'the held-out 30% (322 wallets) plus the full benign set' of 383 wallets, and the benign-fit isolation forest is explicitly fit on the benign population. This is transductive leakage: the forest has seen the very benign wallets it is scored against. The reported 0.638 AUC for the benign-fit forest and the +0.032 ensemble gain therefore measure the detector on its own training negatives, not on unseen data. A proper split must hold out a benign test fold from the forest's training set. Without this, the claim that the ensemble beats its strongest member and the claim that benign-fit 'inverts polarity' are not established. This is the most load-bearing weakness in the empirical support.","section":"§4.3 / Table 4"},{"comment":"The optimal fusion weight λ* is chosen on the same evaluation set used to report the ensemble AUC. The text reports 'best over sweep' and λ*=0.90, and Figure 5 plots the sweep over the labeled evaluation data. Selecting the fusion weight on the test set and then reporting test performance at that selected value inflates the reported gain and can explain part or all of the +0.032 ROC-AUC difference. The evaluation should use nested cross-validation or a separate validation split for λ, with the test set touched only once.","section":"§4.3 / Figure 5"},{"comment":"The benign comparison set consists of only 383 wallets with median 499 transactions versus a malicious median of 21 transactions. The paper itself notes that the magnitude 'should be expected to depend on the benign sampling frame.' This activity skew is not merely a quantitative detail: the isolation forest's polarity inversion is driven by the benign set being heavy-activity outliers relative to the pooled population, and the standalone 0.638 AUC is measured on that skewed in-sample set. In addition, the labeled corpus is 'drawn from public exploit and phishing attributions,' not airdrop farming, so Table 4 measures transfer to a proxy population. The robustness claims for farming therefore rest on an acknowledged proxy and a non-representative comparison set. A representative benign panel and, ideally, labeled farming wallets are needed before the central robustness claim can be accep","section":"§6 / §4.3"},{"comment":"The proposition is correct but definitional. Its proof uses only sw,p ≤ S and the telescoped identity ωp = Vp/VE from Equation (4). The statement is therefore a restatement of the normalization choices, not an independent structural result. The paper presents this as 'closing the most common structural exploit' and as a formal contribution; this overstates the content. The mechanism may still be useful as a design choice, but the proposition should be presented as a design identity that follows from the definition of ωp, and the discussion should make clear that it does not by itself bound total ecosystem-level extraction, only the per-protocol attribution for a single wallet.","section":"§3.3 / Proposition 1"},{"comment":"The production campaign results are reported as point estimates without uncertainty, sample sizes, baseline definitions, or statistical tests: '56% reduction in sybil allocation, 49% increase in quality-wallet participation, 50% reduction in sell pressure.' Section 6 states that the post-distribution memory mechanism is unquantified and no ablation isolates its contribution. Given that Layer 3 is a component of the full stack, the production attribution of effects to the framework is not supported. The authors should either provide campaign-level detail with error bars and matched baselines or explicitly label these as indicative observations rather than validated effect estimates.","section":"§4.7 / §6"}],"minor_comments":[{"comment":"Many hyperparameters are withheld: α, β, γ, τ, T*, λ, penalty breakpoints, and clustering thresholds. Withholding parameters is understandable for a deployed system, but it prevents independent replication. At minimum, report a sensitivity analysis over plausible ranges, or release anonymized feature vectors and evaluation code.","section":"§3.2–3.5"},{"comment":"The table reports λ=1.00 as 'equivalent to PA alone' but does not include a λ=0 row for the isolation forest alone. Since Section 5.2 discusses the cost of removing the reconstruction model, the λ=0 standalone AUCs should be shown explicitly.","section":"§4.3 / Table 4"},{"comment":"The simulation results are point estimates. They should be accompanied by variance across seeds or scenario perturbations, especially because the 'change' percentages are the central quantitative evidence for the asymmetry claim.","section":"§4.4 / Table 5"},{"comment":"The statement that removing the reconstruction model 'costs 0.285' is not directly readable from Table 4. Add the λ=0 row or compute the difference from the shown numbers explicitly.","section":"§5.2"},{"comment":"The coefficient of variation uses sample mean in the denominator. For attributes with near-zero means this is unstable; state the handling of such cases (e.g., a floor or exclusion).","section":"Equation (11)"}],"recommendation":"major_revision","confidential_remarks":"The paper is an industry-authored deployment report with a useful architectural description and an interesting, clearly stated empirical observation about isolation-forest training regime. However, the headline detection claim is compromised by a training/evaluation overlap and test-set selection of λ; these are fixable with a proper re-evaluation. The formal proposition is correct but trivial. I would not reject, but I would require a re-analysis with disjoint benign training/test split, nested validation for λ, and a representative benign sample before the central claims can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague—\n\nZAPs is close to the best kind of industry paper: a real deployed system, a clear write-up, and an explicit account of where it can go wrong. The headline result—that the anomaly ensemble beats the reconstruction model alone (0.923 vs 0.891 AUC) only when the isolation forest is trained on benign data, and that pooled training inverts its polarity entirely—is genuinely interesting and, as far as I know, not in the literature. The paper earns credit for reporting that inversion clearly rather than hiding it.\n\nWhat's actually new: the composite reward score with percentile caps, the nested two-layer weighting that telescopes to V_p/V_E and makes niche-protocol farming structurally unprofitable (Proposition 1 is definitional, but the framing is useful), and the four-layer defense stack feeding graduated penalties instead of binary exclusion. The limitations section is unusually honest—it flags the static quality dimension, the small and activity-skewed benign set, and the constructed scenarios.\n\nThe soft spots are real and central. The evaluation in §4.3/Table 4 trains the isolation forest on the full benign population (383 wallets) and then evaluates it on that same population plus held-out malicious wallets. That's transductive leakage: the benign-fit forest has seen its evaluation negatives. The reported 0.638 AUC and the +0.032 ensemble gain could shrink or disappear under a proper holdout of benign wallets. This is the most load-bearing empirical claim, and it needs fixing before the paper can be relied on. The benign set skew (median 499 vs 21 transactions) compounds the problem, as the paper itself concedes. Add to that the withheld mixing coefficients, λ, and penalty breakpoints, and no independent replication is possible.\n\nNone of this kills the framework—the structural defenses against niche farming and the layered detection stack are sensible, and the live campaign numbers are suggestive even if uncontrolled. But the paper's own evidence for adversarial robustness is conditional on an evaluation design that leaks. If I'm refereeing, I'd ask for a proper split of the benign set, a clearer statement of the leakage issue, and ideally code or data.\n\nIt deserves peer review—the deployed scale and the polarity-inversion finding merit referee time. But it needs major revision, not a quick accept.\n\nBest,","headline":"A genuinely useful deployed DeFi reward-attribution framework, but the headline ensemble gain rests on isolation-forest evaluation that leaks its training negatives—fixable, but central.","tokens_in":13740,"tokens_out":2620,"would_cite":true,"duration_ms":28136,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ZAPs shows that DeFi reward systems can cap rewards by a protocol's global volume share and layer anomaly detection on top, cutting adversarial capture by 30–90% while barely affecting legitimate users.","keywords":["DeFi incentives","reward attribution","sybil resistance","anomaly detection","isolation forest","one-class autoencoder","airdrop farming","cross-domain weighting"],"falsifier":"Re-run the isolation forest on a benign sample frequency-matched to the malicious wallets on transaction count and activity distribution; if the 0.250 inversion does not reproduce, or if the ensemble gain at λ=0.90 vanishes, the central robustness claim fails. An independent labeled corpus of 1,073 farming (not exploit) wallets would also test whether the 0.923 AUC transfers from phishing/exploit attributions to the actual adversarial population of interest.","tokens_in":12854,"feed_emoji":"🛡️","tokens_out":2934,"duration_ms":31148,"temperature":0.7,"pith_summary":"ZAPs asks whether a reward-attribution mechanism can pay genuine economic contribution in proportion to ecosystem significance while resisting bots and sybils that manufacture that contribution. It answers yes by combining two structural safeguards — a composite activity score whose components are each bounded, and a two-layer weight that telescopes into a protocol's global volume share, capping any single wallet's extractable reward via Proposition 1 — with a four-layer behavioral defense whose core is a parallel anomaly ensemble. The headline empirical evidence is a held-out ROC-AUC of 0.923±0.013 for the ensemble versus 0.891±0.016 for the reconstruction model alone, with an unusual training-regime finding: fitting the isolation forest on the pooled population inverts its polarity (0.250 AUC) and destroys the ensemble gain, while fitting it on the benign population works. A sympathetic reader would care because billions in token distributions are currently paid out against naive volume and count heuristics that industrial farming exploits; ZAPs offers a deployed, measured alternative that is robust by construction, not by retrofit.","feed_headline":"DeFi reward framework cuts sybil capture by 30–90%","feed_subtitle":"A volume-share cap and an anomaly ensemble block farming bots while leaving honest users nearly untouched.","key_machinery":"The central mechanism is a composable attribution algebra: a bounded per-(wallet, protocol) composite score s_{w,p} (Equation 1) whose components are individually capped via percentile anchoring and a time ceiling, multiplied by a telescoping cross-domain weight ω_p = V_p/V_σ(p) · V_σ(p)/V_E = V_p/V_E (Equation 4). This weight is what enacts Proposition 1's bound on extractable reward. The second key object is the parallel anomaly ensemble of Section 3.5.2: Detector A is a one-class reconstruction autoencoder with an over-complete latent space (10→64→16→64→10), trained only on malicious wallets so that low reconstruction error signals maliciousness; Detector B is an isolation forest, whose t","core_discovery":"The paper claims that reward attribution in DeFi can be made resistant to adversarial manufacturing by making the scoring function itself the defense. Each wallet gets a composite score with three bounded, additive components: percentile-anchored volume (capped at a high population percentile), engagement duration (capped), and a quality signal. A two-layer cross-domain weight multiplies protocol share within sector by sector share within the ecosystem, telescoping to the protocol's global volume share; Proposition 1 proves that no wallet can extract more than S·V_p/V_E from any protocol, regardless of how cheaply local dominance is bought. On the behavioral side, a four-layer stack (transac","pith_inferences":["Editorial inference: the polarity-inversion finding may generalize beyond DeFi — any unsupervised anomaly detector used in an adversarial, class-imbalanced setting should be fit on the population that defines 'normal' for the purpose at hand, rather than on whatever data is available.","Editorial inference: because the paper admits the quality dimension q_{w,p} is static in the reported deployment, the actual distributional statistics likely understate what the full framework would achieve if sector-specific quality signals were activated.","Editorial inference: a natural testable extension is to apply ZAPs-style weighting to quadratic funding or retroactive public-goods funding, where the same tension between capital-weighted and participant-weighted allocation exists and where a protocol-level volume-share cap could be replaced by a different ecosystem-significance metric.","Editorial inference: the reported 30–90% adversarial capture reduction comes from constructed scenarios; a stronger follow-up would be to run a live randomized trial comparing ZAPs against a naive volume formula on matched populations, measuring both sybil capture and legitimate-user retention."],"forward_implications":["If the central claim holds, any volume-based reward program can structurally eliminate the niche-protocol farming exploit by applying the telescoping weight of Equation 4, since extractable reward at a protocol is bounded by that protocol's global volume share.","The isolation-forest training regime becomes a deployment requirement: fit the detector on the benign population, never on pooled production traffic, or the detector actively inverts and degrades the ensemble.","The graduated penalty structure implies that false-positive harm is bounded: a legitimate wallet misclassified by one tier keeps roughly half its reward instead of losing everything, which is what allows the 1–8% legitimate-user impact.","The framework's composability means the same attribution algebra can be extended to other permissionless incentive systems (airdrops, points, liquidity mining) with the same adversarial assumptions.","In live production, the framework reports a 56% reduction in sybil allocation and a 49% increase in quality-wallet participation, which suggests measurable economic benefit beyond detection accuracy."],"fun_headline_variants":["DeFi reward framework cuts sybil capture by up to 90%","ZAPs caps DeFi rewards, slashes bot farming 30–90%","Parallel anomaly ensemble boosts DeFi fraud detection to 0.923 AUC","DeFi's anti-sybil stack: 56% less sybil, 49% more quality wallets","Reward attribution that bounds bots: DeFi's ZAPs framework"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The empirical results rest on a benign comparison set of only 383 wallets with a median of 499 transactions versus 21 for malicious wallets; if a representative benign panel changes which wallets are statistical outliers, the isolation forest's polarity inversion and the ensemble's +0.032 AUC gain could shrink or disappear.","fun_headline_variants_meta":{"raw":{"variants":["DeFi reward framework cuts sybil capture by up to 90%","ZAPs caps DeFi rewards, slashes bot farming 30–90%","Parallel anomaly ensemble boosts DeFi fraud detection to 0.923 AUC","DeFi's anti-sybil stack: 56% less sybil, 49% more quality wallets","Reward attribution that bounds bots: DeFi's ZAPs framework"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000789,"raw_usage":{"total_tokens":3351,"prompt_tokens":817,"completion_tokens":2534,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":2427}},"tokens_in":561,"tokens_out":2534,"duration_ms":17313,"temperature":1.0,"reasoning_tokens":2427,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T00:08:28.254313+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the isolation forest on a benign sample frequency-matched to the malicious wallets on transaction count and activity distribution; if the 0.250 inversion does not reproduce, or if the ensemble gain at λ=0.90 vanishes, the central robustness claim fails. An independent labeled corpus of 1,073 farming (not exploit) wallets would also test whether the 0.923 AUC transfers from phishing/exploit attributions to the actual adversarial population of interest.","supporting_citations":[],"review_version":1}