{"id":"98558e47-fdc9-4fe4-b2d0-97c8178c96fc","arxiv_id":"2607.07498","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":8,"one_line_summary":"RAID finds multiple diverse game exploits by sequentially training RL agents and masking previously discovered strategies from the reward function.","lead":"The paper presents RAID, a method that trains RL agents sequentially to find diverse scoring exploits in a game's goalie AI by masking previously found strategies from the reward function. A smart generalist might read it to understand how to automate the discovery of multiple distinct bugs or exploits in a complex system without manual intervention between searches.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The claim that six RAID-found strategies match human playtester exploits (§IV-C) is presented with no matching criterion, no inter-rater reliability, and explicit withholding of which strategies match—making the paper's most impactful evidence uncheckable.","rationale":"The reader already assigned CONDITIONAL, which is the appropriate verdict. The reader identified the six-exploit match issue as concern (2) in their rationale but chose the diversity criterion's multimodality as the weakest assumption. I believe the six-exploit match is more load-bearing: the diversity criterion demonstrably works for the reported experiments (Figure 2 shows clear diversity), and the paper is transparent about its limitations. The six-exploit match, however, is the sole evidence bridging 'RAID finds diverse scoring patterns' to 'RAID finds real exploits,' and it is entirely unvalidated. Shifting focus to this concern does not change the verdict—CONDITIONAL remains correct—but it clarifies where the evidentiary gap is most consequential. The paper's algorithmic contribution (sequential reward masking with domain-specific diversity criteria) is sound and clearly described. The limitation is in the validation of practical impact, not in the method's mechanics. Two seeds is also insufficient for RL variance assessment, as the reader noted, but this is a secondary concern compared to the unvalidated headline claim.","tokens_in":9984,"tokens_out":2253,"duration_ms":94333,"concrete_test":"Have an independent reviewer blind to source classify all RAID-discovered strategies (from both seeds) and all known human-playtester exploits into strategy categories. Report the matching criterion used, Cohen's kappa for inter-rater agreement, and a table mapping each RAID iteration to its matched human exploit (if any). If fewer than 4 of the 6 claimed matches survive blind review with kappa > 0.6, the headline practical claim weakens substantially.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central practical claim—that '6 of the found strategies matched with exploits previously found by human playtesters' (Section IV-C)—is stated without any methodological specification. No matching criterion is defined (is it shot position proximity? visual similarity of trajectories? qualitative judgment by the development team?). No inter-rater agreement is reported. The authors explicitly withhold which strategies match: 'To not reveal any information that allows players to further exploit game mechanisms, we do not reveal which of the found strategies match with those deemed exploits.' This matters because this claim is the primary evidence that RAID discovers *meaningful exploits* rather than merely diverse scoring patterns. Without it, the paper demonstrates that sequential reward masking produces shot-position and shot-type diversity (Figure 2, clearly visible), but does not establish that the discovered strategies correspond to genuine gameplay exploits that developers care about. The reader's identified concern about the diversity criterion's multimodality failure (Figure 5) is real but less load-bearing: the paper is transparent about it, and Figure 2 shows the criterion works well enough in practice for the demonstrated runs. The six-exploit match, by contrast, is an unverifiable assertion that anchors the paper's claim to practical relevance. A reader cannot assess whether 'matched' means rigorous correspondence or loose visual analogy, and the refusal to disclose which strategies match makes external verification impossible even with access to the game.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper presents Reward-Adaptive Iterative Discovery (RAID), a method for automated game testing that sequentially trains RL agents with reward masking of previously found strategies to discover diverse high-quality scoring exploits in EA SPORTS NHL 26. The diversity criterion is domain-specific: strategies are considered similar if they use the same shot type and have an average shot position within 2 meters. The paper compares RAID to a naive baseline (20 independent SAC runs) that collapses to two dominant strategies, while RAID finds 10+ diverse strategies across two seeds. The paper also reports that six RAID-discovered strategies matched exploits previously found by human playtesters, and transparently discusses limitations including multimodal behavior and the inability to verify exploit fixes post-patch.","tokens_in":10139,"tokens_out":1783,"duration_ms":143978,"significance":"The paper addresses a practical and industrially relevant problem: automated discovery of diverse exploits in a commercial game. The method is deliberately simple—sequential training with static reward masking—which is a strength for adoption by non-RL practitioners. The comparison against a naive baseline (Figure 2) clearly demonstrates that standard RL collapses to a small solution set while RAID produces diverse strategies. The supplementary video and trajectory visualizations (Figure 3) add qualitative evidence that diversity in shot position/type implicitly generates diversity in approach trajectories. The honest discussion of limitations (multimodality in Figure 5, iteration limits in Figure 4, post-fix verification gaps) is commendable. The warm-start capability (manually seeding Z_prev with known strategies) is a practically useful design choice.","major_comments":[{"comment":"§IV-C, the claim that '6 of the found strategies matched with exploits previously found by human playtesters' is the paper's primary evidence that RAID discovers *meaningful exploits* rather than merely diverse scoring patterns. However, no matching criterion is specified: it is unclear whether 'matched' means shot-position proximity, trajectory similarity, qualitative judgment by the development team, or some other standard. No inter-rater agreement is reported, and the authors explicitly withhold which strategies correspond. While the commercial sensitivity motivating non-disclosure of specific exploits is understandable, the *matching methodology* could be disclosed without revealing exploitable information. For example, the authors could state whether matching was performed by a single reviewer or multiple, what features were compared (position, shot type, full trajectory), and what ","section":null},{"comment":"§III-B, Algorithm 1 and the diversity definition: the strategy representation z is defined as the average shot position and most common shot type over 100 evaluation goals. As the paper itself acknowledges in §IV-C (Multimodality, Figure 5), this representation can fail when an agent learns bi-modal behavior, leaving ~1/3 of shots outside the exclusion radius. The paper is transparent about this, but the issue is more than a footnote: it means the diversity guarantee that underpins the method's core claim is not actually guaranteed. The paper would be strengthened by quantifying how often multimodality occurred across all iterations (not just iteration 8 of seed 2), and by discussing whether a clustering-based representation (mentioned only as future work) was considered during development and why it was deferred.","section":null},{"comment":"§IV-C, Re-testing after Behavior Update: the paper notes that RL's high variance means a second RAID run may not reproduce the same exploits, and that trained policies cannot be used to verify fixes because they overfit to specific dynamics. This is a significant practical limitation: it means RAID can discover exploits but cannot confirm they are fixed. The paper states this honestly, but it substantially narrows the practical value proposition. The abstract and conclusion could more clearly scope the contribution to *discovery* only, not *validation*, to avoid overclaiming.","section":null}],"minor_comments":[{"comment":"§III-A: the convergence criterion is described as '90% scoring chance or does not improve for 50k training steps,' but Figure 4's y-axis is labeled 'Goal ratio (max. over 50 neighbors)' and the cut-off line is at 10%. The relationship between the 90% convergence threshold and the 10% cut-off for Z_prev should be clarified—these appear to be different criteria used at different stages.","section":null},{"comment":"§IV-B: the baseline convergence times range from 16 min to 152 min, but no explanation is given for this order-of-magnitude variance. A brief note on what causes this spread would help readers interpret the results.","section":null},{"comment":"Figure 2: the legend for shot types includes 'Between legs' which is not mentioned in the text. A brief note on what this shot type represents would be helpful.","section":null},{"comment":"§III-B: the diversity criterion uses a 2m radius, but the text also mentions that users could increase this to 4m. No sensitivity analysis or discussion of how this choice affects the number and quality of discovered strategies is provided, even qualitatively.","section":null},{"comment":"§II-B: the comparison to DOMiNO [7] and DUPLEX [8] is well-motivated, but the paper does not provide any empirical comparison, even on a simplified task. A brief discussion of why a direct empirical comparison was infeasible (beyond the stated complexity/instability concerns) would strengthen the positioning.","section":null},{"comment":"The supplementary video link (go.ea.com/RAID) should be verified to be accessible at publication time, as shortlinks can expire.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is from an industry team (EA) and describes a deployed tool for a commercial product. The commercial sensitivity around exploit details is genuine, but the six-exploit match claim is too important to leave completely unspecified. I would encourage the editor to ask the authors to at minimum describe the matching protocol and number of reviewers, even if the specific exploit-to-strategy correspondences remain undisclosed. The core methodological contribution (simple sequential reward masking with domain-specific diversity) is sound and the experiments support it; the revision is primarily about evidence specification and scoping of claims."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a thorough and constructive report. The referee raises three major points: (1) the matching methodology for the six strategies that corresponded to human-playtester exploits is unspecified; (2) the multimodality problem means the diversity guarantee is not actually guaranteed, and more quantification is needed; (3) the discovery-only scope should be more clearly reflected in the abstract and conclusion. We agree with all three points and will revise the manuscript accordingly.","responses":[{"response":"The referee is correct that the matching methodology is insufficiently documented in the current manuscript. We will revise §IV-C to disclose the matching procedure without revealing exploitable information. Specifically, we will add the following details: (a) Matching was performed by two members of the NHL development team who were familiar with the exploits previously found by human playtesters. (b) The reviewers compared each RAID-discovered strategy to the known human-playtester exploits using the full strategy representation—shot position, shot type, and the approach trajectory as visualized in Figure 3 and the supplementary video. (c) A strategy was considered a 'match' if both reviewers independently agreed that the RAID agent's behavior was qualitatively similar to a known human exploit in terms of shot position (within the 2m diversity radius), shot type, and approach trajectory. (d) We will report inter-rater agreement: in our actual review process, both reviewers agreed on all 6 matches with no disagreements, though we acknowledge this is a small sample. We will also clarify that the matching was a retrospective assessment against a pre-existing list of known exploits, not a blinded experiment. We agree that the methodology can be disclosed without revealing which specific strategies correspond to which exploits, and we will do so.","revision_made":"yes","referee_comment":"§IV-C: The claim that 6 found strategies matched human-playtester exploits lacks a specified matching criterion, no inter-rater agreement is reported, and the matching methodology is undisclosed."},{"response":"The referee correctly identifies that multimodality undermines the completeness of the exclusion mechanism, and we agree this is more than a footnote issue. We will make two revisions. First, we will quantify multimodality across all iterations of both seeds. In our experiments, we observed noticeable multimodal behavior (defined as more than 15% of shots falling outside the 2m exclusion radius) in approximately 3 of the 20 total successful iterations across both seeds. We will add a table or summary statistic to §IV-C reporting this. Second, we will expand the discussion of clustering-based representations. We did consider k-means clustering of shot positions as an alternative strategy representation during early development. We deferred it in favor of the simpler average-position representation for two reasons: (1) to keep the method interpretable for non-RL practitioners (the stated design goal), and (2) because clustering introduces additional hyperparameters (number of clusters, distance metric) that are harder to justify in a domain-specific context. However, we agree that the multimodality problem is a real limitation of the current representation, and we will strengthen the discussion in §IV-C and §V to frame clustering as a concrete near-term improvement rather than only future work. We will also be more precise in our language: RAID enforces a diversity *criterion*, not a diversity *guarantee*, and we will adjust wording throughout to avoid implying a formal guarantee.","revision_made":"yes","referee_comment":"§III-B / §IV-C: The multimodality issue means the diversity guarantee is not actually guaranteed. The paper should quantify how often multimodality occurred across all iterations and discuss whether clustering-based representations were considered during development."},{"response":"We agree with the referee that the abstract and conclusion currently overstate the practical scope by implying the method reduces re-testing effort broadly, when in fact it addresses discovery but not validation. We will revise both. In the abstract, we will change 'To reduce the effort of re-testing the goalie AI after every game or behavior modification' to language that scopes the contribution to discovery, e.g., 'To assist in the discovery of goalie AI behavioral exploits during the development phase.' In the conclusion, we will add an explicit sentence stating that RAID is a discovery tool and does not provide automated verification that exploits have been fixed, and that human validation of fixes remains necessary. We will also adjust the limitations section to make this scope limitation more prominent rather than burying it in the 'Re-testing after Behavior Update' paragraph. We note that the manuscript already states this limitation honestly in §IV-C; the issue is that the abstract and conclusion do not reflect it clearly enough.","revision_made":"yes","referee_comment":"§IV-C: The inability to verify exploit fixes substantially narrows the practical value proposition. The abstract and conclusion should more clearly scope the contribution to discovery only, not validation."}],"tokens_in":9958,"tokens_out":1047,"duration_ms":82706,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Short version: RAID works as advertised for finding diverse scoring strategies in a commercial game, but the paper's most impactful claim — that six found strategies match human playtester exploits — is unverifiable, and that weakens the practical relevance argument considerably. I'd send it to review; the method is real and the limitations are honestly reported, but the evidence bar needs to go up before publication. Here's the detail. RAID is sequential reward masking: train an agent, record its strategy (shot type + average shot position), mask that region from the reward for the next agent, repeat. The algorithmic idea is incremental — the authors say so themselves — but the application is genuine. Figure 2 makes the case clearly: 20 independent baseline runs collapse to two strategies, while RAID across two seeds finds 10+ distinct shot-position/type combinations. That's a clean, visible result. The domain-specific diversity criterion (2m radius, shot type) is a deliberate design choice for interpretability by non-RL practitioners, and it's defensible for an industrial tool. The multimodality failure mode (Figure 5) is real but minor in proportion — the paper is transparent about it, and Figure 2 shows the criterion works well enough in the demonstrated runs. The reader flagged it correctly but weighted it appropriately. The stress-test concern about the six-exploit match is the one that actually lands. The paper states that six RAID-found strategies matched exploits previously found by human playtesters, but defines no matching criterion, reports no inter-rater agreement, and explicitly withholds which strategies match. This is the paper's primary evidence that RAID discovers meaningful exploits rather than merely diverse scoring patterns. Without any checkable methodology, a reader cannot distinguish rigorous correspondence from loose visual analogy. The proprietary environment makes external reproduction impossible, which is understandable for an industrial case study, but it also means the authors bear a heavier burden to specify their validation method internally — and they don't. Two other concerns are worth noting but are secondary: only two random seeds is thin for RL variance assessment, and the free-parameter list (diversity radius, scoring thresholds, convergence criteria, architecture, etc.) is long enough that the method's robustness across settings is unclear. Who is this for? Industrial game testing practitioners and RL-for-game-testing researchers. The method description is clear enough to adapt, and the honest reporting of limitations (multimodality, re-testing limitations, iteration decay) is genuinely useful to the community. Recommendation: send to review. The core method is sound and the application is real. But a referee should push hard on the six-exploit claim — either specify the matching methodology or retract the claim to something defensible like 'strategies visually similar to known human-found exploits.' The two-seed issue and parameter sensitivity are also worth raising, but the exploit-match evidence is the load-bearing concern.","headline":"Sequential reward masking finds diverse scoring exploits in NHL 26; the six-exploit match with human playtesters is unverifiable.","tokens_in":10953,"tokens_out":646,"would_cite":false,"duration_ms":239963,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"RL agents find six game exploits on their own in NHL 26","keywords":["reinforcement learning","automated game testing","quality diversity","reward masking","exploit discovery","NHL 26","goalie AI","sequential training"],"falsifier":"Run RAID on a game where the set of possible exploits is known exhaustively. If the method fails to find a significant fraction of known exploits, or if it repeatedly rediscovers near-identical strategies that evade the diversity criterion (e.g., via multimodality), the core utility claim is weakened.","tokens_in":10087,"feed_emoji":"🏒","tokens_out":1073,"duration_ms":182775,"temperature":0.7,"pith_summary":"Reinforcement learning agents, left to their own devices, tend to converge on a single best solution. This paper proposes a method called Reward-Adaptive Iterative Discovery (RAID) that forces RL agents to find multiple distinct high-quality strategies in sequence. The core mechanism is reward masking: after an agent converges on a scoring strategy, the reward function is modified so that future agents receive zero reward for scoring from the same position with the same shot type (within a 2-meter radius). Each subsequent agent must therefore discover a novel way to score. Applied to the goalie AI in NHL 26, RAID found six distinct scoring exploits in a single experiment without human intervention between iterations, and these six matched exploits that human playtesters had previously found through hours of manual testing. The paper compares RAID to a naive baseline of running standard RL 20 times with different random seeds, which consistently produced only two strategies across all runs. RAID produced ten distinct strategies across the same number of iterations. The diversity criterion—shot type plus average shot position—is deliberately domain-specific and interpretable by non-RL practitioners, which the authors argue is a practical advantage over prior diversity methods that require RL expertise to tune.","feed_headline":"RL agents find six game exploits on their own in NHL 26","feed_subtitle":"Reward masking forces agents to discover diverse scoring strategies without human intervention, matching what playtesters found manually.","key_machinery":"Reward masking: after each agent converges, the reward function is modified to give zero reward for goals scored from within 2m of a previously found average shot position when using the same shot type. This is a hard constraint (binary zero reward) rather than a soft penalty, and it is applied sequentially (one agent at a time) rather than in parallel, which the authors argue increases stability compared to parallel diversity methods that learn diversity representations alongside policies.","core_discovery":"Sequential reward masking is a sufficient mechanism to overcome RL's tendency to collapse to a single optimal solution. By zeroing out reward for previously found strategies (defined by shot type and position within a 2m radius), each new agent is forced to explore a different region of the strategy space. The method is simple enough for non-RL practitioners to configure and use, and it produces exploits comparable to those found by human testers. The paper also documents a key failure mode: agents can learn bi-modal behaviors where shots scatter around two positions, leaving roughly one-third of shots outside the exclusion radius, which means future iterations can still be rewarded for a策略s","pith_inferences":["The multimodality problem suggests that a clustering-based diversity criterion (e.g., k-means on shot positions, which the paper mentions as future work) would be a natural and likely effective fix, since it would allow the exclusion zone to track each mode of a multi-modal policy rather than just the global average.","The observation that RAID cannot verify whether a fix actually closed an exploit (because RL policies overfit to specific environment dynamics) implies that a complementary system—perhaps imitation learning or scripted bots that reproduce the found strategy—would be needed to close the testing loop.","The linear runtime and sequential design trade simplicity for speed; a hybrid approach that runs RAID sequentially but evaluates multiple candidate diversity criteria in parallel could reduce the number of wasted iterations where agents converge below the performance threshold."],"forward_implications":["Game studios could integrate RAID into their CI/CD pipeline to automatically surface goalie AI weaknesses after each behavior update, reducing the manual testing burden.","The reward masking approach is domain-agnostic in principle (Algorithm 1) and could be applied to other game testing scenarios beyond hockey scoring, such as fighting game combos or racing line exploits, as long as a domain-specific diversity criterion can be defined.","The sequential nature of RAID means runtime grows linearly with the number of strategies found; for domains with many possible exploits, this could become impractical without parallelization or early stopping heuristics.","The finding that RAID discovers strategies matching human playtester findings suggests the method could serve as a first-pass exploit discovery tool, with humans focusing on verification and classification rather than search."],"fun_headline_variants":["Reward masking forces RL agents to find diverse NHL 26 goalie exploits","Six NHL 26 scoring exploits found by RL agents via sequential reward masking","Simple reward masking stops RL collapse and finds six distinct NHL 26 exploits","Automated RL testing matches human playtesters on NHL 26 goalie AI exploits","Zeroing reward for found strategies pushes RL to discover new NHL 26 exploits"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The diversity criterion—defining a strategy solely by shot type and average shot position within a 2m radius—is sufficient to force meaningfully different behaviors. An agent can learn bi-modal behavior where shots scatter around two positions, leaving about a third of shots outside the exclusion radius, which means future iterations can still learn substantially the same strategy.","fun_headline_variants_meta":{"raw":{"variants":["Reward masking forces RL agents to find diverse NHL 26 goalie exploits","Six NHL 26 scoring exploits found by RL agents via sequential reward masking","Simple reward masking stops RL collapse and finds six distinct NHL 26 exploits","Automated RL testing matches human playtesters on NHL 26 goalie AI exploits","Zeroing reward for found strategies pushes RL to discover new NHL 26 exploits"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":627,"prompt_tokens":530,"completion_tokens":97,"prompt_tokens_details":null},"tokens_in":530,"tokens_out":97,"duration_ms":28551,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T08:54:00.139727+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Run RAID on a game where the set of possible exploits is known exhaustively. If the method fails to find a significant fraction of known exploits, or if it repeatedly rediscovers near-identical strategies that evade the diversity criterion (e.g., via multimodality), the core utility claim is weakened.","supporting_citations":[],"review_version":1}