{"id":"b5c6c558-fe36-4a2b-903c-ec9f850f4e31","arxiv_id":"2505.18520","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MAGE evolves assembly-level variants of the Intruder virus using novelty search over Jaccard similarity, producing diverse variants that evade over 98% of VirusTotal scanners.","lead":"This paper proposes MAGE, an evolutionary algorithm that mutates and recombines assembly code to generate many variants of a source malware, and reports that variants evade over 98% of VirusTotal antivirus scanners. The authors frame these variants as 'antigens' for training antivirus engines to detect future malware variants.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that variants retain maliciousness is unsupported: the paper verifies VirusTotal detection drops but never executes evolved variants to confirm infection behavior, so evasion may simply reflect broken executables.","rationale":"The reader's weakest_assumption correctly identifies the same load-bearing concern: code transformations are assumed to preserve both validity and maliciousness, but only VirusTotal evasion is measured. The paper does provide some independent support for the diversity claim: a Mann-Whitney U test on initial versus final populations for beta gives p = 6.467e-8, supporting the novelty-versus-similarity comparison, and the transformation constraints give a plausible, though not machine-checked, argument for assembly-level validity. However, the central claim of the paper explicitly includes maliciousness retention and the proposed use of generated variants as antigens for antivirus training. Since no dynamic execution or infection-behavior verification is reported, the evasion percentages in Section 3 are equally consistent with the generation of non-functional binaries. This gap does not necessarily invalidate the framework, but it makes the headline claim unverified as stated. The reader's CONDITIONAL verdict is therefore appropriate: the paper should be accepted only if the authors provide execution-based evidence that evolved variants retain Intruder's infection behavior, together with clarification of the fitness definition and release of artifacts. No change to the reader's verdict is needed.","tokens_in":13037,"tokens_out":1841,"duration_ms":18608,"concrete_test":"Sample 50 variants from the beta fitness final population (and another 50 across generations 50, 150, 250) plus the source Intruder virus. Assemble each with MASM and execute it in the isolated Windows 7 VM described in Section 3, seeding a directory tree with clean .EXE files. For each variant, record whether it appends its code to target .EXE files, propagates across subdirectories/drives, and triggers a detectable infection when the infected .EXE is run, comparing the infection rate with the source virus. If a variant that evades all VirusTotal scanners fails to infect any target, the maliciousness-retention claim is falsified for that variant; if infection rates are comparable to the source, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that MAGE generates diverse variants 'while retaining its maliciousness' and that these variants can serve as antigens for antivirus training. The load-bearing assumption is that the transformation operators T_FI, T_FJ, T_UB, T_CZJ, T_CNZJ, and T_CBI preserve both validity and malicious behavior of the evolved executables. The paper never verifies this assumption dynamically. Section 3 reports that variants 'evaded detection by over 98%' of VirusTotal scanners and that MAGE evolved 'valid virus executables for over 600 generations,' but 'valid' here means the transformed assembly code assembles under MASM into an executable, not that the executable runs and still infects .EXE files. The forced-JMP and untouchable-block transformations insert jumps and dead code, and the conditional-JMP transformations alter control flow based on the zero flag; without executing the variants, there is no evidence that these mutations do not skip essential infection logic, corrupt the stack, or otherwise break the virus. If the evolved variants are inert, the evasion result is a trivial consequence of producing non-functional binaries and the proposed 'antigen dataset' would be useless for improving malware detection. This is the single most load-bearing gap because the paper's contribution, as stated in the abstract and Section 4, depends on retaining maliciousness, not merely on producing undetected binaries.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MAGE, an evolutionary framework that starts from an input malware's assembly source and applies code transformation operators (fake instructions, forced jumps, untouchable blocks, conditional jumps, and code-block crossover) together with a novelty-inspired fitness function to evolve variants. The authors claim that the variants are diverse, valid, retain maliciousness, and evade over 98% of VirusTotal scanners, and they suggest that the resulting dataset can serve as \"antigens\" for training antivirus engines. The experiments compare two fitness functions, alpha (Jaccard similarity only) and beta (distance to the population-mean similarity vector), on a single virus, Intruder, using one run of 300 generations. The central claims are that beta produces more diverse variants than alpha and that the variants evade antivirus detection while remaining malicious.","tokens_in":13348,"tokens_out":10044,"duration_ms":81115,"significance":"If substantiated, the framework would be a useful modular testbed for proactive malware variant generation, and the antigen framing is a plausible application for augmenting antivirus training sets. The paper's strengths are its explicit formulation of assembly-level transformation operators with constraints, the modular separation of representation, variation, and fitness, and the inclusion of a baseline comparison and a statistical test. However, the current evidence does not yet support the central claims: maliciousness is never confirmed behaviorally, the diversity advantage of beta is largely encoded in its fitness function, the statistical test ignores within-run dependence, and the VirusTotal data are unreproducible. No code or data artifacts are provided, which further limits reproducibility. The idea is promising, but the manuscript needs substantial additional validation before the core claims can be accepted.","major_comments":[{"comment":"The paper's central claim that variants retain their maliciousness is not verified. The text equates \"valid virus executables\" with assembly code that assembles under MASM, but no experiment executes the evolved binaries to confirm that they still attach to and infect .EXE files. The operators T_FJ, T_UB, T_CZJ, and T_CNZJ insert jumps and dead code in the program body; without dynamic analysis, the observed VirusTotal evasion could simply be a consequence of non-functional executables. Please add behavioral verification, for example by executing each evolved variant in the isolated VM against clean .EXE files and checking infection behavior, and report the fraction of variants that remain infectious. Without this, the antigen-dataset claim in Section 4 is unsupported.","section":"This concerns Section 3 (Experiments and Discussion) and Section 4 (Conclusion)."},{"comment":"The claimed diversity advantage of fitness beta over alpha is partly a consequence of the fitness definition. Eq. (1) defines beta as the Euclidean distance of an individual's similarity vector from the population-mean similarity vector, so selection explicitly rewards deviation from the population mean; lower final similarity to the source under beta is therefore expected. The Mann-Whitney test in Table 2 largely confirms that the beta objective was optimized, rather than demonstrating that beta discovers a more diverse variant space in an independent sense. Please compare alpha and beta using a diversity metric not contained in the fitness, such as pairwise assembly edit distance, instruction n-gram overlap, or behavioral traces, and discuss the relationship between the objective and the measured outcome.","section":"This concerns Section 2.2, Eq. (1), and Section 3, Fig. 5 and Table 2."},{"comment":"The Mann-Whitney U test is applied to the 20 individuals of the initial population versus the 20 individuals of the final population of a single evolutionary run. These individuals are not independent samples: they are produced by one stochastic process, share ancestry through crossover and selection, and the final population is the result of selection on the fitness under test. The reported p-value (6.467e-8) therefore overstates the statistical significance. The hypothesis should be tested over multiple independent runs with different random seeds, using one summary statistic per run, or with an appropriate hierarchical model that accounts for within-run dependence.","section":"This concerns Section 3, Table 2."},{"comment":"The VirusTotal evaluation is not reproducible and the headline number is under-specified. The paper does not provide the SHA-256 hashes or VirusTotal report URLs for the source and evolved variants, the submission date, the exact scanner set, or the raw detection counts behind Figure 6. Since VirusTotal detections change over time and the paper itself notes that minor code modifications evade scanners, the claim that variants \"evaded over 98% of popular antivirus scanners\" cannot be verified or compared with future work. Please include the per-generation detection counts, the total scanner count, and a permanent artifact, such as hashes and reports or a public repository.","section":"This concerns Section 3, Figure 6 and the VirusTotal evaluation."},{"comment":"The fitness function is not defined unambiguously enough to reproduce. Eq. (1) is written as a sum over i=1..P of (\\bar{S}, \\vec{S}_{I_i})^2, which is not the Euclidean distance between an individual similarity vector and the mean vector described in the text; the roles of \\bar{S} and \\vec{S}_{I_i} are not clear, and Algorithm 2 line 4 abbreviates the quantity with an undefined \"J_i\". Please rewrite the per-individual fitness definition with explicit vector components and dimensions, and align Algorithm 2 with that definition.","section":"This concerns Section 2.2, Eq. (1), and Algorithm 2, line 4."}],"minor_comments":[{"comment":"The paper says the EA was run for 300 generations, but later states that MAGE evolved valid virus executables for \"over 600 generations\" and 12,000 variants; please reconcile this discrepancy and report the actual run length.","section":"This concerns Section 3, experimental setup."},{"comment":"The text claims the framework is generic and applicable to \"any malware\", but the experiments use a single virus (Intruder) and a single random seed; please temper the generalization claim or support it with additional malware families and seeds.","section":"This concerns Section 3, generality of the results."},{"comment":"Algorithm 2 mentions adding novel individuals to a \"Unique\" variant dataset, but no novelty threshold or dataset construction rule is specified; please clarify how this dataset is formed and how it relates to the final evaluation.","section":"This concerns Algorithm 2, line 10."},{"comment":"The paper does not report the mutation probabilities p_mi, the crossover probability, the tournament size, or the actual random seed value, even though the comparison is said to use the same seed; please list all hyperparameters.","section":"This concerns Section 3, hyperparameters."},{"comment":"The caption of Figure 5 and the columns of Table 2 do not state whether the reported similarity values are for the best individual per generation or for the whole population; please clarify the aggregation. In addition, Table 2 appears to have formatting errors with missing values in some rows.","section":"This concerns Figure 5 and Table 2."},{"comment":"No repository or artifact link is provided for the MAGE implementation or the evolved variants; please include one to support reproducibility and follow-up research.","section":"This concerns the overall reproducibility of the manuscript."}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be a conference-style paper with no code or data artifacts. The main safety-related issue is that the \"antigen\" variants are not shown to be actually infectious, which is essential if the dataset is to be used for antivirus training. I would ask the authors to add dynamic behavioral verification, independent diversity metrics, proper statistical treatment over multiple runs, and a reproducible VirusTotal artifact before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take. The MAGE framework is a genuine, modest step: it pairs novelty search with assembly-level transformation operators—fake instructions, forced jumps, untouchable blocks, conditional jumps, and block interchange—and shows, with a clean Mann-Whitney U test, that novelty-based fitness (beta) produces variants with lower Jaccard similarity to the source than similarity-only fitness (alpha) over 300 generations. The modular framing is useful; the transformation operators are sensible and the paper positions them clearly within the obfuscation literature. The related work is adequate.\n\nThe soft spot is the one the stress test flags, and I think it lands. The paper claims the variants retain maliciousness, but 'valid' never means 'runs and still infects.' It means the assembly assembles under MASM. So the VirusTotal evasion result—98% of scanners evaded by generation 250—may simply reflect broken executables. That matters because the whole antigen-for-training story depends on the variants being live malware. This is the load-bearing gap, and it is fixable: execute the evolved binaries in the isolated VM and check whether they still infect .EXE files. Without that, the central claim is unsupported.\n\nTwo smaller issues. The fitness function in equation (1) is ambiguous as written; it looks like a population aggregate but is described per-individual. And the diversity result is partly a product of the beta fitness definition, which explicitly rewards distance from the population mean. Not circular, but less surprising than the paper implies. Finally, one malware, one seed, no released artifacts, so the VirusTotal numbers are unreproducible.\n\nAll of this is proportionate. The paper is worth engaging: it is a legitimate EA-malware contribution, the statistical comparison is honest, and the missing verification is an experiment rather than a conceptual flaw. It deserves peer review, and a sensible referee would ask for dynamic behavioral verification, a cleaner fitness definition, and at least a second malware. I'd bring it to a reading group only if the topic is on the table; otherwise it's a solid 'maybe.'","headline":"A modest but real EA-malware contribution undermined by a load-bearing gap: the variants' maliciousness is never dynamically verified, so the 98% evasion rate may just be broken binaries.","tokens_in":13787,"tokens_out":3027,"would_cite":true,"duration_ms":27029,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An evolutionary algorithm guided by novelty search can generate diverse, scanner-evading variants of a malware's assembly code, serving as antigens for improving antivirus detection.","keywords":["Evolutionary Algorithm","Malware Generation","Novelty Search","Antivirus Evasion","Assembly Code Transformation","Proactive Defence","Malware Variants","Antigens"],"falsifier":"Place each evolved variant in an isolated virtual machine and check whether it still exhibits Intruder's infection behaviour, namely attaching itself to .EXE files and spreading to directories and drives. If scanner-evading variants fail to infect any target file, the retained-maliciousness claim collapses; a complementary control would apply the same transformation operators to benign assembly programs and verify that the constraints really produce valid, correctly running executables.","tokens_in":12867,"feed_emoji":"🦠","tokens_out":8547,"duration_ms":65689,"temperature":0.7,"pith_summary":"The paper claims that an evolutionary algorithm that searches for novelty rather than a fixed fitness optimum can generate large numbers of diverse variants of a malware program by mutating its assembly source code, and that these variants still behave maliciously while escaping detection by antivirus scanners. Such variants are proposed as 'antigens': a proactive dataset a detection engine can be trained on before real-world versions of the malware appear. The authors implement this as MAGE, seed it with the Intruder virus, and report that variants evolved with a novelty-supported fitness function are significantly more diverse than those from a similarity-only fitness function, and that over 98% of the scanners on an online multi-engine service fail to detect them by generation 250. If correct, the contribution is a generic, modular framework for producing malware variant datasets for proactive defence, not a single bespoke generator.","feed_headline":"Novelty-guided evolution makes malware variants evade 98% of scanners","feed_subtitle":"Assembly-code evolution with novelty search beats similarity fitness for diverse scanner-evading variants.","key_machinery":"The central object is the novelty-search fitness function acting on a linear representation of the assembly code. Each individual is represented as a vector of assembly statements; the fitness is the Euclidean distance between the individual's Jaccard similarity vector and the mean similarity vector of the population. Jaccard similarity here is the size of the intersection of two code-statement sets divided by the size of their union, so the fitness rewards individuals that are structurally unusual relative to both their neighbours and the source malware. The variation machinery consists of five constrained mutation operators (fake instruction, forced JMP, untouchable block, conditional-zero JMP, conditional-nonzero JMP) and a code-block-interchange crossover operator with a pivot point; the constraints are what keep the transformed code a valid, assemblable program. Together the operators and the fitness function push the population toward divergent code structures instead of convergence to a single optimum.","core_discovery":"The paper's central claim is that MAGE, an evolutionary algorithm operating on the assembly source code of a seed malware, can evolve a large population of diverse variants that remain valid executables, retain the malicious behaviour of the source, and evade the signature-based detection of commercial antivirus scanners. The authors build the algorithm from generic transformation operators—fake NOP instructions, forced jumps, untouchable dead-code blocks, conditional zero and non-zero jumps, and pivot-constrained code block interchange—and combine them with a novelty-search fitness function that rewards assembly programs whose Jaccard similarity profiles differ from their neighbours and from the source. Using the Intruder virus as the seed, 20 of the scanners on the online service detected the original, while after 250 generations of evolution the best variants evaded over 98% of those scanners. A Mann-Whitney U test on the similarity of initial and final populations supports the paper's claim that novelty-based fitness induces significantly greater structural diversity than similarity-only fitness.","pith_inferences":["The paper's 'retained maliciousness' claim is supported only by the fact that the transformations never touch the virus's infection logic, not by observation of infection; the most direct test would be executing the variants, and the evasion numbers would be equally explained by variants that are broken or inert.","The same novelty-search objective could be transferred to adjacent security generation tasks, such as producing diverse fuzzing inputs or adversarial examples, wherever population diversity is the goal rather than a single optimum.","A practical deployment would require showing that the antigen dataset improves detector accuracy on held-out variants, not merely that the variants evade the scanners they were tested against.","Because the transformations are assembly-level, they could in principle be applied at build time to benign programs as a stress test for false positives, giving detection engines a calibration signal."],"forward_implications":["If MAGE works as claimed, a single seed malware can yield a large dataset of structurally distinct variants, with the paper reporting on the order of 12,000 valid executables over 600 generations, that can be fed to antivirus and malware analysis engines for retraining.","The novelty-supported fitness function outperforming similarity-only fitness implies that future malware-generation evolutionary algorithms should adopt divergence-promoting objectives rather than pure similarity to the source.","Since the 98% evasion emerges from constrained assembly-level changes, the result underscores how brittle signature-based detection is; detection engines would need to incorporate the same transformation space to keep up.","The modular framework allows representation, transformation operators, quality indicator, and the underlying evolutionary algorithm to be swapped, so the same design can be applied to other malware families or code representations without redesigning the generator."],"supporting_citations":[{"why":"Supplies the novelty-search algorithm that rewards novel behaviours, the conceptual basis of MAGE's fitness function.","marker":"[17]"},{"why":"Provides novelty search with self-adaptation and archive management, informing how MAGE selects novel individuals.","marker":"[18]"},{"why":"Gives the theoretical perspective and archive strategies for novelty search used in the framework's design.","marker":"[10]"},{"why":"The source of the Intruder virus assembly code used as the seed for all evolution experiments.","marker":"[20]"},{"why":"Catalogues the assembly-level code diversification and obfuscation constructs on which the transformation operators are modelled.","marker":"[13]"},{"why":"Establishes that minor code modifications change malware signatures, motivating the entire evasion problem the paper addresses.","marker":"[22]"},{"why":"Demonstrates an earlier evolutionary malware-generation approach at the application-feature level that MAGE extends to assembly code.","marker":"[31]"},{"why":"Describes the online multi-engine scanning service used to measure detection of the original and evolved variants.","marker":"[39]"}],"fun_headline_variants":["Malware evolution with novelty search evades 98% of antivirus scanners","Novelty-guided assembly evolution beats signature-based antivirus scans","Evolutionary malware generator: 98% scanner evasion via novelty fitness","MAGE: Novelty search evolves diverse malware variants that evade detection","Turning malware into antigens: novelty search evades 98% of scanners"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the transformation operators preserve both the validity and the maliciousness of the evolved executables; the paper never executes the variants to confirm they still infect files, so if the mutations break the virus, the evasion result may just mean the variants are inert.","fun_headline_variants_meta":{"raw":{"variants":["Malware evolution with novelty search evades 98% of antivirus scanners","Novelty-guided assembly evolution beats signature-based antivirus scans","Evolutionary malware generator: 98% scanner evasion via novelty fitness","MAGE: Novelty search evolves diverse malware variants that evade detection","Turning malware into antigens: novelty search evades 98% of scanners"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1346,"prompt_tokens":912,"completion_tokens":434,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":341}},"tokens_in":528,"tokens_out":434,"duration_ms":3591,"temperature":1.0,"reasoning_tokens":341,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:29:22.120252+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Place each evolved variant in an isolated virtual machine and check whether it still exhibits Intruder's infection behaviour, namely attaching itself to .EXE files and spreading to directories and drives. If scanner-evading variants fail to infect any target file, the retained-maliciousness claim collapses; a complementary control would apply the same transformation operators to benign assembly programs and verify that the constraints really produce valid, correctly running executables.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides novelty search with self-adaptation and archive management, informing how MAGE selects novel individuals."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The source of the Intruder virus assembly code used as the seed for all evolution experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Catalogues the assembly-level code diversification and obfuscation constructs on which the transformation operators are modelled."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that minor code modifications change malware signatures, motivating the entire evasion problem the paper addresses."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates an earlier evolutionary malware-generation approach at the application-feature level that MAGE extends to assembly code."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes the online multi-engine scanning service used to measure detection of the original and evolved variants."}],"review_version":1}