{"id":"f520dc7b-44cf-4c3c-868d-efc0985e32b6","arxiv_id":"2506.15506","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic review of 53 papers on adversarial attacks for tabular machine learning finds the field fragmented, with efficacy over-emphasized and transferability, plausibility, and semantic preservation under-addressed.","lead":"This paper reviews 53 published studies that attack machine learning models on tabular data, classifies 61 attack methods, and maps them against eight real-world practicality criteria. It is useful because tabular ML is used in finance, healthcare, and security, but adversarial research in this area has been fragmented.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core quantitative synthesis is internally inconsistent: tables sum to 60 attacks, text and figures disagree on gradient-free (23 vs 21) and learning-based (14 vs 15) counts, and the code-release statistic is reversed in the discussion.","rationale":"This is a structured PRISMA-based review with a transparent flow diagram, detailed extraction tables, and a reasonable eight-dimension coding scheme; those are real strengths. The most load-bearing weakness is not the search string's exact-phrase requirement for \"tabular data\", because snowballing and Google Scholar supplement the database search and mitigate recall risk. The more decisive problem is that the review's own tables contradict its text and figures on basic counts. The total of 60 attack rows in Tables 3–6 does not match the claimed 61 attacks; the gradient-free count is 21 in the figure but 23 in the text; the learning-based count is 14 in Table 5 but 15 in the text and figure; and the code-release statistic is reversed between Sections 5.2 and 6. These are observable internal inconsistencies, not hypothetical threats. Since the review's contribution is precisely the numerical synthesis of 53 studies and 61 attacks, the reported distributions and gap analysis rest on unreliable arithmetic. The reader's weakest assumption identified search and coding reliability; the count inconsistencies are a more specific, directly checkable instance of the same underlying concern, so I mark partial agreement. The appropriate verdict remains conditional: the paper is potentially valuable and the flaws are fixable by re-extraction and correction, but the central quantitative claims cannot be accepted as they stand.","tokens_in":33669,"tokens_out":8328,"duration_ms":72474,"concrete_test":"Independently re-extract every attack from the 53 included papers into a single table mapping each attack to a unique paper ID, and recompute: (a) total unique attacks, (b) counts per optimization family, (c) code-availability count, and (d) the 53-paper ID list. If the corrected totals differ from the 61, 19, 23/21, 15/14, 6, and 21/61 figure combinations reported in the text and figures, all derived percentages, Figure 7, and the open-source claim must be revised before the review can be considered accurate.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is a systematic consolidation of \"53 studies, 61 attacks, and eight practical considerations.\" The review's quantitative findings (category shares, Figure 7, gap analysis) all derive from the extracted counts, but those counts are not internally consistent. Section 5.2 states 61 unique attacks and then reports gradient-free 23, gradient-based 19, learning-based 15, hybrid 6 (sum 63). Figure 6 reports gradient-free 21, learning-based 15 (sum 61). Tables 3–6 contain 19 gradient-based, 21 gradient-free, 14 learning-based, and 6 hybrid rows (sum 60). Table 5 lists 14 learning-based attacks and its own text says \"GAN-based methods (12 out of 14)\", contradicting the 15 in the same paragraph and in Figure 6. The code-release statistic is also inverted: Section 5.2 says \"only 21 out of 61 attacks having public code\", while Section 6 says \"21 out of 61 attacks do not release their code.\" Finally, Table 9 includes a row for reference [98] (Papernot et al., a background citation, not one of the 53 included studies), and Table 7 lists [114] twice with different venues and years, so the 53-study set itself contains anomalies. Because these numbers are the primary evidence for the review's conclusions, a systematic review whose headline counts cannot be reproduced from its own tables does not yet support the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a systematic literature review of adversarial attacks on tabular machine learning, following the PRISMA 2020 workflow. The authors report identifying 53 studies and 61 distinct attacks, organize the attacks into gradient-based, gradient-free, learning-based, and hybrid categories, and code each study on eight practical-consideration dimensions (efficacy, efficiency, transferability, feasibility, semantic preservation, plausibility, defense awareness, dataset suitability). They analyze publication trends, attack methodologies, and datasets, and derive a set of research gaps and future directions for adversarial robustness in tabular ML.","tokens_in":33955,"tokens_out":6222,"duration_ms":55479,"significance":"If its quantitative synthesis were internally consistent, this would be a useful contribution: it is the first dedicated systematic review of adversarial attacks on tabular data, provides a clear taxonomy of attack strategies, documents a transparent PRISMA-style selection process, and compiles a detailed dataset inventory. The qualitative observations about fragmentation, lack of benchmarks, and uneven attention to practical considerations are plausible and valuable for orienting future work. However, the headline numbers that the review's conclusions rest on are not reproducible from the paper's own tables and figures, and the coding procedure for the eight practical considerations lacks evidence of reliability. The central claims are therefore defensible in direction but not yet fully supported in their quantitative form.","major_comments":[{"comment":"The attack counts are internally inconsistent and cannot be reconciled. The text in Section 5.2 reports 61 unique attacks and then states gradient-free 23, gradient-based 19, learning-based 15, hybrid 6, which sum to 63. Figure 6 reports gradient-free 21 and learning-based 15. Tables 3–6 contain 19 gradient-based, 21 gradient-free, 14 learning-based, and 6 hybrid rows, summing to 60. Because the abstract's '61 attacks' claim and the category comparisons in RQ2 derive from these counts, the text, figure, and tables must all be reconciled to a single verified count before the review can support its headline statistics.","section":"Section 5.2, Figure 6, Tables 3–6"},{"comment":"The learning-based category count is self-contradictory. Section 5.2 states that learning-based attacks number 15, and Figure 6 shows 15, while Table 5 lists 14 rows. The text then says 'GAN-based methods (12 out of 14)', which conflicts with the 14 rows in Table 5 and with the stated total of 15. The 'GAN-dominated' narrative and the comparative claims about learning-based attacks depend on this ratio, so the total count and the GAN/non-GAN split must be corrected and made consistent across text, table, and figure.","section":"Section 5.2.3, Table 5, Figure 6"},{"comment":"The code-release statistic is inverted between two parts of the paper. The Section 5.2 summary states that 'only 21 out of 61 attacks having public code', whereas Section 6 states that '21 out of 61 attacks do not release their code'. These statements have opposite meanings for the open-source-availability gap. The 'Code' column in the attack tables should be re-audited and the summary and discussion rewritten to state one consistent, verified figure.","section":"Section 5.2 summary box and Section 6"},{"comment":"The coding of the eight practical considerations, which underpins RQ3 and the gap analysis throughout the paper, is reported without a complete coding protocol or reliability evidence. Section 5.3 defines three labels (Considered, Acknowledged, Not considered) and Appendix B gives a mapping in Table 8, but there is no dual coding, no inter-rater reliability statistic, and no description of how disagreements were resolved. In addition, Table 9 contains a row for reference [98], which is a background citation and not one of the 53 included studies. Because the aggregate numbers in Figure 7 and the 'fewer than 19 studies' conclusions are built on this coding, the authors should provide the full coding sheet, remove non-included references, and report reliability or at least justify the coding's consistency.","section":"Section 5.3, Tables 8 and 9"},{"comment":"Table 7 lists reference [114] twice, with different years and venues (2022/2023 AAAI and 2024/2024 NextGenAISafety workshop). Since Table 7 is the enumeration of the 53 included studies, a duplicate row means the population count is not uniquely identifiable. This must be corrected to ensure that the '53 studies' figure is a count of distinct studies.","section":"Appendix A, Table 7"}],"minor_comments":[{"comment":"The opening sentence is contradictory: '19 of the reviewed studies do not evaluate their attacks against any defense mechanism. Of the 19 that do ...' The first clause should presumably read '19 ... do evaluate their attacks against defense mechanisms', matching the subsequent 'Of the 19 that do'.","section":"Section 5.3.7"},{"comment":"The number of datasets used by only one study is given as 32 in Section 5.3.8 but as 34 in the Section 5.3 summary box. This should be checked and aligned.","section":"Section 5.3.8 and Section 5.3 summary box"},{"comment":"There are several typographical errors: 'is is' in Section 6, 'lipshitz' should be 'Lipschitz' in Section 6, and 'hypothetize' should be 'hypothesize' in Section 5.3.8.","section":"Section 6 and Section 5.3.8"},{"comment":"The search string requires the exact phrase 'tabular data'. Since the review claims to be a systematic consolidation of the field, the authors should either justify this restrictive term or report a sensitivity check with synonyms such as 'structured data' and 'table data' to demonstrate that relevant work was not missed.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"Several of the reviewed papers are authored by members of the review team. This is not by itself a reason for rejection, but given that the coding of practical considerations involves judgment, the editors may wish to ask the authors for an explicit statement of overlapping authorship and for independent validation of the coding. The quantitative inconsistencies are the main obstacle: they affect the paper's headline claims and should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this as the first real attempt to systematically consolidate the tabular adversarial attack literature, and the organization is genuinely helpful. The PRISMA flow is documented, the inclusion criteria are explicit, and the taxonomy of 61 attacks across 53 studies gives the field a common vocabulary. The eight practical considerations (efficacy, efficiency, transferability, feasibility, semantics, plausibility, defense awareness, dataset suitability) are a sensible lens, and the gap analysis lands on real problems: GAN-based learning attacks rarely compare against each other, there are no shared benchmarks, and concepts like feasibility and plausibility are conflated. For someone entering the area, this is a useful map.\n\nBut the soft spots are real and they hit the central quantitative claims. The counts do not reconcile. The text says 23 gradient-free and 15 learning-based; Figure 6 says 21 and 15; Table 5 lists 14 learning-based and then says \"12 out of 14\"; the tables sum to 60 attacks, not 61. The code-release statistic is reversed between Section 5.2 (21 have public code) and Section 6 (21 do not). Table 9 includes reference [98], a background citation that is not one of the 53 included studies, and Table 7 lists [114] twice with different venues and years. These are not cosmetic typos: the review's conclusions about category shares, gaps, and open-source support rest on those numbers. If a reader cannot reproduce the headline count from the paper's own tables, the synthesis is not yet reliable.\n\nThe coding of eight dimensions also rests on an unreported three-label protocol with no inter-rater reliability check. That is a lesser concern because the qualitative patterns are robust to a couple of mislabels, but the authors should at least disclose the protocol and ideally release the coded dataset. A related point: several reviewed papers are authored by the same group, and the coding definitions overlap with their prior work; that is not inherently a flaw, but it should be disclosed.\n\nOverall, the qualitative takeaways (fragmented venues, feasibility prioritized over semantics, GAN dominance in learning-based attacks) likely hold even with the count errors. The paper is not a takedown target; it is a solid first pass that needs a careful revision. I would send it to peer review, but with the requirement that the authors correct the arithmetic, reconcile text with figures and tables, and add transparency on the coding process. With those fixes, this becomes a reference I would point people to.\n\nVerdict: worth a serious referee, conditional on the revision.","headline":"A genuinely useful first systematic map of tabular adversarial attacks, but the headline counts don't add up and need a revision pass before the synthesis can be trusted.","tokens_in":34467,"tokens_out":2270,"would_cite":false,"duration_ms":24339,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims to be the first systematic literature review of adversarial attacks on tabular machine learning, synthesizing 53 studies and 61 attacks into a taxonomy and an eight-dimensional practical-consideration checklist.","keywords":["adversarial attacks","tabular data","systematic literature review","machine learning robustness","evasion attacks","practical considerations","feasibility constraints"],"falsifier":"Run the same review with a broader search (for example, replacing the exact phrase 'tabular data' with 'structured data', 'feature table', or domain terms like 'intrusion detection' or 'credit scoring') and have two independent coders apply the eight-dimension scheme; if substantially more attacks surface or inter-rater agreement is low, the reported 53-study corpus and the relative emphasis on efficacy versus feasibility would shift.","tokens_in":33488,"feed_emoji":"🛡️","tokens_out":5498,"duration_ms":51458,"temperature":0.7,"pith_summary":"This paper tries to establish that adversarial attacks on tabular machine learning models are a distinct, still-fragmented research area and that the field's first systematic map can be drawn from 53 studies and 61 attacks. It claims to be the first systematic literature review focused specifically on tabular-data attacks, as opposed to the mature surveys in computer vision and natural language processing. The review's value would be to give researchers a shared inventory of attack strategies and a checklist of eight practical considerations that determine whether an attack matters outside the lab. Its core finding is that efficacy is essentially universal while robustness-related qualities like semantic preservation, plausibility, transferability, and defense awareness are rarely evaluated.","feed_headline":"53 studies, 61 attacks: first systematic map of tabular ML threats","feed_subtitle":"The review finds efficacy universally measured while feasibility, semantics, and plausibility lag behind.","key_machinery":"The machinery is a three-level coding scheme, Considered/Acknowledged/Not considered, applied to eight practical dimensions, paired with a taxonomy of attacks by optimization strategy. The eight dimensions (efficacy, efficiency, transferability, feasibility, semantic preservation, plausibility, defense awareness, dataset suitability) function as a checklist that lets the review measure, study by study, how much of the real-world attack problem each paper addresses. The feasibility dimension is further decomposed into mutability constraints, structural constraints, and inter-feature relationships, because that is where tabular data differs most from images. The taxonomy of gradient-based, gradient-free, learning-based, and hybrid methods is what turns 53 heterogeneous papers into comparable counts.","core_discovery":"Working from a PRISMA-guided search of four databases plus citation-chaining and supplementary searches, the paper identifies 53 eligible studies and 61 attacks, and claims this is the first domain-agnostic synthesis of adversarial attacks on tabular machine learning. It organizes the attacks by optimization strategy (gradient-based, gradient-free, learning-based, hybrid) and labels every study on eight practical dimensions: efficacy, efficiency, transferability, feasibility, semantic preservation, plausibility, defense awareness, and dataset suitability. The headline results are that all 53 studies measure efficacy; feasibility receives attention in 44 studies, subdivided into mutability, structural, and inter-feature constraints; and the harder-to-define criteria (semantics, plausibility) are addressed by only 6 and 11 studies respectively. The paper also reports fragmented venues, uneven dataset use, and limited shared benchmarking.","pith_inferences":["A broader search that did not require the literal phrase 'tabular data' would likely recover additional domain-specific attacks, especially in cybersecurity and finance, where authors may describe their inputs as network flows or transactions instead.","The finding that all 53 studies measure efficacy but only a handful check plausibility or semantics suggests that publication pressure itself selects for attack success rate as the de facto bar, a dynamic that standardized benchmarks could counteract.","If the eight-dimension coding were applied to computer-vision attack papers, the plausibility and transferability columns would probably look very different; the contrast could give the tabular community a concrete target for maturity.","The review's venue fragmentation (46 venues for 53 papers) implies that progress may depend less on new attack algorithms than on shared infrastructure: a common benchmark, a common threat model, and release of attack code."],"forward_implications":["Future tabular adversarial research will need standardized benchmarks with shared datasets and metrics, because the review finds 34 of 61 datasets used in only one study and no consensus evaluation criteria.","Attack evaluation should move beyond accuracy or attack success rate to report query counts or runtime, transferability across models, and behavior under defenses; currently only 18 studies address efficiency and 19 test defenses.","Feasibility is the best-developed practical constraint, so new attacks should be expected to enforce at least structural constraints and ideally mutability and inter-feature dependencies.","The concepts of plausibility and semantic preservation need formal definitions for tabular data before they can be optimized; the paper finds them addressed in only 11 and 6 studies respectively.","New learning paradigms (pretraining, self-supervised and multitask learning, retrieval-augmented inference) and tasks like question answering over tables are open ground for adversarial robustness research, since current work covers almost exclusively supervised classification."],"supporting_citations":[{"why":"Supplies the systematic literature review procedure the paper follows.","marker":"[68]"},{"why":"Supplies the PRISMA 2020 reporting and screening-flow guidelines used for study selection.","marker":"[96]"},{"why":"The earliest domain-agnostic tabular adversarial attack; anchors the claim that no earlier systematic review exists and supplies the plausibility discussion.","marker":"[13]"},{"why":"The earliest identified attack in the corpus and the most cited; anchors the publication-timeline analysis.","marker":"[78]"},{"why":"Benchmark that filters infeasible adversarial examples; motivates the feasibility dimension.","marker":"[113]"},{"why":"Defines PGD and adversarial training, the most common attack and defense family the review categorizes.","marker":"[81]"},{"why":"Unified constrained attack and defense framework that contributes CAPGD, MOEVA, and constraint augmentation, central to the hybrid category.","marker":"[111]"}],"fun_headline_variants":["Tabular ML attacks: 53 studies, 61 attacks, little semantic testing","First systematic review: tabular ML attacks lack plausibility checks","61 attacks on tabular ML, but only 6 assess semantics","Tabular adversarial attacks: efficacy measured everywhere, feasibility weak","53 studies on tabular attacks: semantics and plausibility ignored"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The review's counts and gap analysis rest on the assumption that its search string, which requires the phrase 'tabular data', and its single-coder labeling of eight dimensions correctly capture all relevant attacks and each study's treatment of them.","fun_headline_variants_meta":{"raw":{"variants":["Tabular ML attacks: 53 studies, 61 attacks, little semantic testing","First systematic review: tabular ML attacks lack plausibility checks","61 attacks on tabular ML, but only 6 assess semantics","Tabular adversarial attacks: efficacy measured everywhere, feasibility weak","53 studies on tabular attacks: semantics and plausibility ignored"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000187,"raw_usage":{"total_tokens":1254,"prompt_tokens":795,"completion_tokens":459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":411,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":411,"tokens_out":459,"duration_ms":4779,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:33:10.067915+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same review with a broader search (for example, replacing the exact phrase 'tabular data' with 'structured data', 'feature table', or domain terms like 'intrusion detection' or 'credit scoring') and have two independent coders apply the eight-dimension scheme; if substantially more attacks surface or inter-rater agreement is low, the reported 53-study corpus and the relative emphasis on efficacy versus feasibility would shift.","supporting_citations":[],"review_version":2}