{"id":"769c5328-7d2a-4b60-8a00-1a75bc41be68","arxiv_id":"2501.00138","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"NiaAutoARM uses a population-based optimizer to automatically construct association rule mining pipelines by selecting the inner algorithm, hyperparameters, preprocessing, and evaluation metrics.","lead":"The authors built a system called NiaAutoARM that automatically chooses the best way to mine association rules from a dataset: it picks a mining algorithm, its settings, data preprocessing steps, and the metrics used to judge the rules. The system searches over these choices using evolutionary or swarm algorithms, and the paper tests it on ten public datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Preprocessing pipeline semantics are under-specified for P>1: Eq. (1) encodes preprocessors as independent binary switches with no execution order, so a genotype can decode to multiple or no coherent pipelines.","rationale":"The reader's weakest assumption points to the same gap I would flag: the paper's central mechanism is the genotype-to-pipeline mapping, and that mapping is incomplete for P>1. This is load-bearing because the claimed contribution is automated construction of ARM pipelines; a pipeline is by definition an ordered sequence of operations, and none of the order information is in Eq. (1) or Eq. (2). The P>1 experiments are the only place the paper exercises this part of the representation, so their results inherit the ambiguity. I considered the indirect VARDE comparison as an alternative concern: Table 9 appears to use previously reported VARDE results rather than a matched re-run, so the 'outperforms' claim is not established at equal budget. That is a real empirical weakness, but it does not threaten the conceptual validity of the method as much as an undefined phenotype space does; it could be fixed by rerunning VARDE under the same evaluation protocol. I also note Eq. (3) never reports alpha and beta, and no code is released, so reproducibility is weaker; these are secondary. The missing order can be repaired by defining a fixed default order or by encoding order in the genotype, so the appropriate verdict remains CONDITIONAL rather than REJECT. My read therefore leaves the reader's CONDITIONAL verdict unchanged.","tokens_in":16914,"tokens_out":5184,"duration_ms":50647,"concrete_test":"Obtain the implementation (or reconstruct it from the NiaARM/NiaPy repositories) and, on one dataset such as Abalone, construct a fixed genotype that selects both min-max normalization and k-means discretization. Execute the decoded pipeline in every possible order of those two methods and compare the fitness value returned by Eq. (3); repeat for a pair such as z-score and min-max. If the fitness changes with order, the missing order specification is consequential and the P>1 results in Tables 7-8 are ambiguous. If the implementation silently applies a fixed order, document that order, verify it matches the genotype semantics, and rerun the Section 4.1.3 experiments with that order made explicit to see whether Tables 7-8 remain reproducible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The genotype in Eq. (1) includes a PREPROCESSING block p_i,1..p_i,P and activates each method independently whenever p_i,j > 0.5; Eq. (2) passes Gamma(Prep,p) to the inner ARM call as if it were an unordered set. The paper never defines the order in which selected preprocessing methods are executed. This matters because the offered pool (MM, ZS, DS, RHC, DK) is order-sensitive: applying min-max normalization before k-means discretization changes the discretization intervals relative to the reverse order, and applying z-score after min-max is not the same as applying it before. Section 4.1.3 explicitly allows combinations such as {MM,RHC,ZS} and reports results in Tables 7-8 and Figures 5-6. Without a fixed order, a single genotype maps to multiple possible executable pipelines with potentially different outputs, so the P>1 fitness values are not well-defined and the claimed automated search over 'pipelines' is incomplete. If the implementation happens to impose a hidden fixed order, that order is part of the pipeline definition and must be stated; otherwise the genotype space cannot represent the class of pipelines the paper claims to construct.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NiaAutoARM, an AutoML approach that models the construction of numerical association rule mining (ARM) pipelines as a continuous optimization problem. An outer population-based metaheuristic (PSO or DE) evolves real-valued vectors that encode the inner ARM algorithm, its hyperparameters (NP and MAXFES), a set of preprocessing methods, a set of evaluation metrics, and metric weights. The inner ARM algorithm is then run on the preprocessed data, and the outer fitness is defined by Eq. (3) as a linear combination of support and confidence. The method is evaluated on ten UCI datasets in four experimental settings: a baseline with one preprocessing method and fixed metric weights, metric-weight adaptation, multiple preprocessing methods, and an indirect comparison against the VARDE algorithm. The paper claims to be the first AutoML solution for ARM pipeline construction, and reports that the generated pipelines outperform VARDE in some instances.","tokens_in":17172,"tokens_out":4122,"duration_ms":46180,"significance":"If the full pipeline semantics were precisely specified, the paper would make a useful contribution to automated numerical ARM: it formulates an interesting search space, provides Algorithm 1 as a concrete optimization loop, evaluates on ten public datasets, and attempts a comparison with a state-of-the-art ARM method. The paper also has the merit of treating metric weights as optimizable parameters and of recognizing preprocessing as a pipeline component. However, the significance of the empirical claims is currently limited by under-specified execution semantics for multiple preprocessing methods and by an incompletely defined outer fitness aggregation over the mined rule set. These gaps affect the reproducibility and interpretation of most reported results.","major_comments":[{"comment":"The preprocessing block in Eq. (1) is an unordered set of binary switches (each method active when p_i,j > 0.5), and Eq. (2) passes this set to the inner ARM call. For P > 1, the selected methods (MM, ZS, DS, RHC, DK) are order-dependent: applying min-max normalization before k-means discretization is not equivalent to applying it after, and applying z-score after min-max is not the same as the reverse order. Section 4.1.3 reports results for combinations such as {MM, RHC, ZS}, but the paper never specifies the execution order. A single genotype therefore maps to multiple different executable pipelines, so the P > 1 fitness values are not well-defined. The authors should either define a deterministic fixed order for applying all selected preprocessing methods, or extend the representation to encode the order explicitly, and then re-run or re-describe the P > 1 experiments under that unambiguous semantics.","section":"Section 3, Eq. (1)-(2); Section 4.1.3, Tables 7-8"},{"comment":"The outer fitness function in Eq. (3) uses support and confidence of a single association rule, but the inner ARM algorithm generally returns a set of rules. The paper does not state how supp and conf are obtained from that rule set: best rule, average over rules, weighted aggregate, or some other summary. This omission makes the objective function of the outer optimization incomplete and prevents the reader from reproducing the fitness values reported in Section 4. The authors should specify the aggregation rule, or alternatively define Eq. (3) directly over the mined rule set.","section":"Section 3, Eq. (3)"},{"comment":"The comparison with VARDE is under-specified. The text says that for each dataset the authors observed which pipeline components provided the best results and then performed 30 independent runs with those settings, but it does not state how those components were selected, whether the selection used information from the same runs that are later compared, what inner hyperparameter budget was used, or how the Wilcoxon signed-rank test was paired across the 30 runs. Table 9 also reports only p-values with no effect sizes or confidence intervals. Consequently, the conclusion that NiaAutoARM is 'distinguished as an effective framework' or achieves 'superior performance' is not sufficiently supported by the presented evidence.","section":"Section 4.1.4, Table 9"},{"comment":"The claims that ARM metric-weight adaptation provides 'slightly higher fitness values' and 'overall better ARM pipelines for the majority of datasets' are not backed by a proper statistical comparison. The only statistical statement is a Wilcoxon test with p-value=0.41 reported in the text, but it is not clear which two configurations are being compared. The same issue applies to Section 4.1.3, where the effect of P > 1 versus P = 1 is described qualitatively. The authors should report pairwise tests for the configurations they compare, or explicitly state that the observed differences are descriptive only.","section":"Section 4.1.2, Tables 5-6 vs Tables 3-4"}],"minor_comments":[{"comment":"In Tables 7 and 8, the preprocessing columns are all reported as '-', so the tables do not contain the preprocessing-selection frequencies that the experiment is meant to analyze. The actual frequencies appear only in the heatmaps of Figures 5 and 6; the tables should either include these data or be merged with the figures consistently.","section":"Section 4.1.3, Tables 7-8"},{"comment":"The parameters α and β in Eq. (3) are never given values or a calibration procedure. The text says they designate the impact of each metric, but the reader cannot tell whether they are fixed constants, user inputs, or optimized values. Please define them.","section":"Section 3, Eq. (3)"},{"comment":"The text refers to a 'Wilcox test'; this should be spelled as the 'Wilcoxon signed-rank test', and the compared configurations should be identified explicitly.","section":"Section 4.1.2"},{"comment":"The column headers 'VARDE pos 15 2000 [18]' and 'VARDE neg 15 2000 [18]' are unclear. The meaning of 'pos' and 'neg', and whether '15 2000' are hyperparameters of VARDE, should be explained in the caption or the text.","section":"Section 4.1.4, Table 9"},{"comment":"The paper lists 'To implement a new method called NiaAutoARM as a Python package' as a contribution, but no repository or package URL is provided in the manuscript. Adding a link would improve reproducibility and allow reviewers and readers to verify the implementation details.","section":"Contributions, Section 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is straightforward: wrap the inner NiaARM optimizer in an outer metaheuristic that also selects preprocessing methods, evaluation metrics, metric weights, and hyperparameters. That is a sensible extension of the authors' earlier NiaAML work, and the genotype-phenotype mapping in Eqs. (1)-(2) plus Algorithm 1 is clearly specified. The genuinely new bit is applying AutoML to numerical association rule mining pipelines, which does appear to be missing from the literature, and the experimental effort is real: ten datasets, 30 runs, two outer optimizers, and statistical tests.\n\nWhere it gets soft. First, the P>1 experiments are under-specified. Eq. (1) encodes each preprocessing method as an independent binary switch, and the paper never states the order in which selected methods like MM, ZS, RHC, and DK are applied. Normalizing before versus after discretization changes the result. If the implementation uses a fixed order, that order is part of the pipeline and must be documented; otherwise a single genotype maps to multiple executable pipelines and the P>1 fitness values are not well-defined. This does not sink the P=1 baseline, but it undermines the claim of searching over the full pipeline space.\n\nSecond, the VARDE comparison is indirect and the budget matching is unclear. The paper says it uses the best reported variations of VARDE from [18], but it never states whether the total number of fitness evaluations was matched. Since NiaAutoARM allocates up to 10,000 inner evaluations per pipeline and runs 1,000 outer pipeline evaluations, the comparison could be severely unequal. Table 9's Wilcoxon results are also hard to parse as presented.\n\nThird, alpha and beta in Eq. (3) are never reported, and there is no baseline like a default pipeline or random search over pipelines. Finally, the paper claims a Python implementation but provides no code or repository link.\n\nNone of this destroys the paper. The central mechanism is credible, the empirical work is substantial, and the authors are honest about the computational cost. But the evaluation needs tightening before I would trust the state-of-the-art claim.\n\nSend it to peer review. It is a legitimate engineering contribution with testable claims, and a good referee can push for the missing details. I would expect a major revision, not a rejection. I would not cite it yet, but I would bring it to a reading group if anyone near you works on AutoML or ARM.","headline":"A credible AutoML wrapper for numerical ARM that needs a tighter experimental and specification story, especially for multi-preprocessing pipelines.","tokens_in":17732,"tokens_out":2371,"would_cite":false,"duration_ms":38203,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"NiaAutoARM claims to be the first AutoML system for association rule mining pipelines, encoding the inner algorithm, hyperparameters, preprocessing steps, and metric weights into one continuous vector that a meta-heuristic optimizes.","keywords":["AutoML","Association Rule Mining","Numerical Association Rule Mining","Pipelines","NiaAutoARM","Metaheuristics","Differential Evolution","Particle Swarm Optimization"],"falsifier":"Take a P > 1 pipeline found by NiaAutoARM that selects, for example, both min-max normalization and k-means discretization, run the same inner algorithm with the same hyperparameters on the same data applying the two preprocessing methods in each possible order, and compare the mined rules and fitness values. If the two orders yield different results, then the genotype does not uniquely define the pipeline, contradicting the method's claim to construct well-defined pipelines when P > 1.","tokens_in":16679,"feed_emoji":"⚙️","tokens_out":7577,"duration_ms":72462,"temperature":0.7,"pith_summary":"The paper proposes NiaAutoARM, an automated machine learning method that builds complete numerical association rule mining (NARM) pipelines without manual intervention. Its central claim is that pipeline construction can be cast as a continuous optimization problem, giving the first AutoML solution for association rule mining. A practical reader would care because NARM normally requires an expert to choose preprocessing, mining algorithm, hyperparameters, and evaluation metrics; NiaAutoARM replaces that manual stage with a meta-heuristic search. Experiments on ten UCI datasets indicate that the automatically constructed pipelines match or exceed the quality of rules found by the state-of-the-art VARDE algorithm on several datasets.","feed_headline":"AutoML arrives for association rule mining","feed_subtitle":"NiaAutoARM encodes algorithm, preprocessing, and metrics into one vector and lets metaheuristics search it.","key_machinery":"The carrying mechanism is the genotype-phenotype mapping of Eq. (1): a single real vector $\\mathbf{x}_i = (x_{i,1}, y_{i,1}, y_{i,2}, p_{i,1},\\dots,p_{i,P}, z_{i,1},\\dots,z_{i,M}, w_{i,1},\\dots,w_{i,M})$ with all values in $[0,1]$, decoded by a mapping $\\Gamma$ into algorithm choice, hyperparameters $NP$ and $MAXFES$, active preprocessing methods, metric selection, and metric weights. The outer meta-heuristic optimizes this vector; for each candidate, the inner algorithm mines rules whose fitness is a weighted sum of the selected metrics, while the outer fitness $\\alpha\\cdot\\mathrm{supp} + \\beta\\cdot\\mathrm{conf}$ over $\\alpha+\\beta$ scores the pipeline itself. This two-layer search is what converts AutoML for ARM from a discrete architecture selection problem into a continuous optimization problem.","core_discovery":"On the paper's own terms, the central discovery is that a complete association rule mining pipeline has a fixed-length real-vector encoding: one component selects the inner mining algorithm from a pool (DE, PSO, GA, LSHADE, iLSHADE, jDE), two components set its population size and evaluation budget, P components toggle preprocessing methods on or off, M components choose which rule-quality metrics enter the fitness, and M components weight those metrics. The outer meta-heuristic (DE or PSO in the experiments) searches this continuous space, decoding each candidate into a concrete pipeline, running the inner algorithm, and scoring the resulting rules with a support-confidence fitness. The paper reports that this procedure finds high-quality pipelines across ten UCI datasets and that, in the comparison experiments, the pipelines it selects outperform VARDE with statistical significance in some instances.","pith_inferences":["The paper leaves implicit that its outer fitness in Eq. (3) only rewards support and confidence, while the inner search can use up to six weighted metrics; a user who cares about comprehensibility or coverage should change the outer fitness, not the inner weights, because only the outer fitness decides which pipeline wins.","Because the preprocessing part of the genotype is a set of independent binary switches, a natural unstated extension is to encode the order of preprocessing methods explicitly, or to use a variable-length genotype; that would make the P > 1 pipelines fully deterministic and testable.","The two-layer design is not tied to association rule mining; the same encoding trick could be applied to other pipeline-based data mining tasks, such as subgroup discovery or outlier detection, by swapping the inner heuristic and the evaluation metrics."],"forward_implications":["Non-experts can apply numerical association rule mining to mixed categorical and numerical data without manually choosing preprocessing, algorithm, hyperparameters, or fitness metrics.","Because the genotype is agnostic to the specific optimizer, any population-based meta-heuristic compatible with the underlying algorithm library can serve as the outer pipeline searcher.","Metric weights become part of the search space, so the fitness function itself adapts to each dataset rather than being fixed in advance.","Allowing multiple preprocessing methods (P > 1) can improve pipeline quality at the cost of a larger search space and longer running times.","Dataset-specific pipelines found by the automatic search can beat a single hand-designed state-of-the-art ARM algorithm, supporting the No Free Lunch motivation for per-dataset search."],"supporting_citations":[{"why":"The NiaAML method supplies the AutoML-as-optimization template that NiaAutoARM adapts from classification pipelines to association rule mining.","marker":"[13]"},{"why":"The NiaARM framework provides the inner numerical ARM implementation, including the weighted-sum fitness over rule metrics.","marker":"[15]"},{"why":"The ARM-DE algorithm is the base mining algorithm that the inner pipeline runs.","marker":"[16]"},{"why":"The NiaPy library supplies the pool of nature-inspired algorithms used for both outer and inner heuristics.","marker":"[17]"},{"why":"VARDE is the state-of-the-art ARM algorithm that NiaAutoARM is compared against, and it also provides the hyperparameter domain mapping.","marker":"[18]"},{"why":"The No Free Lunch theorem is cited to justify why algorithm selection is needed, motivating the search over pipelines.","marker":"[7]"}],"fun_headline_variants":["One vector encodes an entire ARM pipeline for AutoML","Metaheuristics search full association-rule pipelines","AutoML for rule mining: from preprocessing to metrics","Fixed-length encoding turns ARM pipelines into search space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a pipeline is fully described by a fixed-length real vector with independent binary preprocessing switches; in particular, when more than one preprocessing method is selected, the encoding does not state their execution order, so such pipelines are only well-defined if order does not affect the outcome or a fixed default order is assumed.","fun_headline_variants_meta":{"raw":{"variants":["One vector encodes an entire ARM pipeline for AutoML","Metaheuristics search full association-rule pipelines","AutoML for rule mining: from preprocessing to metrics","Fixed-length encoding turns ARM pipelines into search space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00026,"raw_usage":{"total_tokens":1524,"prompt_tokens":813,"completion_tokens":711,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":429,"completion_tokens_details":{"reasoning_tokens":650}},"tokens_in":429,"tokens_out":711,"duration_ms":6723,"temperature":1.0,"reasoning_tokens":650,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:59:36.600523+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a P > 1 pipeline found by NiaAutoARM that selects, for example, both min-max normalization and k-means discretization, run the same inner algorithm with the same hyperparameters on the same data applying the two preprocessing methods in each possible order, and compare the mined rules and fitness values. If the two orders yield different results, then the genotype does not uniquely define the pipeline, contradicting the method's claim to construct well-defined pipelines when P > 1.","supporting_citations":[{"cited_title":"Continuous op- timizers for automatic design and evaluation of classification pipelines,","cited_arxiv_id":null,"evidence_quote":"The NiaAML method supplies the AutoML-as-optimization template that NiaAutoARM adapts from classification pipelines to association rule mining."},{"cited_title":"Niaarm: A minimalistic framework for numerical association rule mining,","cited_arxiv_id":null,"evidence_quote":"The NiaARM framework provides the inner numerical ARM implementation, including the weighted-sum fitness over rule metrics."},{"cited_title":"Differential evolution for association rule mining using cate- gorical and numerical attributes,","cited_arxiv_id":null,"evidence_quote":"The ARM-DE algorithm is the base mining algorithm that the inner pipeline runs."},{"cited_title":"NiaPy: Python microframework for building nature-inspired algorithms,","cited_arxiv_id":null,"evidence_quote":"The NiaPy library supplies the pool of nature-inspired algorithms used for both outer and inner heuristics."},{"cited_title":"Variable-length differential evolution for numerical and discrete association rule mining,","cited_arxiv_id":null,"evidence_quote":"VARDE is the state-of-the-art ARM algorithm that NiaAutoARM is compared against, and it also provides the hyperparameter domain mapping."}],"review_version":1}