{"id":"66a0faaf-5a68-48ff-aae0-f26df3681249","arxiv_id":"2505.21330","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"UGCE reuses and repairs an evolved population of counterfactuals when user constraints change, cutting runtime versus recomputing from scratch, though success rates drop on some datasets.","lead":"The paper introduces a genetic algorithm framework that updates counterfactual explanations when a user adds or changes constraints, by reusing and repairing the previous search population instead of restarting. It reports large runtime savings across five benchmark datasets, but the method finds valid counterfactuals for far fewer cases on two datasets and the source code link is missing.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central quality claim is load-bearing: Table II shows UGCE-Incremental success rates of 47.86% on COMPAS and 57.35% on AdultCA, so 'maintaining high-quality solutions' fails on two of five datasets where no valid counterfactual is found for roughly half of instances.","rationale":"The reader's weakest-assumption diagnosis is consistent with the observed success-rate drops: warm-starting from a population evolved under C_t is only useful if that population retains diversity and relevance after C_{t+1} shifts the feasible region. I partially agree, because the strongest evidence for that failure is Table II's success-rate collapse on COMPAS and AdultCA. However, the single most load-bearing concern is not the mechanism but the abstract's unqualified quality promise. The central claim is a conjunction: faster AND high-quality. The runtime part is convincingly demonstrated; the quality part is contradicted by the paper's own validity metric on two of five datasets. This is an internal inconsistency between the abstract and Table II, not a disagreement with external consensus. It could be repaired by qualifying the claim to datasets with mild constraint shifts, by adding a success-rate constraint as an evaluation criterion, or by presenting a Pareto trade-off between runtime and coverage. Since the reader already issued a CONDITIONAL verdict and my concern reinforces that condition, no verdict change is needed. I do not treat the missing-code or statistics issues as the primary load-bearing concern because the success-rate contradiction alone is sufficient to condition the main claim.","tokens_in":12661,"tokens_out":3628,"duration_ms":44068,"concrete_test":"Reproduce Table II on COMPAS and AdultCA, but give UGCE-Incremental the same per-instance wall-clock budget as UGCE-Baseline (e.g., UGCE-Baseline total time divided by its number of instances) and recompute CF%. If success rate remains substantially below the baseline (e.g., more than 20 percentage points lower), the 'maintaining high-quality solutions' claim is not supported. Additionally, report a runtime-versus-success-rate curve by varying the number of generations; if the incremental method's success rate does not converge toward the baseline within a modest multiple of the baseline runtime, the quality loss is intrinsic to the warm-start strategy rather than just a budget artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims UGCE 'significantly improves computational efficiency while maintaining high-quality solutions compared to a static, non-incremental approach.' The efficiency half is supported by Table II. The quality half is not. Success rate (CF%) is a validity quality measure reported in Table II: UGCE-Baseline reaches 93.11% on COMPAS and 99.99% on AdultCA, while UGCE-Incremental reaches 47.86% and 57.35%. A method that returns no valid counterfactual for over half of COMPAS instances and about 43% of AdultCA instances cannot be described as maintaining high-quality solutions. The paper itself acknowledges 'slightly lower success rates' in Section IV-B, but a 45- and 43-point drop is not slight. Since the abstract makes a paired efficiency/quality claim, the quality part is load-bearing: if a user cannot obtain a counterfactual, speed is moot. The paper does not define an acceptable success-rate threshold, a quality budget, or a Pareto analysis trading runtime against coverage, so the central claim is internally contradicted by its own headline table on two of five datasets.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents UGCE, a genetic-algorithm framework for generating counterfactual explanations when the user's feasibility constraints change over time. The method retains the evolved population from the previous step and repairs individuals that violate the new constraints, then continues the evolutionary search, rather than restarting from scratch. The evaluation compares UGCE-Incremental with UGCE-Baseline (full restart) and DiCE on five datasets, reporting runtime, number of generations, success rate (the percentage of negatively classified instances for which a valid counterfactual is found), proximity, and sparsity. The central claim is that incremental warm-starting substantially reduces runtime while maintaining solution quality.","tokens_in":12906,"tokens_out":8669,"duration_ms":90761,"significance":"If the runtime figures in Table II hold, UGCE-Incremental shows a real and practically useful efficiency gain over a restart baseline on all five datasets, with low variance relative to DiCE. The paper's main strength is that these are measured runtimes over a concrete implementation with code made available, and the comparison includes a state-of-the-art static method (DiCE). There is no circularity: the efficiency result is an empirical comparison of measured runtimes, not a consequence of fitting. The central paired claim is not yet established, however: on COMPAS and AdultCA the incremental success rate falls to 47.9% and 57.4%, respectively, so the positive result should be framed as an efficiency-coverage tradeoff unless additional evidence or a quality budget is provided.","major_comments":[{"comment":"The abstract's claim that UGCE 'maintains high-quality solutions' is not supported by Table II. On COMPAS, UGCE-Incremental finds valid counterfactuals for 47.86% of instances versus 93.11% for UGCE-Baseline; on AdultCA the corresponding figures are 57.35% versus 99.99%. Section IV-B describes these as 'slightly lower success rates,' but a drop of 45 and 43 percentage points means that the incremental method fails to produce any valid counterfactual for roughly half (COMPAS) and 43% (AdultCA) of the explained instances. Since a speedup is of little use when no valid counterfactual is returned, the paper must either add an explicit quality/coverage budget with a Pareto analysis of runtime against CF(%) or revise the central claim to describe an efficiency-coverage tradeoff.","section":"Abstract; Section IV-B; Table II"},{"comment":"The reported success rates for UGCE-Incremental are inconsistent across the two tables. Table II lists CF(%) of 95.36, 92.43, and 57.35 for German Credit, HELOC, and AdultCA under UGCE-Incremental, while Table IV, which also attributes results to UGCE-Incremental, lists 98.81, 99.46, and 99.99 for those datasets under every ordering. The protocols appear to differ (Table II uses only immutability constraints; Table IV uses three-step sequences of all constraint types), but this is not explained. The two tables cannot both describe the same method without a concrete reconciliation; otherwise the reader cannot tell which success-rate figure represents the incremental algorithm.","section":"Table IV vs Table II"},{"comment":"Section IV-C states that the violator-fixing strategy 'consistently outperforms' the random-restart alternative in counterfactual success rate. Table III contradicts this on AdultCA: Fix achieves 57.35% CFs versus 57.48% for Random, and the reported p-value is 5.2e-1, so the random strategy is slightly better and the difference is not significant. The same holds on HELOC, where the p-value is 7.5e-1. The text should be changed to say the fixing strategy gives statistically significant improvements on German Credit, COMPAS, and Adult, and is statistically equivalent on HELOC and AdultCA. The efficiency advantage of fixing remains supported.","section":"Section IV-C; Table III"},{"comment":"The central mechanism, line 13 of Algorithm 1 ('Repair Pt to satisfy Ct+1'), is not defined. There is no description of how each constraint type (immutability, range, directionality) is enforced during repair, how invalid individuals are modified, what happens when the repaired population is empty or has no diversity, what convergence/early-stopping criterion is used, or what GA hyperparameters (population size, crossover and mutation rates, number of generations) are used beyond the claim that they follow DiCE. Without these details, the runtime and CF(%) results are tied to an underspecified implementation, and the claimed dominance of the repair-based warm start cannot be independently reproduced or attributed to the method rather than to particular parameter choices.","section":"Algorithm 1; Section III-B"}],"minor_comments":[{"comment":"The dynamic problem statement lists only feasibility and label-flip conditions and omits the cost-minimization objective that appears in the static formulation; please add the objective for the updated constraint set.","section":"Section II, Problem 1"},{"comment":"The entry '1 .12' is not formatted like the other CFs(%) entries; clarify whether this is 1.12% and note that only one run is reported.","section":"Table II, AdultCA row"},{"comment":"Table II and Table I report different instance counts for HELOC, COMPAS, and Adult (365 vs 368, 505 vs 550, 2,049 vs 2,063); please define exactly which instances are included in each experiment.","section":"Table I and Table II"},{"comment":"The notation 'CFEs %' is used in Section IV-C while Section IV-B uses 'CFs (%)'; standardize the notation.","section":"Section IV-C"},{"comment":"The sentence 'Ramsey and Grefenstette [30] says' should be 'state'; also the phrase 'and section VII presents' in the introduction is inconsistent with the actual section numbering, where Section VI is Summary and Section VII is Future Directions.","section":"Section V"},{"comment":"The code availability footnote references 'Project Repository' without a working URL in the visible text; please include the actual link.","section":"Data availability"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper's efficiency result is plausible and the idea is reasonable, but the central abstract claim overreaches relative to Table II. The authors should be asked to reframe the contribution as an efficiency/coverage tradeoff and to reconcile Tables II and IV. I do not see a fundamental correctness flaw in the method; the issues are experimental framing and missing algorithmic detail, so I regard major revision as appropriate rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate, modest extension of dynamic GA techniques to interactive counterfactual generation, and the efficiency result is real. But the abstract's 'maintaining high-quality solutions' claim is contradicted by the paper's own Table II on COMPAS and AdultCA, so the framing needs to be fixed before it can be trusted.\n\nWhat's new: the paper takes the warm-start-and-repair machinery from dynamic evolutionary optimization (citations 23-31) and applies it to counterfactual generation under user-updated feasibility constraints (immutability, range, directionality). That specific combination is not in the cited prior work, and the problem formalization in Section II is clean. The paper also does the right control experiment: comparing violator-fixing warm start against random reinitialization (Table III), which isolates the value of repair. The runtime wins in Table II are consistent across all five datasets, and the constraint-order stability analysis in Table IV is a nice practical check.\n\nSoft spots. The biggest one is the success-rate drop. UGCE-Incremental finds a valid counterfactual for only 47.86% of COMPAS instances and 57.35% of AdultCA instances, versus 93.11% and 99.99% for the baseline. The paper calls these 'slightly lower' in Section IV-B, but a 45-point and 43-point drop is not slight, and it directly undercuts the 'high-quality solutions' phrase in the abstract. Since the whole point of a counterfactual is to give the user a valid recourse, a speedup that loses half the solutions is not an unqualified win. The paper needs a success-rate threshold or a Pareto-style quality-runtime tradeoff to justify the claim.\n\nSecond, the statistical reporting in Table III is shaky. Several p-values are printed as '0.0 x 100', which is either a rounding artifact or a zero truncated to one significant figure, and one p-value for German Credit CFs is 3.9e-1, which is not significant, yet the text says 'statistically significant advantages' without noting the exceptions. That needs to be cleaned up.\n\nThird, the code link is missing. The footnote says 'source code is available online' but gives no URL. For reproducibility, that has to be fixed.\n\nFourth, the constraint-type analysis in Table V is purely internal. There is no external baseline for directionality or range, so it is descriptive rather than comparative. The paper acknowledges this, but the 'preliminary' caveat should be explicit in the abstract or conclusions.\n\nWho this is for: researchers working on interactive counterfactual explanation or recourse; also anyone building user-in-the-loop XAI systems. The paper deserves a serious referee, but not as-is. The efficiency claim is solid; the quality claim needs recalibration, and the verification gaps need closing. I would ask for major revision before acceptance.","headline":"A genuine incremental-counterfactual mechanism with a real runtime win, but the abstract's quality claim is contradicted by the paper's own success-rate numbers on COMPAS and AdultCA.","tokens_in":13459,"tokens_out":2189,"would_cite":true,"duration_ms":21701,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"UGCE reuses and repairs the previously evolved counterfactual population after each user constraint change, so interactive refinement avoids restarting the search.","keywords":["counterfactual explanations","genetic algorithms","incremental optimization","dynamic constraints","warm-start","user-guided refinement","feasibility constraints","explainable AI"],"falsifier":"Run the same incremental loop on constraint sequences where each new set excludes most of the previously feasible counterfactuals, quantified by the fraction of formerly valid solutions that remain valid; if the repaired population's success rate approaches or falls below that of a random reinitialization on such sequences, the warm-start premise fails precisely when the feasible region moves substantially.","tokens_in":1325,"feed_emoji":"🔄","tokens_out":2369,"duration_ms":78152,"temperature":0.7,"pith_summary":"The paper tries to establish that counterfactual explanations can be updated across rounds of user constraint changes without restarting the optimization from scratch. The key move is to carry over the already-evolved population, repair only the members that violate the new constraints, and resume the genetic search from that warm-started population. If this is right, interactive tools become practical: users can tighten or loosen feasibility requirements step by step instead of committing to one fixed constraint set. On five benchmark datasets the scheme shows much lower runtime than regeneration, stable performance under different constraint orderings, and better quality than a random reinitialization warm start, though its success rate drops on some datasets such as COMPAS and AdultCA.","feed_headline":"Reuse, don't restart: faster counterfactual updates","feed_subtitle":"UGCE carries each evolved population into the next round, cutting runtime while keeping high-quality explanations.","key_machinery":"The central mechanism is the incremental warm-start loop in Algorithm 1: after each user update, line 13 repairs the existing population $P_t$ to satisfy $C_{t+1}$ and continues evolution from that repaired population. Around this loop sits a genetic algorithm, a population-based evolutionary optimizer that keeps a set of candidate counterfactuals and improves them by selection, crossover, and mutation, with a fitness function that jointly rewards a flipped prediction, proximity to the original instance, and sparsity of changed features. The repair operator carries the search state forward across constraint changes, and the comparison against a random reinitialization warm start isolates the contribution of repair.","core_discovery":"UGCE claims that iterative counterfactual generation under user-driven feasibility changes is best treated as a warm-started evolutionary search. After the user revises the constraint set from $C_t$ to $C_{t+1}$, the algorithm keeps the population $P_t$ already evolved under $C_t$, repairs only the individuals that violate $C_{t+1}$, and resumes the genetic search with that repaired population as the initial gene pool. The paper argues that this repair-don't-restart policy is the load-bearing difference: it consistently beats a warm start that re-samples random individuals and it is far faster than regenerating from scratch, with only a moderate reduction in the fraction of instances for which a feasible counterfactual is found. A secondary claim is that runtime and quality are largely insensitive to the order in which constraint types are introduced, and that individual constraint types differ in difficulty, with directionality constraints being the hardest across datasets.","pith_inferences":["Editorial: if the feasible region moves far enough, a natural extension would be to measure the overlap between $F(C_t)$ and $F(C_{t+1})$ and inject fresh diversity only when that overlap is low, rather than always relying on repair alone.","Editorial: the reported speedups suggest UGCE could sit behind a slider- or chat-style interface with second-level response times; a user study measuring whether people actually refine constraints more effectively could test this claimed practicality.","Editorial: the dataset-dependent failure of directionality constraints could be converted into a design signal, warning the user in advance when a one-sided restriction is unlikely to admit any valid counterfactual."],"forward_implications":["Repeated constraint refinement becomes cheap enough for interactive tools with several rounds of user feedback, since each update resumes from the previous population rather than reinitializing.","The order in which users add immutability, range, and directionality constraints has little effect on runtime or quality for this method, per the three orderings tested.","Starting from a repaired population rather than fresh random samples is the strategy validated for preserving quality and speed after constraint updates.","Constraint types are not interchangeable: directionality constraints lower success rates most, so explanation systems may need type-specific planning.","On datasets where the new feasible region overlaps the old one only weakly, such as COMPAS and AdultCA, the incremental success rate falls below 60 percent, marking a boundary of the repair strategy."],"supporting_citations":[{"why":"Introduces counterfactual explanations as minimal feature perturbations that flip a prediction, giving the problem setup UGCE builds on.","marker":"[14]"},{"why":"Formulates feasible and actionable counterfactual explanations, which motivates the feasibility-constrained setting UGCE targets.","marker":"[15]"},{"why":"Defines the genetic-algorithm counterfactual baseline whose default hyperparameters UGCE adopts and whose from-scratch regeneration is the main comparison point.","marker":"[16]"},{"why":"Establishes random-immigrant and related diversity strategies for genetic algorithms in changing environments, background supporting the repair-based design.","marker":"[25]"},{"why":"Shows memory-based immigrants help genetic algorithms adapt to changing environments, one of the dynamic-GA mechanisms behind warm-starting.","marker":"[31]"},{"why":"Surveys evolutionary strategies for dynamic optimization, supplying the population-repair and memory-reuse rationale that UGCE extends to user constraints.","marker":"[39]"}],"fun_headline_variants":["UGCE repairs, not restarts, for faster counterfactuals","Incremental counterfactuals via population reuse","Warm-start evolution cuts counterfactual runtime","Repair-don't-restart: efficient counterfactual updates"],"cache_read_input_tokens":15616,"weakest_assumption_plain":"The efficiency claim rests on the assumption that the population evolved under the old constraints is largely still useful after the constraints change, so repairing the violators preserves enough progress to beat restarting from scratch.","fun_headline_variants_meta":{"raw":{"variants":["UGCE repairs, not restarts, for faster counterfactuals","Incremental counterfactuals via population reuse","Warm-start evolution cuts counterfactual runtime","Repair-don't-restart: efficient counterfactual updates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000994,"raw_usage":{"total_tokens":4176,"prompt_tokens":873,"completion_tokens":3303,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":3236}},"tokens_in":489,"tokens_out":3303,"duration_ms":22740,"temperature":1.0,"reasoning_tokens":3236,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:29:37.449840+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same incremental loop on constraint sequences where each new set excludes most of the previously feasible counterfactuals, quantified by the fraction of formerly valid solutions that remain valid; if the repaired population's success rate approaches or falls below that of a random reinitialization on such sequences, the warm-start premise fails precisely when the feasible region moves substantially.","supporting_citations":[{"cited_title":"Counterfactual explanations without opening the black box: Automated decisions and the gdpr,","cited_arxiv_id":null,"evidence_quote":"Introduces counterfactual explanations as minimal feature perturbations that flip a prediction, giving the problem setup UGCE builds on."},{"cited_title":"Face: feasible and actionable counterfactual explanations,","cited_arxiv_id":null,"evidence_quote":"Formulates feasible and actionable counterfactual explanations, which motivates the feasibility-constrained setting UGCE targets."},{"cited_title":"Explaining machine learning classifiers through diverse counterfactual explanations,","cited_arxiv_id":null,"evidence_quote":"Defines the genetic-algorithm counterfactual baseline whose default hyperparameters UGCE adopts and whose from-scratch regeneration is the main comparison point."},{"cited_title":"Genetic algorithms for changing environments,","cited_arxiv_id":null,"evidence_quote":"Establishes random-immigrant and related diversity strategies for genetic algorithms in changing environments, background supporting the repair-based design."},{"cited_title":"Memory-based immigrants for genetic algorithms in dynamic environments,","cited_arxiv_id":null,"evidence_quote":"Shows memory-based immigrants help genetic algorithms adapt to changing environments, one of the dynamic-GA mechanisms behind warm-starting."},{"cited_title":"Designing evolutionary algorithms for dy- namic optimization problems,","cited_arxiv_id":null,"evidence_quote":"Surveys evolutionary strategies for dynamic optimization, supplying the population-repair and memory-reuse rationale that UGCE extends to user constraints."}],"review_version":1}