{"id":"6b575d1b-7427-467f-8c78-772d44f66ecb","arxiv_id":"2501.12362","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper learns adaptive resilience metrics as reward functions using adversarial inverse reinforcement learning and shows these reduce episode lengths versus PPO in cyber-physical power system restoration tasks.","lead":"Researchers used inverse reinforcement learning to automatically learn a resilience metric, a reward function for control actions, from expert-like demonstrations in cyber-physical power systems. The method, called ARM-IRL, was tested on rerouting communication traffic, reconfiguring power distribution feeders, and combined restoration tasks, showing faster goal-reaching than standard reinforcement learning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The learned reward is never validated as a resilience metric; the reported episode-length gains are equally explained by imitation of the heuristic demonstrations, so the central metric claim rests on an untested identification step.","rationale":"The paper is best read as an empirical demonstration that AIRL can be trained on heuristic demonstrations and that the resulting policy shortens episodes relative to PPO in three cyber-physical tasks. The code is available on Code Ocean [31], which is real support for reproducibility, and the AIRL-vs-GAIL sample-efficiency claim has some internal support in Figs. 8, 9, 14, and 15. However, the title and abstract make a stronger claim: that the method quantifies an adaptive resilience metric. That claim requires the learned reward to be more than a function that rationalizes the heuristic demonstrations in one environment. The paper does not provide any such validation. The reader's conditional verdict identifies roughly the same weakness, centered on the heuristic demonstrations being a non-optimal proxy. My concern sharpens that point: even if the demonstrations were optimal, AIRL's reward is identifiable only up to shaping, and the paper never checks the learned reward against any external criterion, such as held-out contingencies, known optimal restoration sequences, or transfer to different threat scenarios. The reported episode-length improvements are therefore consistent with imitation alone and do not by themselves establish the resilience-metric claim. This does not change the conditional verdict; it specifies the condition more precisely: add an external validation of the learned reward, not just confidence intervals, before the central metric claim can be accepted.","tokens_in":15834,"tokens_out":6500,"duration_ms":69890,"concrete_test":"Take the AIRL reward learned on the N6 rerouting task and use it, with no further IRL, as the reward for a PPO agent in two held-out settings: (a) a compromised router excluded from the demonstration set, and (b) simultaneous attacks on two routers. Compare average episode length and successful-restoration rate against PPO with a hand-specified delay/drop reward and against a random baseline. If the AIRL reward does not yield better resilience behavior than the hand-specified reward in at least one held-out setting, the learned function has not been demonstrated to be an adaptive resilience metric. An alternative analytical check is to evaluate the learned reward on the known spanning-tree solutions from [32] and verify that it ranks them above randomly sampled restoration sequences with the same episode length.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that AIRL recovers an adaptive resilience metric, not merely a policy, and that this metric drives the reported performance gains. The load-bearing step is therefore the identification of the learned reward function as a valid resilience metric. That step is never tested. AIRL's theoretical guarantee, as presented in Sec. IV.B.5 following [30], is that the discriminator recovers the reward only up to potential-based shaping; in general, many reward functions rationalize the same demonstrations. Here the demonstrations are generated by heuristics: Alg. 2 for rerouting and spanning-tree search [32] for reconfiguration, which the authors concede in Sec. VII 'might not be optimal.' Consequently, the learned reward may simply be the one that reproduces the heuristic's choices in the training environment; it is not shown to be invariant across threats or contingencies, to transfer to held-out scenarios, or to correspond to an external notion of resilience. All reported evidence (Figs. 9, 11, 15) is episode length on the same training distribution, which imitation alone can explain. Without an external validity check on the reward, the 'adaptive resilience metric' claim is not established even if the policy comparisons are correct.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ARM-IRL, an approach that uses adversarial inverse reinforcement learning (AIRL) to learn an \"adaptive resilience metric\" in the form of a reward function from expert demonstrations, and then uses the learned reward to train a restoration policy. The method is evaluated on three cyber-physical tasks: communication network rerouting, distribution feeder reconfiguration, and a combined cyber-physical critical-load restoration problem on an IEEE 123-bus system with a SimPy/OpenDSS co-simulation environment. The reported results show that policies trained with AIRL achieve lower average episode lengths than forward RL (PPO), GAIL, DAgger, behavioral cloning, and random baselines in these tasks. The paper also visualizes the learned reward functions as functions of selected state and action pairs.","tokens_in":16095,"tokens_out":3454,"duration_ms":34087,"significance":"If the central claim were established, the work would provide a useful data-driven alternative to static, hand-weighted resilience metrics for power distribution cyber-physical systems. The authors build a nontrivial co-simulation environment, provide a Code Ocean repository, and compare several imitation and inverse RL methods, which is a strength. However, the evidence supports the weaker claim that AIRL can imitate heuristic expert demonstrations and reduce episode length; it does not establish that the learned reward is a valid resilience metric, because the identification step is never tested. The paper is therefore more a demonstration of AIRL for these restoration tasks than a validated resilience-quantification method.","major_comments":[{"comment":"The central claim that the reward recovered by AIRL is an \"adaptive resilience metric\" is not tested. As the paper itself notes in §IV.B.5, AIRL recovers the reward only up to potential-based shaping, so many reward functions can rationalize the same expert demonstrations. All of the quantitative evidence in Figs. 9, 11, and 15 is average episode length on the same training distribution, which can be explained by imitation of the heuristic demonstrations alone. To support the metric claim, the learned reward would need to be validated externally, for example by transfer to held-out threat types, contingencies, load profiles, or by comparison against a separately defined resilience index. Without such a test, the paper supports an imitation-learning result, not a resilience-metric result.","section":"§IV.B.5 and §VI"},{"comment":"The expert demonstrations are generated by the authors' own heuristics — Algorithm 2 for rerouting and spanning-tree search [32] for reconfiguration — and §VII concedes that these heuristics \"might not be optimal\" in the combined cyber-physical environment. Since IRL assumes the demonstrations come from an optimal policy (Eq. 4), the learned reward is, by construction, an encoding of those heuristics rather than an independently grounded model of operator resilience objectives. This is a load-bearing issue for the title and abstract claims, and it is not resolved by the sentence in §VII that real operator demonstrations could be incorporated in the future.","section":"§V.B and §VII"},{"comment":"The empirical comparison lacks the statistical detail needed to support the reported gains. The figures plot average episode length with no variance, no number of seeds, and no confidence intervals, and the baseline label \"random PPO agents\" is ambiguous: it is not clear whether the PPO policy is random before training, trained with random weights, or evaluated with random exploration. Hyperparameters for the PPO generator and for the other baselines are not specified, and the sample counts (\"30K transition trajectories\", \"150K transition samples\") are used inconsistently. The claim that AIRL \"is better than the expert\" in §VI.C.4 appears to rest on a single scalar comparison without uncertainty quantification.","section":"§VI, Figs. 7, 9, 11, and 15"}],"minor_comments":[{"comment":"Reference [18] appears to be identical to reference [1]; the duplicate should be removed and the in-text citation corrected.","section":"References"},{"comment":"The component labeled \"Optimization Solver/Outing\" appears to contain a typo; \"Outing\" should likely be \"Routing\" or a similarly meaningful word.","section":"§V.A and Fig. 2"},{"comment":"The text alternates among \"transition trajectories\", \"transition samples\", and \"training samples\"; these terms should be used consistently so the reader can compare sample counts across methods.","section":"§VI.A.4"},{"comment":"The table header is confusing: the columns labeled \"R1,R2\" contain encoded action values rather than clear column names; please add explicit column headings and a description of the encoding.","section":"Table I"},{"comment":"The captions for Fig. 10 describe reward surfaces but do not state which network size, training run, or training sample count produced the visualized reward function; this information is needed for reproducibility.","section":"Fig. 10"},{"comment":"In Eq. (3), the index i is not defined, and the text does not say which candidate metrics R_i are included in the linear adaptive resilience metric; please define the index and the candidate set.","section":"§III.A and Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The available code and co-simulation environment are genuine contributions, and the paper may be publishable if reframed as an empirical study of imitation learning for cyber-physical restoration policies. The main risk is the unsupported identification of the learned reward as a resilience metric; this is not a stylistic issue but a matter of what the evidence establishes. I would encourage the editor to require either an external validation of the reward or a revised framing that drops the metric-identification claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful engineering study applying AIRL to three cyber-physical power system tasks, with the authors honestly flagging the main limitation themselves. What's genuinely new is the application: using adversarial IRL to learn a reward that serves as an adaptive resilience metric for rerouting, reconfiguration, and combined restoration on the IEEE 123-bus system. The paper also ships code on Code Ocean. The empirical comparisons against PPO and random agents show AIRL reducing episode length in most settings, which is a real result if the numbers hold up.\n\nThe soft spots are the ones the stress-test note names. The learned reward is never validated as a resilience metric. Because the expert demonstrations come from the authors' own heuristics (Alg. 2 and spanning tree search [32]), and AIRL recovers reward only up to potential-based shaping, the 'adaptive resilience metric' is, by construction, an encoding of those heuristics in the training environment. The paper acknowledges in Section VII that the heuristics 'might not be optimal' and that the SimPy cyber environment is low fidelity. That's honest, but it means the central claim—that AIRL learns a genuine resilience metric—is not established. The episode-length comparisons, if reported with variance, would show imitation improves over forward RL, but that is a different and weaker claim.\n\nThe statistical presentation is also thin: figures show average episode length with no error bars, no seed counts, and 'random PPO agents' is ambiguous. Hyperparameters for PPO aren't specified. These are fixable but should be addressed before the paper is used as a baseline.\n\nWho this is for: researchers working on cyber-physical power system resilience and on IRL applications where demonstrations come from engineered heuristics. The paper is a solid engineering demonstration, not a method paper; the IRL theory is textbook. It deserves a serious referee because the application is timely, the code is available, and the limitations, while real, are clearly acknowledged and potentially addressable in revision.\n\nMy recommendation: engage with it. A referee should push for error bars, multiple seeds, clear baseline specs, and—crucially—some external validity check of the learned reward, such as transfer to held-out contingencies or comparison against a known resilience index. Without that, the metric claim stays unverified.","headline":"AIRL applied to learn resilience rewards in power distribution is a plausible engineering contribution, but the 'adaptive resilience metric' label overreaches: the learned reward is never validated independently of the heuristic demonstrations that produced it.","tokens_in":16567,"tokens_out":1639,"would_cite":false,"duration_ms":16004,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adversarial inverse reinforcement learning can turn heuristic expert demonstrations into a state-dependent adaptive resilience metric that outperforms a fixed-reward forward RL baseline on cyber-physical restoration…","keywords":["inverse reinforcement learning","adaptive resilience metric","cyber-physical systems","power distribution network reconfiguration","network rerouting","adversarial inverse reinforcement learning","IEEE 123-bus system","imitation learning"],"falsifier":"Run the same AIRL pipeline with demonstrations from a different expert source—for example, recorded operator decisions or a high-fidelity network emulator—and compare the learned reward surfaces and policy episode lengths. If the AIRL policy no longer beats PPO, or the reward function changes drastically with the demonstrator, then the adaptive resilience metric is an artifact of the particular heuristic rather than a general resilience objective.","tokens_in":15655,"feed_emoji":"⚡","tokens_out":7126,"duration_ms":64246,"temperature":0.7,"pith_summary":"This paper tries to establish that the resilience of a cyber-physical power system can be captured by a single adaptive metric learned from data instead of a hand-weighted static formula. The authors model each restoration task as a Markov decision process with an unknown reward, and use adversarial inverse reinforcement learning (AIRL) to recover that reward from demonstrations produced by heuristic algorithms. They claim that the policy trained on the learned reward reaches its goal in fewer average steps than a forward reinforcement learning baseline (PPO) on three tasks: communication rerouting under denial-of-service, distribution feeder reconfiguration after outages, and combined cyber-physical critical-load restoration on the IEEE 123-bus system. If correct, this would give grid operators a way to derive response and recovery objectives from observed operator behavior without specifying weights in advance.","feed_headline":"Inverse RL learns adaptive grid resilience metric","feed_subtitle":"AIRL cuts average steps to restore service in rerouting, reconfiguration, and combined cyber-physical tasks.","key_machinery":"The load-bearing mechanism is adversarial inverse reinforcement learning (AIRL), a GAN-style method that learns the reward function while training a policy. A discriminator network is trained to distinguish expert state-action pairs from generator rollouts, and the generator, a PPO policy, is trained to maximize the reward implied by the discriminator; the discriminator's output is interpreted as the resilience metric. AIRL's design choice of parameterizing the reward as a function of state rather than action is what the paper credits for producing a disentangled reward that is less tied to environment dynamics. Expert demonstrations come from two heuristic sources: Algorithm 2, which reroutes traffic around compromised routers by choosing the next hop with the lowest packet drop rate, and a spanning-tree search method that produces switching sequences for distribution network reconfiguration.","core_discovery":"The central claim, stated on the paper's own terms, is that AIRL can learn an adaptive resilience metric—a state- and time-dependent reward function represented by a neural network—from heuristic expert demonstrations, and that the resulting policy performs better than forward RL. Across the rerouting, reconfiguration, and combined cyber-physical problems, the AIRL-trained agent reduces the average number of steps to reach the goal state compared with a PPO baseline, and it does so with fewer transition samples than GAIL. The learned reward functions can be visualized against selected state-action pairs, showing, for example, that the reward for a router action rises as the packet drop rate of a compromised router increases, and that the reconfiguration reward grows as more critical loads are restored. The paper also reports that AIRL is more robust than GAIL in the combined cyber-physical setting, where GAIL's performance deteriorates.","pith_inferences":["This is an extension of the disentanglement claim: if the learned reward is truly a function of state only, it should transfer to larger network topologies without retraining; the paper does not test this directly.","Because the 'expert' is a heuristic, the learned metric inherits the heuristic's notion of resilient behavior; a practical validation—again beyond the paper—would retrain on logged operator actions from a utility and compare the reward surfaces.","The comparison with PPO is not purely algorithmic: PPO is both the baseline and the generator inside AIRL, so part of the gain may come from the learned reward shaping PPO's exploration rather than from AIRL's training objective alone; this is an interpretation, not a paper claim."],"forward_implications":["If the learned reward is a valid adaptive resilience metric, grid operators can obtain response policies from demonstrations without committing to a fixed weighted sum of resilience indices.","AIRL's sample efficiency relative to GAIL means the approach can be applied to larger feeder and communication networks where collecting expert trajectories is expensive.","The learned reward can be visualized as a function of state and action, offering operators a view of what the agent treats as resilient behavior rather than a black-box policy.","The same AIRL framework extends to other cyber-physical control problems, such as volt-var control, automatic generation control, and automatic voltage regulation, as the conclusion states.","The combined cyber-physical MDP shows that a single learned reward can coordinate actions across two asynchronous simulators to reach a joint goal."],"supporting_citations":[{"why":"Supplies the AIRL algorithm that recovers the reward function used as the adaptive resilience metric.","marker":"[30]"},{"why":"Provides the GAIL baseline and the occupancy-measure matching objective that frames the GAN training.","marker":"[29]"},{"why":"Provides the DAgger imitation learning algorithm used as another comparison method.","marker":"[28]"},{"why":"Gives the spanning-tree search that generates expert switching sequences for reconfiguration demonstrations.","marker":"[32]"},{"why":"Defines the co-simulation RL environment for the three MDPs and the expert-rollout loop.","marker":"[24]"},{"why":"Provides the categorization of IRL methods that motivates the choice of AIRL.","marker":"[26]"},{"why":"Motivates using RL for response and recovery, framing the learned reward as an adaptive metric.","marker":"[9]"},{"why":"Earlier static cyber-physical security assessment metric that this work positions as non-adaptive and aims to replace.","marker":"[7]"}],"fun_headline_variants":["Adaptive metric via IRL beats static resilience measures","IRL derives adaptive resilience metric from expert actions","Learned adaptive reward speeds grid restoration tasks","Expert-demo IRL learns adaptive metric for grid resilience"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the heuristic algorithms used to generate expert demonstrations are a faithful stand-in for a real operator's resilience objectives; if those demonstrations do not encode true resilience preferences, the learned reward is an artifact of the heuristics rather than a valid resilience metric.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive metric via IRL beats static resilience measures","IRL derives adaptive resilience metric from expert actions","Learned adaptive reward speeds grid restoration tasks","Expert-demo IRL learns adaptive metric for grid resilience"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1389,"prompt_tokens":943,"completion_tokens":446,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":385}},"tokens_in":559,"tokens_out":446,"duration_ms":5455,"temperature":1.0,"reasoning_tokens":385,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:13:54.855830+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same AIRL pipeline with demonstrations from a different expert source—for example, recorded operator decisions or a high-fidelity network emulator—and compare the learned reward surfaces and policy episode lengths. If the AIRL policy no longer beats PPO, or the reward function changes drastically with the demonstrator, then the adaptive resilience metric is an artifact of the particular heuristic rather than a general resilience objective.","supporting_citations":[{"cited_title":"Distribution system restoration with microgrids using spanning tree search,","cited_arxiv_id":null,"evidence_quote":"Gives the spanning-tree search that generates expert switching sequences for reconfiguration demonstrations."},{"cited_title":"Reinforcement learning environment for cyber-resilient power distribution system,","cited_arxiv_id":null,"evidence_quote":"Defines the co-simulation RL environment for the three MDPs and the expert-rollout loop."},{"cited_title":"A survey of inverse reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the categorization of IRL methods that motivates the choice of AIRL."},{"cited_title":"Reinforcement Learning for Feedback-Enabled Cyber Resilience","cited_arxiv_id":"2107.00783","evidence_quote":"Motivates using RL for response and recovery, framing the learned reward as an adaptive metric."},{"cited_title":"CP-SAM: Cyber- physical security assessment metric for monitoring microgrid resiliency,","cited_arxiv_id":null,"evidence_quote":"Earlier static cyber-physical security assessment metric that this work positions as non-adaptive and aims to replace."}],"review_version":1}