{"id":"d3352c56-7433-4083-a372-b35f6631a0c7","arxiv_id":"2505.00055","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A pruned multi-agent PPO algorithm with exploration incentives is designed to approximate the Stackelberg equilibrium of a vehicular AI twin migration game.","lead":"Electric vehicles rely on cloud-hosted AI helpers, called twins, that must move between roadside computers as the car travels. This paper proposes a game-based pricing model and a slimmed-down multi-agent reinforcement learning algorithm to automate that migration while using fewer computing resources.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (38) collapses to p^t = p^{t1} whenever the intrinsic reward is nonnegative, so the core self-adaptive pruning claim is not implemented by the algorithm as written.","rationale":"The reader's REJECT is correct, but the most load-bearing vulnerability is not the quasi-concavity gap named in the reader's weakest_assumption. The equilibrium analysis is indeed flawed: Theorem 1's proof derives a second derivative of the AV utility while the theorem statement refers to a game among RSUs, and the cross-partial terms introduced by Phi_i and beta_i are never examined. However, even a complete equilibrium proof would not rescue the paper if the pruning algorithm does not adapt to exploration incentives. The collapse of Eq. (38) is directly checkable from the manuscript and undermines the abstract's and contribution list's central claim. I therefore keep the reader's REJECT but assign the largest weight to this internal inconsistency. Agreement with the reader is partial because the rationale mentions Eq. (38), but the formally identified weakest assumption is a different equilibrium issue.","tokens_in":21746,"tokens_out":6220,"duration_ms":65590,"concrete_test":"Symbolically evaluate Eq. (38) with phi = 0.1, r'_k,int = 0 and r'_k,int = 0.7 at a fixed training step t, using p^{t1} and p^{t2} from Eqs. (39)-(40). Both evaluations return the same p^{t1}, showing p^t is independent of r'_k,int. As a runtime check, instrument Algorithm 3 to log p^t and r'_k,int across training; if p^t remains constant while r'_k,int varies, the self-adaptive claim is refuted. A corrected formula or a clear specification of a negative-phi regime would be needed to settle the issue in the authors' favor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is self-adaptive dynamic structured pruning driven by individual exploration incentives. The mechanism is defined by Eq. (38): p^t = min(max(p^{t1}*(1 + phi*r'_k,int), p^{t2}*(1 + phi*r'^{t-1}_k,int)), p^{t1}). Since the JS divergence r'_k,int in Eq. (41) lies in [0,1] and phi is a sensitivity hyperparameter, for any nonnegative phi the first argument of the inner max is at least p^{t1}; hence the inner max is at least p^{t1}, and the outer min forces p^t = p^{t1} identically. The pruning rate therefore never depends on the exploration incentive, and Eq. (37) inherits only the static time schedule from Eq. (39). Consequently, Fig. 10 cannot demonstrate exploration-adaptive pruning; it compares a fixed progressive-pruning PPO variant against baselines. This is not a tuning issue but an internal inconsistency in the defining equation of the headline algorithm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the problem of migrating Vehicular Embodied Agent AI Twins (VEAATs) between roadside units in vehicular embodied AI networks. It proposes a multi-leader multi-follower Stackelberg game that couples AV and RSU utilities with social influence, service complementarity/substitutability, and a matching probability based on an image-quality immersion metric. On the learning side, it introduces TinyMA-IEI-PPO, a MAPPO variant that adds an intrinsic exploration incentive (measured as Bayesian surprise estimated by a CVAE) and a self-adaptive dynamic structured pruning scheme claimed to adjust the pruning threshold based on individual exploration incentives. The authors prove (or claim) existence and uniqueness of the Stackelberg equilibrium via standard-function arguments, and present simulations on an embedded platform showing convergence and closeness to the computed equilibrium. The core algorithmic novelty is the self-adaptive pruning mechanism described by Eq. (38).","tokens_in":22101,"tokens_out":5032,"duration_ms":53294,"significance":"If the pruning mechanism worked as described and the equilibrium proofs were complete, the paper would offer a practical way to deploy lightweight multi-agent DRL on resource-constrained vehicles, which is a real need in vehicular edge intelligence. The intrinsic-exploration module and the time-window neuron-importance metric are useful ideas. However, the central contribution is internally inconsistent: Eq. (38) reduces the adaptive pruning rate to the static progressive schedule, so the headline claim of exploration-driven pruning is not implemented by the algorithm as written. In addition, the game-theoretic uniqueness proofs are incomplete. These are load-bearing issues, and the numerical evaluation cannot compensate because it is largely a self-consistency check against the paper's own equilibrium model. The manuscript in its current form does not support its central claims.","major_comments":[{"comment":"The self-adaptive pruning mechanism is a mathematical no-op. In Eq. (38), p^t = min(max(p^{t1}*(1 + φ r'_k,int), p^{t2}*(1 + φ r'^{t-1}_k,int)), p^{t1}). Since Eq. (41) states that the JS divergence r'_k,int lies in [0,1], and φ is a sensitivity hyperparameter that is presumably nonnegative, p^{t1}*(1 + φ r'_k,int) ≥ p^{t1}. Therefore the inner max is always at least p^{t1}, and the outer min with p^{t1} forces p^t = p^{t1} identically for every time step. Consequently, the pruning rate never depends on the exploration incentive, and the threshold ψ in Eq. (37) follows only the static progressive schedule given by p^{t1}. This invalidates the paper's central claim that the algorithm 'adapts to changes in individual exploration incentives during training, dynamically formulating pruning strategies.' The mechanism must be redesigned so that p^t can actually vary with r'_k,int, and the experimental results in Fig. 10 must be re-run against the corrected mechanism.","section":"Section V-C, Eq. (38)"},{"comment":"The proof of existence and uniqueness of the follower-level Nash equilibrium is incomplete. Theorem 1 asserts quasi-concavity of U_i^F based only on the diagonal second derivative ∂²U_i^F/∂b_ij² < 0 in Eq. (15). The utility in Eq. (9) contains bilinear cross-terms such as ζ_ik b_ij b_kj and η_js b_ij b_is; the Hessian has nonzero off-diagonal entries, and a negative diagonal does not imply quasi-concavity of the full function over the vector b_i. Furthermore, the constraint Σ_j θ_ij T_ij ≤ T_i^max in problem P1 involves T_ij, which depends on b_ij through a logarithm in the denominator (and on the nonconvex M/M/1 queuing term), so the feasible strategy set is not shown to be convex. Theorem 2's standard-function argument also assumes without proof that the positive social/complementarity terms dominate the negative substitutability terms, and Eq. (18) does not provide a clean proof of scalability. Because the uniqueness of the follower equilibrium is a load-bearing step for Theorem 3, the Stackelberg equilibrium uniqueness claim is not established.","section":"Section IV-B-1, Theorem 1 and Theorem 2"},{"comment":"The numerical evaluation is not sufficient to support the claimed contributions. The baselines are compared against 'the SE value' computed from the paper's own utility model, which makes the approximation-to-equilibrium claim a self-consistency check rather than a validation against an external optimum; the authors should also compare with a direct numerical solver (e.g., iterative best response) or report the gap to the true equilibrium of the system. More importantly, since Eq. (38) collapses to p^t = p^{t1}, the experiments in Fig. 10 cannot demonstrate exploration-adaptive pruning; they only compare a fixed progressive-pruning schedule against other fixed-schedule baselines. The paper would need new experiments that directly measure how the pruning rate and threshold change with the intrinsic reward, and that compare the corrected adaptive mechanism against static schedules.","section":"Section VI, Figs. 3-10"}],"minor_comments":[{"comment":"The algorithm name is used inconsistently: the title and abstract say TinyMA-IEI-PPO, while the main text (e.g., Section V and Fig. 3 caption) refers to TinyMA-IMI-PPO. Please standardize.","section":"Throughout (abstract/title vs. Sections V-VI)"},{"comment":"There is a typo in the displayed equation: 'Ωt,(l)n t' should read 'Ωt,(l)n'.","section":"Eq. (35)"},{"comment":"The text says 'the key parameters of the experiment are similar to [12] [23]' but does not provide a table of parameter values. Please include a table with the simulation parameters and their settings, including the hyperparameters c1, c3, φ, λ, and the pruning schedule values.","section":"Section VI, first paragraph"},{"comment":"The caption mentions an '85& pruning rate'; this is presumably a typo for '85% pruning rate'.","section":"Section VI, Fig. 10 caption"},{"comment":"The CVAE objective in Eq. (31) is introduced without derivation or reference to the exact formulation used in prior work [26]. Please clarify how the variational lower bound is obtained and how the two KL terms are combined, since this objective is central to the intrinsic-reward definition.","section":"Section V-B-2, Eq. (31)"}],"recommendation":"reject","confidential_remarks":"The paper's main innovation is not operational: Eq. (38) makes the self-adaptive pruning rate identically equal to the static schedule, so the headline contribution is empty. The equilibrium proofs are sketches with substantial gaps, and the simulations are run only against the paper's own model. The manuscript also has many typos and inconsistencies (algorithm name, 'systerm', '85&'), which suggests it was not ready for review. I would not encourage a revise-and-resubmit unless the authors redesign the pruning rule, complete the equilibrium proofs, and provide an external validation of the equilibrium approximation; those changes would amount to a new paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou should know that the paper's headline contribution doesn't survive contact with its own equations. The self-adaptive pruning mechanism defined by Eq. (38) is supposed to adjust the pruning rate according to exploration incentives, but because the JS divergence r' is in [0,1] and phi is nonnegative, the first argument of the inner max is always at least p^{t1}, forcing the outer min to set p^t = p^{t1} identically. In other words, the pruning rate never adapts—it just follows the static schedule from Eq. (39). This is not a tuning issue; it's an internal inconsistency in the defining equation of the headline algorithm.\n\nThat said, the paper is not a total loss. The system model is elaborate and the application scenario (VEAAT migration in vehicular embodied AI networks) is well motivated. Running the experiments on a Jetson Orin Nano is a nice practical touch, and the exploration-incentive module via Bayesian surprise and CVAE is a reasonable adaptation of prior work. The authors also build transparently on their own earlier papers and on [26], which is appropriate in this subfield.\n\nThe soft spots beyond Eq. (38) are real. Theorem 1 is mislabeled: it claims to prove existence of Nash equilibrium among RSUs but then differentiates the AV utility function. The quasi-concavity proof for the AV utility checks only the diagonal second derivative, ignoring the cross-partials introduced by the social-influence and complementarity terms, so the uniqueness of the Stackelberg equilibrium is not established. The uniqueness condition in Theorem 2 looks dimensionally garbled as well. On the empirical side, the paper provides no hyperparameters, seeds, or code, so the figures are not reproducible. And the evaluation compares against the paper's own Stackelberg equilibrium computed from the same internal utility model, which makes the target circular.\n\nIn short: the engineering idea is fine and the components are familiar, but the current version is not sound and the core adaptivity claim is falsified by the paper's own formula. I would still send it to peer review, because the flaws are concrete and fixable, and a corrected version could be a useful contribution to the vehicular edge computing literature. As submitted, I would reject.\n\nBest","headline":"Eq. (38) makes the pruning rate identically equal to the static schedule, so the paper's central self-adaptive pruning claim is not implemented; the rest is a competent but internally evaluated MARL/Stackelberg paper.","tokens_in":22550,"tokens_out":4279,"would_cite":false,"duration_ms":43549,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TinyMA-IEI-PPO shows that a multi-agent PPO algorithm whose structured pruning threshold is driven by individual exploration incentives can keep VEAAT migration decisions close to the Stackelberg equilibrium of the AV-RSU bandwidth game…","keywords":["embodied AI","VEAAT migration","Stackelberg game","multi-agent deep reinforcement learning","self-adaptive structured pruning","exploration incentive","vehicular embodied AI networks","resource allocation"],"falsifier":"Compute the full Hessian of $U_i^F$ with respect to the bandwidth vector $b_i$, including the off-diagonal terms $\\partial^2 U_i^F/\\partial b_{ij}\\partial b_{is}$ for $j\\neq s$, under the parameter ranges used in Section VI; if any principal minor is positive or the Hessian is indefinite at a candidate best response, quasi-concavity fails and the uniqueness argument collapses. A direct numerical check would be to grid-search the best-response correspondence for a two-RSU, two-AV instance with strong social influence $\\zeta_{ik}$ and substitutability $\\eta_{js}<0$ and look for two distinct local maxima of $U_i^F$.","tokens_in":21537,"feed_emoji":"🚗","tokens_out":6846,"duration_ms":68439,"temperature":0.7,"pith_summary":"This paper tries to establish that migrating Vehicular Embodied Agent AI Twins (VEAATs) between roadside units can be solved by a tiny multi-agent PPO algorithm that prunes its own neural network during training, and that the resulting bandwidth and pricing decisions stay close to the equilibrium of a Stackelberg game between vehicles and roadside units. A sympathetic reader would care because autonomous vehicles have limited onboard compute: if pruning removes redundant neurons without sacrificing decision quality, migration choices can be made in real time on the vehicle rather than offloaded to the cloud. The paper pairs the algorithm with a utility model that folds in social influence among vehicles, complementarity and substitutability of virtual services, and an immersion-based matching probability, then argues this game has a unique Stackelberg equilibrium that the learned policies approximate.","feed_headline":"Pruned tiny RL matches equilibrium for vehicle-twin migration","feed_subtitle":"A self-pruning multi-agent PPO keeps vehicle AI twin migrations near the optimal game outcome, even at 85% sparsity.","key_machinery":"The load-bearing object is the exploration-incentive signal: the Bayesian surprise rate $r_{k,\\mathrm{int}}^t = I(z_{t+1}; a_k^t \\mid s_t, a_{-k}^t) = D_{\\mathrm{KL}}(p(z_{t+1}|s_t,a^t)\\|p(z_{t+1}|s_t,a_{-k}^t))$, estimated by a conditional variational autoencoder. This signal enters the PPO objective as an intrinsic reward and, after conversion to a bounded JS divergence $r'_{k,\\mathrm{int}}$, adaptively raises or lowers the pruning rate $p_t$ that sets the neuron-importance threshold $\\psi = \\sum_n \\sum_l S_n^{(l)} p_t$. The importance metric $S_n^{(l)}$ is a time-windowed, decayed sum of products of incoming and outgoing weight norms, and pruning is executed by a binary mask updated against $\\psi$, with a group-sparse regularizer tying mask updates to actor updates. The rest of the machinery, including the MLMF Stackelberg game, the $\\theta_{ij}$ matching probability built from A-DISTS image quality, and the migration delay model, feeds the rewards that the pruned policies optimize.","core_discovery":"The central claim is that individual exploration incentives can serve double duty: they improve exploration during training and, once converted to a bounded Jensen-Shannon divergence signal, they modulate the pruning threshold so that neurons are removed only when exploration pressure is low. The resulting TinyMA-IEI-PPO converges at speeds comparable to baseline algorithms, runs at pruning rates as high as 85 percent, and produces bandwidth-demand and pricing strategies whose average rewards sit closest to the theoretical Stackelberg equilibrium among all compared methods. The paper further claims that the underlying multi-leader multi-follower Stackelberg game, with vehicle utilities containing social-influence and service-interconnection terms, has a unique equilibrium established through standard-function best responses.","pith_inferences":["The same recipe, using an intrinsic novelty or impact signal to modulate a structured-pruning threshold, should transfer to any edge-deployed multi-agent reinforcement learning setting, not just Stackelberg migration games, because the mechanism only depends on the reward signal being bounded and reflecting exploration need.","Because the uniqueness proof never checks cross-partial derivatives of the AV utility, the claim of closely approximating the Stackelberg equilibrium should be read as conditional on that equilibrium being unique; if quasi-concavity fails in some parameter regime, the algorithm may still converge to a reasonable policy but the equilibrium benchmark loses its meaning.","A cheaper alternative to the CVAE estimator would be any bounded surprise proxy, such as prediction error in a learned forward model; if such a proxy also keeps pruning stable, the CVAE could be dropped, making the pipeline lighter still."],"forward_implications":["Deploying TinyMA-IEI-PPO on a computation-constrained platform keeps training and inference feasible on autonomous vehicles, since the pruned actor network is physically smaller and the exploration module is removed after training.","At an 85 percent pruning rate the algorithm still converges, and in the reported runs it outperforms the unpruned MA-IEI-PPO baseline, so aggressive sparsity does not automatically sacrifice solution quality.","Across varying numbers of AVs and RSUs, the learned average rewards for both followers and leaders stay closest to the theoretical Stackelberg equilibrium compared with MAPPO, MAA2C, MAA3C, Greedy, and Random, supporting the claim that the game model and learner are consistent.","The hybrid reward with an annealing intrinsic weight preserves exploration early and exploitation late, which is why the algorithm converges faster than baselines despite larger early fluctuations."],"supporting_citations":[{"why":"Introduces VEAATs and the twin-migration problem in vehicular embodied AI networks that this paper solves.","marker":"[5]"},{"why":"Supplies the multi-leader multi-follower Stackelberg game structure and socially-aware utility modeling used for the AV-RSU interaction.","marker":"[12]"},{"why":"Supplies the Stackelberg offloading framework and the M/M/1 queuing delay model used in the migration cost.","marker":"[13]"},{"why":"Provides the transmission-rate model and the Stackelberg equilibrium benchmark that the numerical results are compared against.","marker":"[21]"},{"why":"Establishes the multileader multifollower game for vehicular twins migration and the queuing and social-awareness resource allocation this paper extends.","marker":"[23]"},{"why":"Supplies the dynamic structured pruning with neuron-importance group sparse regularization and the immersion-based matching metric adapted here.","marker":"[22]"},{"why":"Supplies the neuron-importance metric based on weight norms and the dynamic structured pruning method for deep reinforcement learning that equations (35)-(36) build on.","marker":"[31]"},{"why":"Supplies the individual-exploration-incentive Bayesian surprise formulation that becomes the intrinsic reward and drives the adaptive pruning threshold.","marker":"[26]"},{"why":"Supplies the progressive pruning schedule $p_{t1}$ and $p_{t2}$ that the exploration incentive modulates.","marker":"[30]"}],"fun_headline_variants":["Exploration incentives tune pruning for fast AI twin migration","Self-adaptive pruning keeps multi-agent RL near game equilibrium","85% pruned model still hits Stackelberg-optimal AI twin moves","Exploration-driven pruning: tiny RL for vehicle twin migration","Pruned tiny PPO matches full-model equilibrium in twin migration"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument that the game has a unique equilibrium rests on the vehicle's utility being single-peaked (quasi-concave) across all of its bandwidth purchases at once, but the proof checks only the curvature of each purchase separately and ignores the cross-terms between purchases; if those cross-terms create multiple peaks, the unique equilibrium that the algorithm is said to approximate may not exist.","fun_headline_variants_meta":{"raw":{"variants":["Exploration incentives tune pruning for fast AI twin migration","Self-adaptive pruning keeps multi-agent RL near game equilibrium","85% pruned model still hits Stackelberg-optimal AI twin moves","Exploration-driven pruning: tiny RL for vehicle twin migration","Pruned tiny PPO matches full-model equilibrium in twin migration"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1538,"prompt_tokens":950,"completion_tokens":588,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":503}},"tokens_in":566,"tokens_out":588,"duration_ms":6454,"temperature":1.0,"reasoning_tokens":503,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:01:53.806535+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the full Hessian of $U_i^F$ with respect to the bandwidth vector $b_i$, including the off-diagonal terms $\\partial^2 U_i^F/\\partial b_{ij}\\partial b_{is}$ for $j\\neq s$, under the parameter ranges used in Section VI; if any principal minor is positive or the Hessian is indefinite at a candidate best response, quasi-concavity fails and the uniqueness argument collapses. A direct numerical check would be to grid-search the best-response correspondence for a two-RSU, two-AV instance with strong social influence $\\zeta_{ik}$ and substitutability $\\eta_{js}<0$ and look for two distinct local maxima of $U_i^F$.","supporting_citations":[{"cited_title":"Generative diffusion-based contract design for efficient ai twin migration in vehicular embodied ai networks,","cited_arxiv_id":null,"evidence_quote":"Introduces VEAATs and the twin-migration problem in vehicular embodied AI networks that this paper solves."},{"cited_title":"A multi- leader multi-follower game-based analysis for incentive mechanisms in socially-aware mobile crowdsensing,","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-leader multi-follower Stackelberg game structure and socially-aware utility modeling used for the AV-RSU interaction."},{"cited_title":"Stackelberg game-based computation offloading in social and cognitive industrial internet of things,","cited_arxiv_id":null,"evidence_quote":"Supplies the Stackelberg offloading framework and the M/M/1 queuing delay model used in the migration cost."},{"cited_title":"Learning-based incentive mechanism for task freshness-aware vehic- ular twin migration,","cited_arxiv_id":null,"evidence_quote":"Provides the transmission-rate model and the Stackelberg equilibrium benchmark that the numerical results are compared against."},{"cited_title":"When metaverses meet vehicle road cooperation: Multiagent drl-based stackelberg game for vehicular twins migration,","cited_arxiv_id":null,"evidence_quote":"Establishes the multileader multifollower game for vehicular twins migration and the queuing and social-awareness resource allocation this paper extends."},{"cited_title":"Tiny multiagent drl for twins migration in uav metaverses: A multileader multifollower stackelberg game approach,","cited_arxiv_id":null,"evidence_quote":"Supplies the dynamic structured pruning with neuron-importance group sparse regularization and the immersion-based matching metric adapted here."},{"cited_title":"Compressing deep reinforcement learning networks with a dynamic structured prun- ing method for autonomous driving,","cited_arxiv_id":null,"evidence_quote":"Supplies the neuron-importance metric based on weight norms and the dynamic structured pruning method for deep reinforcement learning that equations (35)-(36) build on."},{"cited_title":"Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning","cited_arxiv_id":"2405.18110","evidence_quote":"Supplies the individual-exploration-incentive Bayesian surprise formulation that becomes the intrinsic reward and drives the adaptive pruning threshold."}],"review_version":1}