Pith. sign in

REVIEW 3 major objections 5 minor 57 references

On Corruption-Robustness in Performative Reinforcement Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a repeated-retraining algorithm for performative reinforcement learning retains its convergence guarantee when an epsilon fraction of data is adversarially corrupted, ending near a performatively stable policy with…

desk verdict Solid robust-OFTRL analysis under corruption, but the performative RL convergence proof has two load-bearing gaps: the trimmed-mean bias in Theorem 3 and the +c fixed-point break in Algorithm 2. read the letter →

arxiv 2505.05609 v1 pith:N33USZ6J submitted 2025-05-08 cs.LG

Vasilis Pollatos , Debmalya Mandal , Goran Radanovic This is my paper · ORCID
classification cs.LG
keywords performativereinforcementlearningcorruptionrobustnessHubercontaminationrobustmeanestimationconvex-concaveoptimizationrepeatedretrainingOFTRLoccupancymeasures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether repeated retraining in performative reinforcement learning—where the reward and transition functions shift in response to the deployed policy—can still converge when an adversary corrupts a fraction epsilon of the training data. It claims yes: under Huber's epsilon-contamination model, a retraining scheme built from robust convex-concave optimization converges in its last iterate to a policy close to a performatively stable one, with approximation error growing only as the square root of epsilon. The difficulty is that naive gradient averaging in the saddle-point formulation fails badly under corruption, while a coordinate-wise trimmed-mean estimator combined with a robust version of Optimistic Follow-the-Regularized-Leader keeps the estimation error bounded. A sympathetic reader should care because recommendation systems, chatbots, and other policy-dependent systems are exposed to data poisoning, and this provides finite-sample convergence guarantees for performative RL in that regime.

What carries the argument

The load-bearing machinery is a robust version of Optimistic Follow-the-Regularized-Leader (OFTRL), an online-learning algorithm that updates by alternately minimizing regularized objectives with optimistic gradient predictions, paired with a coordinate-wise trimmed-mean estimator. For each coordinate of the gradient, the estimator keeps the (1-epsilon)m samples closest to the coordinate-wise median and averages them, cutting arbitrarily large adversarial corruption down to a bounded bias that scales linearly with epsilon. This estimator feeds gradients of the Lagrangian of the regularized reinforcement learning problem over occupancy measures, turning each retraining round into a convex-concave saddle-point optimization; the resulting gradient error bounds are what Lemma 1, Theorem 4, and Theorem 5 use to convert per-round accuracy into last-iterate convergence to a near-stable policy.

What would settle it

A targeted test: generate a clean set of gradient samples from the MDP Lagrangian, apply Algorithm 3's trimming step with an epsilon-fraction of large adversarial outliers, and compare the mean of the retained clean samples to the true gradient. If the bias is not O(epsilon) for fixed finite sample size, Theorem 3's proof fails at its load-bearing step; at the system level, a simulation exhibiting last-iterate cycling with final error above O(sqrt(epsilon)) under Assumptions 1-3 would falsify Theorem 5.

Watch

Extended reading notes

Core claim

On its own terms, the central discovery is Theorem 5: robust repeated retraining (Algorithm 2) has last-iterate convergence to a nearly performatively stable policy. Concretely, under Assumption 1 (bounded corruption in each batch), Assumption 2 (epsilon-sensitivity of rewards and transitions), and Assumption 3 (boundedness of an instance-specific constant), there exist a regularizer lambda and a number of rounds N such that with probability at least 1-delta the final occupancy measure lies within a ball of radius O(sqrt(epsilon)) around a performatively stable occupancy measure d_S. The sqrt(epsilon) dependence enters because the gradient-estimation error in Theorem 3 is linear in epsilon, and the distance-to-stability bound in Theorem 4 takes a square root of that error. The same analysis shows that the robust OFTRL optimizer alone attains O(1/T) duality-gap convergence and information-theoretically optimal terminal error under bounded gradient noise, matching the lower bound of Theorem 2.

Load-bearing premise

The load-bearing premise is that after Algorithm 3 trims each coordinate by keeping samples closest to the coordinate-wise median, the remaining clean gradient samples are still i.i.d. draws from the original clean distribution, so the standard concentration bound applies; trimming actually makes the retained clean samples correlated and shifts their conditional mean, and the proof does not bound that shift.

Editorial extensions

If this is right

  • If Theorem 5 is right, repeated retraining can be used in performative RL settings where an epsilon-fraction of feedback is poisoned, and the price is only an O(sqrt(epsilon)) radius around the stable policy instead of losing convergence entirely.
  • The robust OFTRL component gives O(1/T) convergence of the duality gap to an approximate saddle point under bounded gradient corruption, so it is usable outside performative RL in any smooth convex-concave optimization with corrupted first-order oracle feedback.
  • The coordinate-wise trimmed-mean estimator converts unbounded adversarial corruption of rewards and next states into bounded gradient noise, assuming the coverage condition on occupancy measures is maintained.
  • With sufficiently large regularization and number of rounds, the return of the deployed approximately stable policy in its own environment is worse than that of a performatively stable policy by only an instance-dependent constant, as shown in the appendix.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next step is to replace coordinate-wise trimming with a joint or covariance-aware robust estimator; that could reduce the dimension factors in the gradient error bound and possibly tighten the final O(sqrt(epsilon)) radius without changing the contraction argument.
  • The proof's reliance on retained clean samples staying i.i.d. after trimming suggests a testable variant: design a filtering rule that provably preserves the clean conditional distribution, or explicitly bound the trimming bias, and the gradient error analysis would rest on a fully stated premise.
  • The same robust-OFTRL machinery could plausibly be imported to other decision-dependent optimization problems with corrupted gradients, such as performative prediction with poisoned labels, since only the gradient estimator and the contraction step are specific to performative RL.
  • The paper leaves open whether O(sqrt(epsilon)) is the best possible dependence; constructing a matching lower bound for the stable-policy approximation error would settle the tightness of the guarantee.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies performative reinforcement learning in the finite-sample regime under Huber's epsilon-contamination model. It proposes a repeated-retraining algorithm (Algorithm 2) that combines a robust variant of Optimistic Follow the Regularized Leader (OFTRL) with a coordinate-wise trimmed mean estimator for the gradients of a regularized MDP Lagrangian. The main theoretical claim is last-iterate convergence to an approximately performatively stable policy, with approximation error linear in sqrt(epsilon). The paper also provides an analysis of robust OFTRL for convex-concave optimization, an information-theoretic lower bound on the terminal error, and a gridworld simulation study comparing the proposed robust estimator with naive gradient averaging.

Significance. If the central convergence claim were established, the paper would be a meaningful step toward making performative RL robust to data poisoning: the setting is well motivated, the idea of combining repeated retraining with robust mean estimation is natural, and the explicit lower bound for convex-concave optimization under bounded gradient noise is of independent interest. The paper also makes a useful practical contribution by demonstrating, through experiments, that naive gradient averaging is significantly more sensitive to corruption than the proposed trimming-based estimator. However, the main theoretical result is not established as written: the deployed occupancy measure is not the object to which the contraction argument is applied, and the proof of the robust gradient estimator applies an i.i.d. concentration lemma to a dataset that is no longer i.i.d. after trimming. These are load-bearing gaps in the central claim, so the paper needs substantial revision before it can be accepted.

major comments (3)
  1. [Algorithm 2, line 5; Appendix, Proof of Theorem 5] The contraction argument in the proof of Theorem 5 is applied to the wrong state. Algorithm 2 sets \tilde d_{n+1} = \bar d_{n+1} + c and deploys \pi_{n+1} = \pi^{\downarrow}\tilde d_{n+1}. As the Preliminaries explicitly note, d^{\pi^{\downarrow} d} and d may differ, and the \bar d produced by robust OFTRL is not guaranteed to satisfy the Bellman flow constraints. The proof nevertheless defines POC(\bar d_n) as the quantity that includes the addition of c, then states the update rule as \bar d_{n+1} = POC(\bar d_n), and applies the fixed-point property PO(d_S) = d_S and the contraction of PO. For c > 0, PO(d_S) is the optimum in M(\pi^{\downarrow}(d_S + c)), not in M(\pi_S), so d_S is not a fixed point of the implemented operator. No bound is supplied for \|d^{\pi^{\downarrow}(\bar d_n + c)} - (\bar d_n + c)\|, and the c\sqrt{SA} term in \bar C accounts only for the Euclidean distance between \bar d and \tilde d, not for the Bellman-flow violation or the resulting distribution shift. Thus Theorem 5 is not established as written.
  2. [Theorem 3; Appendix, Proof of Theorem 3] The proof applies Lemma 6 to the dataset after coordinate-wise trimming in Algorithm 3, but Lemma 6 requires the good samples to remain i.i.d. draws from the original distribution with expectation \bar x. Keeping the samples closest to the median makes the retained clean samples dependent and changes their conditional mean; this bias is not bounded anywhere in the proof. Consequently the stated high-probability bound for \hat g_d is not justified. Because Lemma 1, Theorem 4, and Theorem 5 all rely on the gradient error bound E_2 from Theorem 3, the central convergence claim inherits this gap. A correct proof would need to control the bias of the coordinate-wise trimmed mean or replace the estimator with one whose analysis does not require the retained samples to be i.i.d.
  3. [Appendix, Proof of Theorem 5] The step "after some iterations (suppose this is less than N, without loss of generality, since we did not assume anything for N)" does not prove the existence of a finite N. The theorem asserts that there exist \lambda and N such that the stated guarantee holds, and the proof should provide a concrete threshold for N in terms of the contraction constant, the initial distance, and \bar C(\delta/N). Citing the proof of Theorem 1 from prior work is not sufficient here, because the operator and the convergence criterion differ.
minor comments (5)
  1. [Assumption 1] Assumption 1 states that every D_n can be split into 2T batches with corruption level at most "epsilon < 0."; the intended bound is epsilon < 0.5, which is used throughout the proofs.
  2. [Figure 1 caption] The caption says that panels (a) and (d) use naive gradient averaging and (b) and (c) use robust gradient estimation, but the panel labels show (a) and (c) as naive and (b) and (d) as robust; the caption should be corrected to match the labels.
  3. [Theorem 3 proof] The proof states that the median at each coordinate is equal to one uncorrupted sample for epsilon < 0.5; this ignores the even-cardinality tie case and the exact rounding convention in Algorithm 3, though this is a patchable issue if the rest of the estimator analysis is repaired.
  4. [Throughout] There are several typographical errors, including "perfomrative," "convergce," "Robust OFTR," and inconsistent spellings of "convex-concave"; these should be cleaned up in revision.
  5. [Theorem 2 statement] The lower bound statement would be clearer if it specified the norm used for the distance to the saddle point and if the condition on Z_X, Z_Y relative to D were stated before the theorem rather than inside the proof.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the corruption-robustness result is derived from explicit gradient-error bounds and an independently published contraction theorem; no prediction reduces to a fitted parameter or to a definition.

full rationale

The paper's central claims do not reduce to their inputs by construction. The robust OFTRL guarantee in Theorem 1 is a standard optimistic-FTRL regret bound (citing Orabona 2019) with gradient noise terms added, not a self-defined result. Theorem 3 bounds the robust gradient estimators using concentration inequalities and the boundedness of clean samples under the contamination model; the error scales with epsilon and problem constants, and no fitted constant is renamed as a prediction. Theorem 4 converts the gradient-error bound into a distance to the optimal occupancy measure via strong concavity and Hoffman's bound, which is a self-contained argument. Theorem 5 invokes the contraction of the performative policy-optimization operator from Mandal, Triantafyllou, and Radanovic 2023; although this is a self-citation, it is used as an external prior theorem whose assumptions do not include the corruption-robustness conclusion, so it does not make the result circular. The information-theoretic lower bound in Theorem 2 is an explicit adversarial construction, not a reformulation of the algorithm's output. The proof contains genuine correctness gaps that are not circularity: after coordinate-wise trimming the retained clean samples are not i.i.d. from the original distribution, and the positive constant c added in Algorithm 2 can make the returned occupancy measure violate the Bellman flow constraint, so the deployed occupancy may differ from the quantity used in the proof. However, these are unsupported steps in the derivation rather than cases where an output is equivalent to an input by definition or where a fitted parameter is later called a prediction. Accordingly, the appropriate finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 9 assumptions · 0 invented entities

The theoretical claims rest on a standard set of performative RL and robust statistics assumptions. No new physical or mathematical entities are introduced. The experimental hyperparameters lambda and cp are manually tuned, and the mixing constant c is unspecified in the experiments, but none of these are fitted to force the central convergence result.

free parameters (3)
  • mixing constant c = not specified in experiments
    Added to \bar{d} to ensure coverage; the value affects the constant B(c) in the bounds but is not fitted to data. In experiments the paper does not report the exact value used.
  • regularization lambda = 0.001 in experiments
    Chosen by hand (cp=1, lambda=0.001) to make repeated retraining converge in about 25 iterations; the theory only requires lambda large enough.
  • performative penalty cp = 1
    Reward parameter in the gridworld, tuned for convergence speed.
assumptions (9)
  • domain assumption Huber epsilon-contamination model with (si, ai) uncorrupted
    The adversary can corrupt next states and rewards but not the state-action pair itself; the paper argues in the appendix this can be relaxed.
  • domain assumption Bounded rewards and bounded gradient norms
    Rewards are bounded by R, and the Lagrangian gradients are bounded, enabling the OFTRL analysis.
  • domain assumption Coverage assumption: dn(s,a) > 0 and there exists B(c) > 0 for deployed policies
    Needed for the gradient estimators to have bounded importance weights; stated in the Data Generation Process section.
  • domain assumption Assumption 1 (per-batch corruption level at most epsilon < 0.5)
    Needed for the robust estimator to have enough clean samples; stated in the Convergence Analysis section.
  • domain assumption Assumption 2 (epsilon-sensitivity of rewards and transitions)
    Standard performative prediction assumption taken from prior work; used to establish contraction of the repeated retraining operator.
  • domain assumption Assumption 3 (alpha-boundedness)
    Assumes alpha(M(pi),delta) is uniformly bounded across all deployed policies; this is a regularity condition on the MDP family.
  • standard math Hoffman's bound applies to the occupancy polytope with constant sigma_n
    Used in the proof of Theorem 4 to bound the distance from the constrained optimum to the flow constraint set.
  • domain assumption Contraction property of the policy optimization operator PO(.)
    Taken from Mandal et al. (2023), used in the proof of Theorem 5; requires lambda large enough.
  • domain assumption Stationarity and Lipschitz smoothness of the Lagrangian L
    The paper states these are satisfied for the objective; needed for the OFTRL convergence rate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On Corruption-Robustness in Performative Reinforcement Learning." pith.science (2026). https://pith.science/paper/N33USZ6J

@misc{pith2026250505609,
  author       = {Pith},
  title        = {Pith review of: On Corruption-Robustness in Performative Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N33USZ6J}},
  note         = {Machine review of arXiv:2505.05609}
}
abstract

In performative Reinforcement Learning (RL), an agent faces a policy-dependent environment: the reward and transition functions depend on the agent's policy. Prior work on performative RL has studied the convergence of repeated retraining approaches to a performatively stable policy. In the finite sample regime, these approaches repeatedly solve for a saddle point of a convex-concave objective, which estimates the Lagrangian of a regularized version of the reinforcement learning problem. In this paper, we aim to extend such repeated retraining approaches, enabling them to operate under corrupted data. More specifically, we consider Huber's $\epsilon$-contamination model, where an $\epsilon$ fraction of data points is corrupted by arbitrary adversarial noise. We propose a repeated retraining approach based on convex-concave optimization under corrupted gradients and a novel problem-specific robust mean estimator for the gradients. We prove that our approach exhibits last-iterate convergence to an approximately stable policy, with the approximation error linear in $\sqrt{\epsilon}$. We experimentally demonstrate the importance of accounting for corruption in performative RL.

Figures

Figures reproduced from arXiv: 2505.05609 by the authors.

Figure 1
Figure 1. Convergence of repeated retraining approaches: (a) and (d) consider a non-robust variant of Algorithm 3 that utilizes [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 47 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    M.; and Sun, W

    Agarwal, A.; Jiang, N.; Kakade, S. M.; and Sun, W. 2019. Reinforcement learning: Theory and algorithms. CS Dept., UW Seattle, Seattle, WA, USA, Tech. Rep, 32

    work page 2019

  4. [4]

    Ahn, K.; Jain, P.; Ji, Z.; Kale, S.; Netrapalli, P.; and Shamir, G. I. 2022. Reproducibility in optimization: Theoretical framework and limits. Advances in Neural Information Processing Systems, 35: 18022--18033

    work page 2022

  5. [5]

    Beznosikov, A.; Sadiev, A.; and Gasnikov, A. 2020. Gradient-free methods with inexact oracle for convex-concave stochastic saddle-point problem. In International Conference on Mathematical Optimization Theory and Operations Research, 105--119. Springer

    work page 2020

  6. [6]

    Brown, G.; Hod, S.; and Kalemaj, I. 2022. Performative prediction in a stateful world. In International conference on artificial intelligence and statistics, 6045--6061. PMLR

    work page 2022

  7. [7]

    Chiang, C.-K.; Yang, T.; Lee, C.-J.; Mahdavi, M.; Lu, C.-J.; Jin, R.; and Zhu, S. 2012. Online optimization with gradual variations. In Conference on Learning Theory, 6--1. JMLR Workshop and Conference Proceedings

    work page 2012

  8. [8]

    d'Aspremont, A. 2008. Smooth optimization with approximate gradient. SIAM Journal on Optimization, 19(3): 1171--1183

    work page 2008

Show all 57 references
  1. [9]

    Devolder, O. 2013. Exactness, inexactness and stochasticity in first-order methods for large-scale convex optimization. Ph.D. thesis, CORE UCLouvain Louvain-la-Neuve, Belgium

    2013

  2. [10]

    Devolder, O.; Glineur, F.; and Nesterov, Y. 2014. First-order methods of smooth convex optimization with inexact oracle. Mathematical Programming, 146: 37--75

    2014

  3. [11]

    M.; and Pensia, A

    Diakonikolas, I.; Kane, D. M.; and Pensia, A. 2020. Outlier robust mean estimation with subgaussian rates via stability. Advances in Neural Information Processing Systems, 33: 1830--1840

    2020

  4. [12]

    Dvinskikh, D.; Tominin, V.; Tominin, Y.; and Gasnikov, A. 2022. Gradient-free optimization for non-smooth minimax problems with maximum value of adversarial noise. arXiv preprint arXiv, 2202

    2022

  5. [13]

    Garber, D. 2019. Logarithmic regret for online gradient descent beyond strong convexity. In The 22nd International Conference on Artificial Intelligence and Statistics, 295--303. PMLR

    2019

  6. [14]

    Hardt, M.; and Mendler-D \"u nner, C. 2023. Performative prediction: Past and future. arXiv preprint arXiv:2310.16608

    2023 arXiv

  7. [15]

    Hoffman, A. J. 2003. On approximate solutions of systems of linear inequalities. In Selected Papers Of Alan J Hoffman: With Commentary, 174--176. World Scientific

    2003

  8. [16]

    Huang, K.; and Zhang, S. 2022. New first-order algorithms for stochastic variational inequalities. SIAM Journal on Optimization, 32(4): 2745--2772

    2022

  9. [17]

    Izzo, Z.; Ying, L.; and Zou, J. 2021. How to learn when data reacts to your model: performative gradient descent. In International Conference on Machine Learning, 4641--4650. PMLR

    2021

  10. [18]

    Juditsky, A.; Nemirovski, A.; and Tauvel, C. 2011. Solving variational inequalities with stochastic mirror-prox algorithm. Stochastic Systems, 1(1): 17--58

    2011

  11. [19]

    Korpelevich, G. M. 1976. The extragradient method for finding saddle points and other problems. Matecon, 12: 747--756

    1976

  12. [20]

    Letchford, J.; MacDermed, L.; Conitzer, V.; Parr, R.; and Isbell, C. 2012. Computing optimal strategies to commit to in stochastic games. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 26, 1380--1386

    2012

  13. [21]

    Li, Q.; and Wai, H.-T. 2022. State dependent performative prediction with stochastic approximation. In International Conference on Artificial Intelligence and Statistics, 3164--3186. PMLR

    2022

  14. [22]

    Lu, S. 2023. Bilevel optimization with coupled decision-dependent distributions. In International Conference on Machine Learning, 22758--22789. PMLR

    2023

  15. [23]

    Mandal, D.; Nika, A.; Kamalaruban, P.; Singla, A.; and Radanovi \'c , G. 2024. Corruption Robust Offline Reinforcement Learning with Human Feedback. arXiv preprint arXiv:2402.06734

    2024 arXiv

  16. [24]

    Mandal, D.; Triantafyllou, S.; and Radanovic, G. 2023. Performative reinforcement learning. In International Conference on Machine Learning, 23642--23680. PMLR

    2023

  17. [25]

    Mendler-D \"u nner, C.; Perdomo, J.; Zrnic, T.; and Hardt, M. 2020. Stochastic optimization for performative prediction. Advances in Neural Information Processing Systems, 33: 4929--4939

    2020

  18. [26]

    P.; Perdomo, J

    Miller, J. P.; Perdomo, J. C.; and Zrnic, T. 2021. Outside the echo chamber: Optimizing the performative risk. In International Conference on Machine Learning, 7710--7720. PMLR

    2021

  19. [27]

    Mohri, M.; and Yang, S. 2015. Accelerating optimization via adaptive prediction. arXiv preprint arXiv:1509.05760

    2015 arXiv

  20. [28]

    Mokhtari, A.; Ozdaglar, A.; and Pattathil, S. 2019. Proximal point approximations achieving a convergence rate of o (1/k) for smooth convex-concave saddle point problems: Optimistic gradient and extra-gradient methods. arXiv preprint arXiv:1906.01115, 3

    2019 arXiv

  21. [29]

    Mokhtari, A.; Ozdaglar, A.; and Pattathil, S. 2020. A unified analysis of extra-gradient and optimistic gradient methods for saddle point problems: Proximal point approach. In International Conference on Artificial Intelligence and Statistics, 1497--1507. PMLR

    2020

  22. [30]

    Narang, A.; Faulkner, E.; Drusvyatskiy, D.; Fazel, M.; and Ratliff, L. J. 2023. Multiplayer performative prediction: Learning in decision-dependent games. Journal of Machine Learning Research, 24(202): 1--56

    2023

  23. [31]

    Nedi \'c , A.; and Ozdaglar, A. 2009. Subgradient methods for saddle-point problems. Journal of optimization theory and applications, 142: 205--228

    2009

  24. [32]

    Neff, G. 2016. Talking to bots: Symbiotic agency and the case of Tay. International Journal of Communication

    2016

  25. [33]

    Nemirovski, A. 2004. Prox-method with rate of convergence O (1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM Journal on Optimization, 15(1): 229--251

    2004

  26. [34]

    Nesterov, Y. 2007. Dual extrapolation and its applications to solving variational inequalities and related problems. Mathematical Programming, 109(2-3): 319--344

    2007

  27. [35]

    Nika, A.; Mandal, D.; Singla, A.; and Radanovic, G. 2024. Corruption-Robust Offline Two-Player Zero-Sum Markov Games. In International Conference on Artificial Intelligence and Statistics, 1243--1251. PMLR

    2024

  28. [36]

    Orabona, F. 2019. A modern introduction to online learning. arXiv preprint arXiv:1912.13213

    2019 arXiv

  29. [37]

    Ouyang, Y.; and Xu, Y. 2021. Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems. Mathematical Programming, 185(1-2): 1--35

    2021

  30. [38]

    Perdomo, J.; Zrnic, T.; Mendler-D \"u nner, C.; and Hardt, M. 2020. Performative prediction. In International Conference on Machine Learning, 7599--7609. PMLR

    2020

  31. [39]

    Piliouras, G.; and Yu, F.-Y. 2023. Multi-agent performative prediction: From global stability and optimality to chaos. In Proceedings of the 24th ACM Conference on Economics and Computation, 1047--1074

    2023

  32. [40]

    Polyak, B. T. 1987. Introduction to optimization. optimization software. Inc., Publications Division, New York, 1: 32

    1987

  33. [41]

    S.; Balakrishnan, S.; and Ravikumar, P

    Prasad, A.; Suggala, A. S.; Balakrishnan, S.; and Ravikumar, P. 2020. Robust estimation via robust gradient estimation. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 82(3): 601--627

    2020

  34. [42]

    Rank, B.; Triantafyllou, S.; Mandal, D.; and Radanovic, G. 2024. Performative Reinforcement Learning in Gradually Shifting Environments. In The 40th Conference on Uncertainty in Artificial Intelligence

    2024

  35. [43]

    Robinson, J. 1951. An iterative method of solving a game. Annals of mathematics, 296--301

    1951

  36. [44]

    Triantafyllou, S.; Singla, A.; and Radanovic, G. 2021. On blame attribution for accountable multi-agent sequential decision making. Advances in Neural Information Processing Systems, 34: 15774--15786

    2021

  37. [45]

    Tseng, P. 1995. On linear convergence of iterative methods for the variational inequality problem. Journal of Computational and Applied Mathematics, 60(1-2): 237--252

    1995

  38. [46]

    Tseng, P. 2008. On accelerated proximal gradient methods for convex-concave optimization. submitted to SIAM Journal on Optimization, 2(3)

    2008

  39. [47]

    Wang, Y.; Mianjy, P.; and Arora, R. 2021. Robust Learning for Data Poisoning Attacks. In Meila, M.; and Zhang, T., eds., Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, 10859--10869. PMLR

    2021

  40. [48]

    Wei, C.-Y.; Lee, C.-W.; Zhang, M.; and Luo, H. 2021. Linear Last-iterate Convergence in Constrained Saddle-point Optimization. In International Conference on Learning Representations

    2021

  41. [49]

    Wu, F.; Li, L.; Xu, C.; Zhang, H.; Kailkhura, B.; Kenthapadi, K.; Zhao, D.; and Li, B. 2022. Copa: Certifying robust policies for offline reinforcement learning against poisoning attacks. arXiv preprint arXiv:2203.08398

    2022 arXiv

  42. [50]

    Yan, W.; and Cao, X. 2024. Zero-regret performative prediction under inequality constraints. Advances in Neural Information Processing Systems, 36

    2024

  43. [51]

    Yang, T.; Mahdavi, M.; Jin, R.; and Zhu, S. 2011. Regret Bound by Variation for Online Convex Optimization. arXiv preprint arXiv:1111.6337

    2011 arXiv

  44. [52]

    Ye, C.; Yang, R.; Gu, Q.; and Zhang, T. 2024. Corruption-robust offline reinforcement learning with general function approximation. Advances in Neural Information Processing Systems, 36

    2024

  45. [53]

    Zhan, W.; Huang, B.; Huang, A.; Jiang, N.; and Lee, J. 2022. Offline reinforcement learning with realizability and single-policy concentrability. In Conference on Learning Theory, 2730--2775. PMLR

    2022

  46. [54]

    Zhang, L.; Yang, J.; Karbasi, A.; and He, N. 2024. Optimal guarantees for algorithmic reproducibility and gradient complexity in convex optimization. Advances in Neural Information Processing Systems, 36

    2024

  47. [55]

    Zhang, X.; Chen, Y.; Zhu, X.; and Sun, W. 2022. Corruption-robust offline reinforcement learning. In International Conference on Artificial Intelligence and Statistics, 5757--5773. PMLR

    2022

  48. [56]

    Zhao, P.; Zhang, Y.-J.; Zhang, L.; and Zhou, Z.-H. 2020. Dynamic Regret of Convex and Smooth Functions. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20. Red Hook, NY, USA: Curran Associates Inc. ISBN 9781713829546

    2020

  49. [57]

    Zhong, H.; Yang, Z.; Wang, Z.; and Jordan, M. I. 2021. Can Reinforcement Learning Find Stackelberg-Nash Equilibria in General-Sum Markov Games with Myopic Followers? arXiv preprint arXiv:2112.13521

    2021 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.