Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Sensitivity of Filter Kernels and Robustness Bounds to Transition and Measurement Kernel Perturbations in Partially Observable Stochastic Control

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves explicit, uniform bounds on how much the belief-update kernels of a POMDP change when transition and observation kernels are perturbed, and turns them into suboptimality guarantees for policies designed on quantized or app

desk verdict Explicit filter-kernel sensitivity bounds are a real contribution, but the finite-model application as written compares kernels on different spaces and needs a revision before the headlin read the letter →

arxiv 2508.10658 v3 pith:UZBVPTP5 submitted 2025-08-14 math.OC

Yunus Emre Demirci , Ali Devran Kara , Serdar Yüksel This is my paper
classification math.OC MSC 93E2090C4060J05
keywords POMDPfilterkernelbelief-MDPWassersteindistancetotalvariationrobustnessboundsquantizationpartiallyobservablestochasticcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes explicit, non-asymptotic bounds on how much the belief-update dynamics of a partially observable Markov decision process change when both the state-transition kernel and the observation kernel are perturbed, and converts those bounds into performance guarantees: a policy that is optimal for an approximate model is provably near-optimal for the true model, with the gap bounded by Wasserstein-1 or total-variation distances between the kernels. The bounds cover both discounted and average-cost criteria. As a concrete application, the authors show how to quantize state and measurement spaces into a finite POMDP and give error bounds that decay to zero as the quantization becomes finer. A sympathetic reader would care because this turns qualitative continuity results into computable certificates for model reduction, learning, and numerical control design.

What carries the argument

The central object is the nonlinear-filter kernel of the belief-MDP reduction, $\eta(\cdot|\pi,u)=\int_Y \mathbf{1}_{\{F(\pi,u,y)\in\cdot\}} H(dy|\pi,u)$, where $F$ is the one-step posterior update and $H$ the predictive measurement distribution. The argument's workhorse is a two-step triangle inequality through the intermediate model $(T,Q^n)$ that changes only one kernel at a time, together with the Lipschitz modulus $L_Q$ of the observation kernels, which converts total-variation errors in measurements into Wasserstein-1 errors in transitions. The value-function bounds then rest on the filter contraction constant $K_2$ (e.g. $K_2=\alpha D(3-2\delta(Q))/2$ under Assumption 2.3, with $\delt

What would settle it

Take a two-state Markov chain with binary observations where both true and perturbed kernels are fully specified, compute the filter transition kernels $\eta$ and $\eta^{T^n,Q^n}$ exactly as finite stochastic matrices, and check whether $d_{W_1}(\eta,\eta^{T^n,Q^n}) \le (D/2+2)d_{\mathrm{TV}}(Q^n,Q)$ when $T^n=T$; a single violation would refute Theorem 3.4. Alternatively, simulate the quantized linear-Gaussian POMDP of Example 6.5 and check whether the realized suboptimality of the approximate policy exceeds the Corollary 5.5 bound.

Watch

Extended reading notes

Core claim

The paper's central discovery is a quantitative coupling between model perturbations and filter-kernel perturbations. Writing $\eta$ for the belief-MDP transition kernel of the true model $(T,Q)$ and $\eta^{T^n,Q^n}$ for that of the approximate model, it shows under a Lipschitz observation-kernel condition that the uniform Wasserstein-1 distance obeys $d_{W_1}(\eta,\eta^{T^n,Q^n}) \le (D/2+2)(L_Q d_{W_1}(T^n,T)+d_{\mathrm{TV}}(Q^n,Q))$, with $D$ the diameter of the compact state space; the bounded-Lipschitz analogue is $2(L_Q d_{W_1}(T^n,T)+d_{\mathrm{TV}}(Q^n,Q))$. These bounds are proved by an intermediate-model triangle inequality comparing $(T,Q^n)$ with both endpoints, plus a measurable

Load-bearing premise

Every value-function and quantization bound in Sections 4 and 5 requires the nonlinear filter to be contracting, with a constant $K_2<1$ that this paper inherits from earlier contraction theorems rather than proving anew; if $K_2 \ge 1$, the displayed bounds are vacuous or infinite.

Editorial extensions

If this is right

  • A policy computed for a jointly quantized finite POMDP is provably within an explicit distance of optimal for the original continuous model, and that distance shrinks to zero as $L_X^n$ and $L_Y^n$ go to zero.
  • The same kernel-proximity bounds certify robustness under simultaneous transition- and observation-kernel perturbations, covering cases earlier work handled only separately or asymptotically.
  • The bounds are uniform over all initial beliefs, so they give worst-case performance guarantees rather than guarantees for a single prior.
  • Under the average-cost criterion, the analogous suboptimality bound holds with denominators $1-K_2$ instead of $(1-\beta)(1-\beta K_2)$, showing the same qualitative dependence on filter contraction.
  • Given any pair of models whose transition and observation kernels are close in the relevant metrics, the optimal discounted values differ by at most an explicitly computed constant times the kernel distance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The linear dependence of the error on $L_Q(\alpha+1)L_X^n + \alpha_Y L_Y^n$ suggests a practical design rule for automatic discretization: allocate quantization levels so the two terms are roughly equal, rather than over-refining one space.
  • Read as a filtering statement (the control-free hidden Markov case), the results imply that predictive and filtering distributions are Lipschitz stable in model parameters, which could support online detection of model drift without re-deriving filters.
  • The bounds inherit the conservatism of the worst-case diameter $D$ and worst-case Lipschitz constants; a likely refinement would replace them with belief-dependent or local Lipschitz constants for tighter per-instance guarantees.
  • Since the contraction constant $K_2$ is the bottleneck, a testable extension is to check numerically whether filter-contraction failures genuinely destroy the stated robustness, or whether looser but finite bounds could be obtained from average-case contraction conditions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies robustness of POMDPs under simultaneous perturbations of the transition kernel T and observation kernel Q. The main contribution is a set of explicit bounds (Theorems 3.1–3.4) on the distance between the belief-MDP (filter) kernels η and η^{T^n,Q^n} in the bounded-Lipschitz and Wasserstein-1 metrics, with constants depending on the diameter of X, the Lipschitz modulus of the observation channels, and the distances d_TV(T^n,T), d_W1(T^n,T), d_TV(Q^n,Q). These bounds are then combined with Lipschitz regularity of value functions, imported from prior work under contraction conditions K_2<1, to obtain explicit robustness bounds for the discounted and average-cost criteria (Section 4). Section 5 applies the results to quantized finite-state/finite-observation POMDPs, claiming explicit rates decaying to zero as the quantization fineness L_X^n, L_Y^n tends to zero (Theorem 5.1, Corollaries 5.5–5.6).

Significance. The paper addresses a genuinely important problem: obtaining non-asymptotic, quantitative robustness guarantees for POMDPs under model mismatch, going beyond the mostly asymptotic results in the prior literature. The filter-kernel bounds in Section 3, if correct, are useful and fairly general, and the finite-model application is a natural headline application. The paper also gives constructive examples illustrating the perturbation quantities. However, two load-bearing issues prevent the results from being accepted in their current form: (i) the proof of Theorem 3.1 relies on a compactness/measurable-selection argument that is not valid without a compactness assumption on X; (ii) the finite-model results in Section 5 compare kernels and channels acting on different spaces, so the main quantized bounds are not formally well-defined as stated. Because these issues affect the central claims and the headline Corollaries 5.5–5.6, a major revision is required.

major comments (4)
  1. [§3.1, proof of Theorem 3.1, around Eqs. (19)–(21)] Theorem 3.1 is stated for Polish X with no compactness assumption, but the proof uses Arzela–Ascoli to conclude that BL_1(X) is compact. BL_1(X) is not compact in the sup-norm when X is noncompact, so the measurable-selection step (existence of a measurable selector g_y ∈ arg sup over BL_1(X)) is not justified. The same argument is reused in Theorems 3.2–3.4, so the gap propagates. Please either add compactness of X to the statements, or replace the selection step with a valid argument (e.g., a measurable sup-integral interchange under weaker conditions).
  2. [§5.1–§5.3, Theorem 5.1 and Lemmas 5.2–5.4] The quantized kernels live on different spaces: T^n: X×U→P(X^n), Q^n: X→P(Y^n), and hence η^{T^n,Q^n} is a kernel on P(X^n). The expressions d_W1(η^{T,Q}, η^{T^n,Q^n}), d_TV(Q^n,Q) and d_W1(T^n,T) in Theorem 5.1, Lemma 5.3, Lemma 5.4, and Corollaries 5.5–5.6 are undefined unless one embeds X^n into X and Y^n into Y, or explicitly replaces Q^n by the informationally equivalent intermediate channel \tilde Q^n on Y. The constants and even the validity of the results depend on the chosen embedding. This is not a cosmetic issue: Corollaries 5.5–5.6 are the advertised finite-POMDP guarantees, and as written they are not well-posed.
  3. [Corollary 5.5 and Corollary 5.6] There is an assumption mismatch. Theorem 5.1 requires Assumption 2.4(i)–(ii), i.e., a Wasserstein-1 Lipschitz modulus θ (called α in Lemma 5.2) for T. Corollary 5.5, however, is stated only under Assumption 2.3 and Assumption 5.1. Under Assumption 2.3 the available modulus is a total-variation Lipschitz constant α, which implies only W1(T(·|x,u),T(·|x',u)) ≤ αD/2 · d(x,x'), not α d(x,x'). Thus the factor (α+1)L_X^n in Corollary 5.5 (and similarly in Corollary 5.6) is not justified from the stated hypotheses. The corollaries either need to add Assumption 2.4(i)–(ii) or use a corrected constant involving D·α.
  4. [§4.3, Theorem 4.6 and Corollaries 4.4–4.5, 5.2, 5.4, 5.6] The average-cost results are all conditional on the Abelian condition in Theorem 4.6(iv) and on the contraction conditions K_2<1 (and K^{T^n,Q^n}_2<1 for the approximate model). The paper does not verify any of these conditions for the quantized finite models, nor does it provide sufficient conditions in terms of the quantization parameters. Since the stated bounds blow up as 1−K_2 → 0 and the finite-model corollaries inherit this, the practical applicability of the average-cost guarantees is unclear. Please add a discussion, or better, explicit sufficient conditions, for when the finite approximate model satisfies these hypotheses.
minor comments (5)
  1. [Eq. (27)] The inequality at the end of the proof of Theorem 3.1 is labeled d_TV(η, η^{T,Q^n}); it should be d_BL(η, η^{T,Q^n}).
  2. [Eq. (37)] In the smoothing step for the second term of part (ii), the first integral should be with respect to T^n(dx_1|z_0), not T(dx_1|z_0), because the posterior z_2^1 is under (T^n,Q^n). The subsequent inequality remains valid by symmetry of total variation, but the displayed equality is wrong.
  3. [§5.2, around Eq. (43)] The notation d_TV(Q^n,Q) is used even though Q^n maps into P(Y^n) while Q maps into P(Y); this is part of the embedding issue flagged above. Please introduce \tilde Q^n explicitly and state all distances for kernels with a common codomain.
  4. [Theorem 4.6(iv)] The assumption that lim_{β→1} ||J*_β(η)||_Lip and lim_{β→1} ||J*_β(η^{T^n,Q^n})||_Lip exist is silent on whether these limits are finite. If they are infinite, the derived bound is vacuous; please state finiteness explicitly.
  5. [General notation] The symbol α is used both for the total-variation Lipschitz constant in Assumption 2.3 and for the Wasserstein-Lipschitz constant in Lemma 5.2 and Theorem 5.1. Please use distinct symbols (e.g., α_TV and α_W1) to avoid the mismatch identified above.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: Section 3 filter-kernel bounds are first-principles; Section 4 composes independent published results; finite-model formal gap is a correctness issue, not circularity.

full rationale

The filter-kernel proximity bounds (Theorems 3.1-3.4) are proven in Section 3 from Lemmas 3.1 and 3.2 by direct measure-theoretic estimates and triangle inequalities; they do not assume the value-function or quantization conclusions that they later feed. The contraction constants K2 imported from [8,10] and the value Lipschitz/robustness estimates imported from [8,10] and [51] are published theorems with stated assumptions (Assumptions 2.3/2.4) that do not include the target filter-distance or value-difference conclusions, so they function as legitimate external premises rather than self-justifying definitions. The finite-model application (Theorem 5.1, Corollaries 5.5-5.6) has a formal well-posedness gap: it compares filter kernels on P(X) with kernels on P(X^n) without specifying embeddings, and the (alpha+1)L_X^n constant is not justified under Assumption 2.3 alone. However, this is a correctness/rigor issue, not a circular reduction: no equation in the paper is equivalent by construction to its own input, and no fitted parameter is relabeled as a prediction. Hence no circularity is found.

Assumptions & free parameters 0 free parameters · 9 assumptions · 0 invented entities

The central bounds do not fit any constant to data. All constants (D, alpha, L_Q, K1, K2, alpha_Y) are structural assumptions on the true and approximate models. The main burden is that Section 4 and 5 results inherit the contraction condition K2<1 and the weak Feller property of the approximate filter from the authors' prior work [8,10,31], plus a technical Abelian condition for the average-cost case.

assumptions (9)
  • domain assumption X is a compact metric space with diameter D (Assumption 2.3-1).
    Needed for the W_1 diameter argument, for the Arzela-Ascoli compactness of BL_1(X) in the proof of Theorem 3.1, and for quantization rates in Section 5.
  • domain assumption The transition kernel T is continuous in total variation in (x,u) and satisfies ||T(.|x,u)-T(.|x',u)||_TV <= alpha d(x,x') (Assumptions 2.3-2,3).
    Enters the filter contraction constant K2 = alpha D (3 - 2 delta(Q)) / 2 used in all Section 4 bounds.
  • domain assumption The observation channels Q and Q^n are Lipschitz continuous in total variation in x with a common constant L_Q (Assumption 3.1).
    Allows replacing d_TV(T^n,T) by L_Q d_W1(T^n,T) in Theorems 3.2 and 3.4; used in Lemma 5.2-5.4 and Theorem 5.1.
  • domain assumption The cost function c is bounded and Lipschitz in x with constant K1 (Assumptions 2.3-4,5).
    Gives the Lipschitz constant of the value function, K1/(1-beta K2).
  • domain assumption Contraction condition K2 < 1, with K2 = alpha D (3-2 delta(Q))/2 or K2 = theta + 3 theta gamma D / 2 (Assumptions 2.3/2.4).
    Necessary for Lipschitz continuity of the value function and for average cost ACOE existence; all robust bounds carry denominators 1-K2 or 1-beta K2.
  • domain assumption The approximate filter kernel eta^{T^n,Q^n} is weak Feller (used in Corollaries 4.1-4.3, 5.5).
    Imported from [8, Lemma 4.2] applied to the approximate system; not established in this paper.
  • domain assumption Assumption 5.1: Y is compact, Q(dy|x)=g(x,y) lambda(dy), and g is Lipschitz in y with constant alpha_Y.
    Gives d_TV(Q^n,Q) <= alpha_Y L_Y^n for measurement quantization.
  • ad hoc to paper Average-cost Abelian condition: for each pi there exists beta_n(pi)->1 with (1-beta_n) J_{beta_n}(eta,gamma*)(pi) -> J_infinity(eta,gamma*)(pi) (Theorem 4.6(iv)).
    A limiting/stationarity condition imposed to convert discounted bounds to average-cost bounds; the paper notes it 'imposes essentially a stationary condition'.
  • domain assumption The intermediate channel Q~^n is informationally equivalent to the finite quantized channel Q^n (Section 5.2).
    Bridges the domain mismatch between Q^n on Y^n and Q on Y in the TV distance computation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sensitivity of Filter Kernels and Robustness Bounds to Transition and Measurement Kernel Perturbations in Partially Observable Stochastic Control." pith.science (2026). https://pith.science/paper/UZBVPTP5

@misc{pith2026250810658,
  author       = {Pith},
  title        = {Pith review of: Sensitivity of Filter Kernels and Robustness Bounds to Transition and Measurement Kernel Perturbations in Partially Observable Stochastic Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UZBVPTP5}},
  note         = {Machine review of arXiv:2508.10658}
}
read the original abstract

Studying the stability of partially observed Markov decision processes (POMDPs) with respect to perturbations in either transition or observation kernels is a significant problem. While asymptotic robustness/stability results as approximate transition kernels and/or measurement kernels converge to the true ones have been previously reported, studies on explicit bounds on value differences and mismatch costs have been limited in scope for POMDPs. In this paper, we provide such explicit bounds under both discounted and average cost criteria. To this end, and also as an independent contribution, we first study the perturbations induced on the filter kernels (that is, the kernels of the belief-MDP reduction of POMDPs) as the transition and measurement kernels are perturbed. The bounds are given in terms of Wasserstein and total variation distances between the original and approximate transition and observation kernels. We then show that control policies optimized for approximate models yield performance guarantees when applied to the true model with explicit bounds. As a particular application, we consider the case where the state space and the measurement spaces are quantized to obtain finite models, and we obtain explicit error bounds which decay to zero as the approximations get finer. This provides explicit performance guarantees for model reduction in POMDPs.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Robustness Analysis of POMDP Policies to Observation Perturbations

    cs.AI 2026-04 unverdicted novelty 7.0 of 10

    POMDP policies can be checked for robustness to observation model changes by solving a bi-level optimization via root-finding with the Robust Interval Search algorithm, which runs in polynomial time for non-sticky his...

Reference graph

Works this paper leans on

51 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    R. J. Aumann. Mixed and behavior strategies in infinite extensive games. Technical report, Princeton University NJ, 1961

    work page 1961

  2. [2]

    Billingsley.Convergence of probability measures

    P. Billingsley.Convergence of probability measures. New York: Wiley, 2nd edition, 1999

    work page 1999

  3. [3]

    Blackwell

    D. Blackwell. Memoryless strategies in finite-stage dynamic programming. Annals of Mathematical Statistics, 35:863–865, 1964

    work page 1964

  4. [4]

    Blanchet and K

    J. Blanchet and K. Murthy. Quantifying distributional model risk via op- timal transport.SSRN Electronic Journal, 04 2016

    work page 2016

  5. [5]

    V. S. Borkar. White-noise representations in stochastic realization theory. SIAM J. on Control and Optimization, 31:1093–1102, 1993

    work page 1993

  6. [6]

    Bozkurt, A

    B. Bozkurt, A. Mahajan, A. Nayyar, and Y. Ouyang. Model approximation in mdps with unbounded per-step cost.IEEE Transactions on Automatic Control, 2025

    work page 2025

  7. [7]

    Crisan and A

    D. Crisan and A. Doucet. A survey of convergence results on particle fil- tering methods for practitioners.IEEE Transactions on Signal Processing, 50(3):736–746, 2002

    work page 2002

  8. [8]

    Demirci, A.D

    Y.E. Demirci, A.D. Kara, and S. Y¨ uksel. Average cost optimality of par- tially observed mdps: Contraction of non-linear filters and existence of optimal solutions.SIAM Journal on Control and Optimization, 62:2859– 2883, 2004

    work page 2004

Show all 51 references
  1. [9]

    Demirci, A.D

    Y.E. Demirci, A.D. Kara, and S. Y¨ uksel. Refined bounds on near optimality finite window policies in pomdps and their reinforcement learning.arXiv, 2024

    2024

  2. [10]

    Demirci, A.D

    Y.E. Demirci, A.D. Kara, and S. Y¨ uksel. Wasserstein regularity of non- linear filters as belief-mdps, and implications on ergodicity, optimality and learning for pomdps. In2025 American Control Conference (ACC). IEEE, 2025

    2025

  3. [11]

    Dobrushin

    R.L. Dobrushin. Central limit theorem for nonstationary Markov chains. i. Theory of Probability & Its Applications, 1(1):65–80, 1956

    1956

  4. [12]

    Domanovitz and A

    E. Domanovitz and A. Khina. Markovian continuity of the mmse.arXiv preprint arXiv:2504.14659, 2025

    2025 arXiv

  5. [13]

    Dufour, A

    F. Dufour, A. G´ enadot, and R. Namyst. The bearing only localization prob- lem via partially observed markov decision process.Mathematical Methods of Operations Research, 2025. 32

    2025

  6. [14]

    P. M. Esfahani and D. Kuhn. Data-driven distributionally robust optimiza- tion using the Wasserstein metric: Performance guarantees and tractable reformulations.Mathematical Programming, pages 1–52, 2017

    2017

  7. [15]

    Feinberg and P.O

    E.A. Feinberg and P.O. Kasyanov. Equivalent conditions for weak conti- nuity of nonlinear filters.Systems & Control Letters, 173:105458, 2023

    2023

  8. [16]

    Feinberg, P.O

    E.A. Feinberg, P.O. Kasyanov, and M.Z. Zgurovsky. Partially observable total-cost Markov decision process with weakly continuous transition prob- abilities.Mathematics of Operations Research, 41(2):656–681, 2016

    2016

  9. [17]

    Feinberg, P.O

    E.A. Feinberg, P.O. Kasyanov, and M.Z. Zgurovsky. Markov decision pro- cesses with incomplete information and semiuniform feller transition prob- abilities.SIAM Journal on Control and Optimization, 60(4):2488–2513, 2022

    2022

  10. [18]

    I. I. Gihman and A. V. Skorohod.Controlled Stochastic Processes. Springer Science & Business Media, 2012

    2012

  11. [19]

    Gordienko, E

    E. Gordienko, E. Lemus-Rodr ´ ıguez, and R. Montes de Oca. Discounted cost optimality problem: stability with respect to weak metrics.Mathematical Methods of Operations Research, 68(1):77–96, 2008

    2008

  12. [20]

    Gordienko, E

    E. Gordienko, E. Lemus-Rodr ´ ıguez, and R. Montes de Oca. Average cost markov control processes: stability with respect to the kantorovich metric. Mathematical Methods of Operations Research, 70:13–33, 2009

    2009

  13. [21]

    Hern´ andez-Lerma.Adaptive Markov control processes, volume 79

    O. Hern´ andez-Lerma.Adaptive Markov control processes, volume 79. Springer Science & Business Media, 2012

    2012

  14. [22]

    Hern´ andez-Lerma and J

    O. Hern´ andez-Lerma and J. B. Lasserre.Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, 1996

    1996

  15. [23]

    C. J. Himmelberg, T. Parthasarathy, and F. S. Van Vleck. Optimal plans for dynamic programming problems.Mathematics of Operations Research, 1(4):390–394, 1976

    1976

  16. [24]

    Hogeboom-Burr and S

    I. Hogeboom-Burr and S. Y¨ uksel. Continuity properties of value functions in information structures for zero-sum and general games and stochastic teams.SIAM Journal on Control and Optimization, 61, 2023

    2023

  17. [25]

    Hogeboom-Burr and S

    I. Hogeboom-Burr and S. Y¨ uksel. Sequential stochastic control (single or multi-agent) problems nearly admit change of measures with independent measurements.Applied Mathematics and Optimization, 2023

    2023

  18. [26]

    A. D. Kara, M. Raginsky, and S. Y¨ uksel. Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control.Auto- matica, 139:110179, 2022. 33

    2022

  19. [27]

    A. D. Kara and S. Y¨ uksel. Robustness to approximations and model learn- ing in MDPs and POMDPs. In A. B. Piunovskiy and Y. Zhang, editors, Modern Trends in Controlled Stochastic Processes: Theory and Applica- tions, Volume III. Luniver Press, 2021

    2021

  20. [28]

    A.D. Kara, E. Bayraktar, and S. Y¨ uksel. Near optimal approximations and finite memory policies for pompds with continuous spaces.Journal of Systems Science and Complexity, page 0, 2025

    2025

  21. [29]

    Saldi, and S

    A.D Kara, N. Saldi, and S. Y¨ uksel. Weak Feller property of non-linear filters.Systems & Control Letters, 134:104–512, 2019

    2019

  22. [30]

    Saldi, and S

    A.D Kara, N. Saldi, and S. Y¨ uksel. Q-learning for MDPs with general spaces: Convergence and near optimality via quantization under weak con- tinuity.Journal of Machine Learning Research, pages 1–34, 2023

    2023

  23. [31]

    Kara and S

    A.D. Kara and S. Y¨ uksel. Partially observed optimal stochastic control: Regularity, optimality, approximations, and learning. InIEEE 51st IEEE Conference on Decision and Control (CDC); arXiv:2412.06735

    arXiv

  24. [32]

    Y¨ uksel

    A.D Kara and S. Y¨ uksel. Robustness to incorrect system models in stochas- tic control.SIAM Journal on Control and Optimization, 58(2):1144–1182, 2020

    2020

  25. [33]

    Y¨ uksel

    A.D Kara and S. Y¨ uksel. Near optimality of finite memory feedback poli- cies in partially observed markov decision processes.Journal of Machine Learning Research, 23(11):1–46, 2022

    2022

  26. [34]

    M. L. Kleptsyna and A. Yu. Veretennikov. On robustness of discrete time optimal filters.Mathematical Methods of Statistics, 25(3):207–218, 2016

    2016

  27. [35]

    Kreitmeier

    W. Kreitmeier. Optimal vector quantization in terms of Wasserstein dis- tance.J. Multivariate Anal., 102(8):1225–1239, 2011

    2011

  28. [36]

    Krishnamurthy.Partially Observed Markov Decision Processes: Fil- tering, Learning and Controlled Sensing

    V. Krishnamurthy.Partially Observed Markov Decision Processes: Fil- tering, Learning and Controlled Sensing. Cambridge University Press, 2 edition, 2025

    2025

  29. [37]

    H.J. Langen. Convergence of dynamic programming models.Mathematics of Operations Research, 6(4):493–512, Nov. 1981

    1981

  30. [38]

    Timothy L. Molloy. Isc-pomdps: Partially observed markov decision pro- cesses with initial-state dependent costs.IEEE Control Systems Letters, 9:3–8, 2025

    2025

  31. [39]

    Molloy and Girish N

    Timothy L. Molloy and Girish N. Nair. Active fixed-sample-size hypothe- sis testing via pomdp value function lipschitz bounds. In2024 American Control Conference (ACC), pages 4176–4181, 2024

    2024

  32. [40]

    Parthasarathy.Probability Measures on Metric Spaces

    K.R. Parthasarathy.Probability Measures on Metric Spaces. AMS Book- store, 1967. 34

    1967

  33. [41]

    D. Rhenius. Incomplete information in Markovian decision models.Ann. Statist., 2:1327–1334, 1974

    1974

  34. [42]

    Saldi, T

    N. Saldi, T. Linder, and S. Y¨ uksel.Finite Approximations in Discrete- Time Stochastic Control: Quantized Models and Asymptotic Optimality. Springer, Cham, 2018

    2018

  35. [43]

    Saldi, S

    N. Saldi, S. Y¨ uksel, and T. Linder. On the asymptotic optimality of finite approximations to Markov decision processes with Borel spaces.Mathe- matics of Operations Research, 42(4):945–978, 2017

    2017

  36. [44]

    Saldi, S

    N. Saldi, S. Y¨ uksel, and T. Linder. Finite model approximations for par- tially observed Markov decision processes with discounted cost.IEEE Transactions on Automatic Control, 65, 2020

    2020

  37. [45]

    R. Serfozo. Convergence of Lebesgue integrals with varying measures. Sankhy¯ a: The Indian Journal of Statistics, Series A, pages 380–402, 1982

    1982

  38. [46]

    Wu and S

    Y. Wu and S. Verd´ u. Functional properties of minimum mean-square er- ror and mutual information.IEEE Transactions on Information Theory, 58(3):1289–1301, 2012

    2012

  39. [47]

    Y¨ uksel

    S. Y¨ uksel. Another look at partially observed optimal stochastic control: Existence, ergodicity, and approximations without belief-reduction.Applied Mathematics & Optimization, 91(1):16, 2025

    2025

  40. [48]

    Y¨ uksel and T

    S. Y¨ uksel and T. Ba¸ sar.Stochastic Teams, Games, and Control under Information Constraints. Springer, Cham, 2024

    2024

  41. [49]

    Y¨ uksel and T

    S. Y¨ uksel and T. Linder. Optimization and convergence of observation channels in stochastic control.SIAM J. on Control and Optimization, 50:864–887, 2012

    2012

  42. [50]

    Yushkevich

    A.A. Yushkevich. Reduction of a controlled Markov model with incomplete data to a problem with complete information in the case of Borel state and control spaces.Theory Prob. Appl., 21:153–158, 1976

    1976

  43. [51]

    Robustness to model ap- proximation, learning, and sample complexity in wasserstein regular mdps

    Yichen Zhou, Yanglei Song, and Serdar Y¨ uksel. Robustness to model ap- proximation, learning, and sample complexity in wasserstein regular mdps. arXiv preprint arXiv:2410.14116, 2024. 35

    2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.