Paper Citation Record · LEDGER

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise

As of 22 August 2026, this Paper Citation Record lists 12 of 12 outbound references and 5 inbound Pith citation observations for arXiv:2411.13711.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2411.13711 v1

Coverage vector

measured 12 of 12 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-12T16:28:45.790771Z

measured 17 of 17 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-22T06:32:14.747728+00:00

measured 5 of 5 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-08-01T17:43:48.412070Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-06-28T23:22:46.684996Z

Reference resolution

12 of 12 outbound references displayed

verified exact1
verified fuzzy0
unresolved11
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation a7af71e4-8914-49b6-9e05-64c45cd97d1f · outbound

This paper cites Revisiting Step-Size Assumptions in Stochastic Approximation.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise Revisiting Step-Size Assumptions in Stochastic Approximation

Reference 7

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.672894Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1ecdfd84-6a8f-4e7d-91e9-1675a1ce3c25 · outbound

This paper cites Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization

Reference 10

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.685956Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 62a08250-6b85-424d-a53d-2c42207e67e2 · outbound

This paper cites A Concentration Bound for Stochastic Approximation via Alekseev's Formula.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise A Concentration Bound for Stochastic Approximation via Alekseev's Formula

Reference 11

Resolution: verified exact; local_arxiv, observed 2026-08-12T16:28:45.924261Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 858de273-6044-4e78-ae29-077b49b5148b · outbound

This paper cites A concentration bound for td (0) with function approximation.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise A concentration bound for td (0) with function approximation

Reference 2009

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.650228Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 209a0c69-b10c-497d-8966-ed9f76f9893f · outbound

This paper cites Optimal variance-reduced stochastic approximation in Banach spaces.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise Optimal variance-reduced stochastic approximation in Banach spaces

Reference 2011

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.681249Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4b728f1b-7855-4b5a-9597-9853f0dbb4b7 · outbound

This paper cites A Unified Switching System Perspective and O.D.E. Analysis of Q-Learning Algorithms.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise A Unified Switching System Perspective and O.D.E. Analysis of Q-Learning Algorithms

Reference 2015

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.677005Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5ef20281-605c-4df1-8f02-9745a618e722 · outbound

This paper cites Simple and optimal high-probability bounds for strongly-convex stochastic gradient descent.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise Simple and optimal high-probability bounds for strongly-convex stochastic gradient descent

Reference 2016

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.668409Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 4ce5fb79-b1f8-42f3-b960-8f01125195a1 · outbound

This paper cites The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning

Reference 2018

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.645167Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d525b9a2-4279-4cb1-b182-8a632a53c370 · outbound

This paper cites A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants

Reference 2020

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.659398Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation dddba068-43cb-4e21-9dda-2a7ce58eb053 · outbound

This paper cites Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise

Reference 2021

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.664062Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d855268a-8c1f-48c5-86cf-b152f17a6e97 · outbound

This paper cites Finite-Sample Analysis of Stochastic Approximation Using Smooth Convex Envelopes.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise Finite-Sample Analysis of Stochastic Approximation Using Smooth Convex Envelopes

Reference 2022

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.654440Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1fc03eb2-5d19-4bfe-81a9-8fec8f1e403d · outbound

This paper cites Stochastic approximation with cone-contractive operators: Sharp $\ell_\infty$-bounds for $Q$-learning.

Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise Stochastic approximation with cone-contractive operators: Sharp $\ell_\infty$-bounds for $Q$-learning

Reference 2023

Resolution: unresolved; no resolver link, observed 2026-08-12T16:28:45.790771Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation 46969412-f26c-42cc-86e3-e8d8dc8a448a · inbound

From Set Convergence to Pointwise Convergence: Finite-Time Guarantees for Average-Reward Q-Learning with Adaptive Stepsizes cites this paper.

From Set Convergence to Pointwise Convergence: Finite-Time Guarantees for Average-Reward Q-Learning with Adaptive Stepsizes Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise

Reference 63

Resolution: verified exact; arxiv_id, observed 2026-05-22T17:35:00.886167Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 37d4a5d2-ea98-4e26-b95b-3f9da7b8d639 · inbound

Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise cites this paper.

Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise

Reference 189

Resolution: verified exact; arxiv_id, observed 2026-05-21T02:29:25.120406Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 09600e6d-f149-4f19-8e66-c4adcce72e9a · inbound

Non-Asymptotic Convergence of Stochastic Iterative Algorithms: A Lyapunov Framework cites this paper.

Non-Asymptotic Convergence of Stochastic Iterative Algorithms: A Lyapunov Framework Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise

Reference 96

Resolution: verified exact; arxiv_id, observed 2026-06-28T23:12:46.708825Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation a2d748cd-b955-4d18-a041-7b23109f17d6 · inbound

Large-scale Uncertainty Quantification for Latent Variable Models Using Subsampling Markov Chain Monte Carlo cites this paper.

Large-scale Uncertainty Quantification for Latent Variable Models Using Subsampling Markov Chain Monte Carlo Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise

Reference 41

Resolution: metadata mismatch; arxiv_id, observed 2026-06-28T23:22:46.686693Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-22T06:32:14.747728+00:00.

Observation 71b9b92d-810f-49c1-8961-216e7c228253 · inbound

Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach cites this paper.

Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach Almost Sure Convergence Rates and Concentration of Stochastic Approximation and Reinforcement Learning with Markovian Noise

Reference 24

Resolution: unresolved; no resolver link, observed 2026-08-01T17:43:48.412070Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.