Paper Citation Record · LEDGER

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning

As of 22 August 2026, this Paper Citation Record lists 35 of 35 outbound references and 2 inbound Pith citation observations for arXiv:2506.01261.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2506.01261 v1

Coverage vector

measured 35 of 35 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-07T11:54:26.316448Z

measured 37 of 37 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-21T06:32:19.484+00:00

measured 2 of 2 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links, observed 2026-06-28T11:44:53.211503Z

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: arxiv_reference, observed 2026-07-02T01:36:25.635046Z

Reference resolution

35 of 35 outbound references displayed

verified exact0
verified fuzzy22
unresolved11
parse uncertain0
malformed identifier0
metadata mismatch2

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation f866580d-3a6a-4ac7-b845-022d9dddbfc3 · outbound

This paper cites Fitted q-iteration in continuous action-space mdps.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Fitted q-iteration in continuous action-space mdps

Reference 1

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:32.085703Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 7ad5d9ca-845d-4ea8-8c06-04e49da4417b · outbound

This paper cites S., and Guin, S.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning S., and Guin, S

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:23.284017Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c672ef05-d8e8-4300-b67b-2cfaae3b0fc8 · outbound

This paper cites OpenAI Gym.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning OpenAI Gym

Reference 3

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:23.368455Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c7cacfe5-bcd0-4e18-b0f7-f160f1699d2f · outbound

This paper cites D., and Wang, Z.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning D., and Wang, Z

Reference 4

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:31.888111Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation c8deb5e1-1c7a-4d43-b690-9ae93ad6d974 · outbound

This paper cites W., Hilton, J., Klimov, O., and Schulman, J.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning W., Hilton, J., Klimov, O., and Schulman, J

Reference 5

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:31.641212Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 300c19b8-a938-4320-b4dd-c41e32cb8cff · outbound

This paper cites Linear off-policy actor-critic.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Linear off-policy actor-critic

Reference 6

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:31.431771Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation c0ff3ad0-e46a-4700-a7cc-47aed292b5ad · outbound

This paper cites A theoretical analysis of deep q-learning.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning A theoretical analysis of deep q-learning

Reference 7

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:31.246319Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 3a257f89-2ad1-41ae-9a14-5468bd6c9ba5 · outbound

This paper cites an unresolved cited work.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Unresolved cited work

Reference 8

Resolution: unresolved; raw_fallback, observed 2026-08-07T11:54:31.047177Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 2e2979de-4976-4bf6-87ac-ef722abf575a · outbound

This paper cites Error propagation for approximate policy and value iteration.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Error propagation for approximate policy and value iteration

Reference 9

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:30.864069Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 8dbfb1cd-09cf-482d-acd3-d80dfdbb919e · outbound

This paper cites Reinforcement learning with deep energy- based policies.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Reinforcement learning with deep energy- based policies

Reference 10

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:30.668546Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 358e5bf4-13f5-43e1-9ef0-c75e466949be · outbound

This paper cites Federated reinforcement learning with environment heterogeneity.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Federated reinforcement learning with environment heterogeneity

Reference 11

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:30.469003Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 12327596-a491-4a38-8cb4-3196ee54b005 · outbound

This paper cites P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A

Reference 12

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:30.225430Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 66c651a9-28fd-4962-84c3-d6b1b7bb6311 · outbound

This paper cites and Tsitsiklis, J.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning and Tsitsiklis, J

Reference 13

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:30.016740Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation ba962be4-b9d1-4ccc-86b7-75adc426ddac · outbound

This paper cites K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V

Reference 14

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:29.781855Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation a6420aa0-3851-418a-b662-b8c9ab552015 · outbound

This paper cites On the convergence of fedavg on non-iid data.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning On the convergence of fedavg on non-iid data

Reference 15

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:29.555582Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 34a3dda1-b350-45d5-bab2-6d89dd56ba48 · outbound

This paper cites Neural trust region/proximal policy optimization attains globally optimal policy.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Neural trust region/proximal policy optimization attains globally optimal policy

Reference 16

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:29.346041Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 65f24c31-4e0e-473a-ab9e-ebeb3b62479b · outbound

This paper cites an unresolved cited work.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Unresolved cited work

Reference 17

Resolution: unresolved; raw_fallback, observed 2026-08-07T11:54:29.152442Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 709cd9a4-1602-451f-85c3-1e542b3155fc · outbound

This paper cites On the global convergence rates of softmax policy gradient methods.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning On the global convergence rates of softmax policy gradient methods

Reference 18

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:28.968600Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation b5d387ed-4cd7-43e2-8d98-86763ca50cc6 · outbound

This paper cites A., Veness, J., Bellemare, M.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning A., Veness, J., Bellemare, M

Reference 19

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:28.764589Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 29a9c553-d78e-438d-bf62-611abe8ae074 · outbound

This paper cites and Szepesvári, C.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning and Szepesvári, C

Reference 20

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:28.584788Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 9718f0b3-1123-4839-9bce-329c0def2396 · outbound

This paper cites Planet dump retrieved from https://planet.osm.org.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Planet dump retrieved from https://planet.osm.org

Reference 21

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:28.421351Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 12f480a1-5c6c-4313-b891-fa47167bf3b7 · outbound

This paper cites Trust Region Policy Optimization.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Trust Region Policy Optimization

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:25.098661Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9292f581-f4b7-4a79-858f-3cf4e5080240 · outbound

This paper cites Proximal Policy Optimization Algorithms.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Proximal Policy Optimization Algorithms

Reference 23

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:25.165690Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 12536ad8-a079-4510-876b-58edb0fc1fb0 · outbound

This paper cites an unresolved cited work.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Unresolved cited work

Reference 24

Resolution: unresolved; raw_fallback, observed 2026-08-07T11:54:28.208016Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 54b8813e-da05-4c67-8138-8a79fcc5c049 · outbound

This paper cites S., McAllester, D., Singh, S., and Mansour, Y.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning S., McAllester, D., Singh, S., and Mansour, Y

Reference 25

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:27.978909Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation fd1f72d0-98e4-4f32-9d04-6696078872f9 · outbound

This paper cites Boosted fitted q-iteration.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Boosted fitted q-iteration

Reference 26

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:27.778136Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 5eb2fc9f-e882-4ecd-927b-9ba6abec0656 · outbound

This paper cites Deep reinforcement learning with double q-learning.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Deep reinforcement learning with double q-learning

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:25.514852Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 014e2b7f-8963-441d-95b9-07541ce6162d · outbound

This paper cites E., Srivastava, S., Tuia, D., and Falcão, A.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning E., Srivastava, S., Tuia, D., and Falcão, A

Reference 28

Resolution: metadata mismatch; raw_fallback, observed 2026-08-07T11:54:26.944638Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 27b574ad-db67-4b56-b86b-4083bd2da289 · outbound

This paper cites L., Kheterpal, N., Jang, K., Wu, C., Wu, F., Liaw, R., Liang, E., and Bayen, A.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning L., Kheterpal, N., Jang, K., Wu, C., Wu, F., Liaw, R., Liang, E., and Bayen, A

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:27.583825Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 935a9aa8-71f1-4b99-8138-d8b4689fdcc5 · outbound

This paper cites Neural Policy Gradient Methods: Global Optimality and Rates of Convergence.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Neural Policy Gradient Methods: Global Optimality and Rates of Convergence

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:25.833975Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 98bad912-d5dd-449b-b01d-99871ff1812b · outbound

This paper cites P., and Kakade, S.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning P., and Kakade, S

Reference 31

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:27.391913Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 39660861-fed3-4826-b7ea-26838d4e7be3 · outbound

This paper cites and Song, S.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning and Song, S

Reference 32

Resolution: metadata mismatch; raw_fallback, observed 2026-08-07T11:54:26.697142Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 8b3badc4-4a1d-4f43-83be-6a93508fab2d · outbound

This paper cites A General Approach to Adding Differential Privacy to Iterative Training Procedures.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning A General Approach to Adding Differential Privacy to Iterative Training Procedures

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:26.115849Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 852b693b-da84-4ba9-8f6e-ee8146aed99f · outbound

This paper cites Federated natural policy gradient and actor critic methods for multi-task reinforcement learning.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Federated natural policy gradient and actor critic methods for multi-task reinforcement learning

Reference 34

Resolution: verified fuzzy; raw_fallback, observed 2026-08-07T11:54:27.168863Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 722dd00c-e413-424a-b058-98232ef9a157 · outbound

This paper cites Federated Learning with Non-IID Data.

The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning Federated Learning with Non-IID Data

Reference 35

Resolution: unresolved; no resolver link, observed 2026-08-07T11:54:26.316448Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

Observation b32d63e9-5e95-4638-a392-534a5ecc3069 · inbound

Collaborative Yet Personalized Policy Training: Single-Timescale Federated Actor-Critic cites this paper.

Collaborative Yet Personalized Policy Training: Single-Timescale Federated Actor-Critic The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning

Reference 48

Resolution: verified exact; arxiv_id, observed 2026-05-15T02:08:29.340835Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.

Observation 1377c879-6903-497f-b020-dba0efa9ead2 · inbound

FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data cites this paper.

FGRPO: Federated GRPO with Adaptive Aggregation on Non-IID Data The Actor-Critic Update Order Matters for PPO in Federated Reinforcement Learning

Reference 32

Resolution: metadata mismatch; arxiv_id, observed 2026-07-02T01:36:25.637603Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-21T06:32:19.484+00:00.