pith. sign in

arxiv: 2607.05179 · v1 · pith:2BWX66IF · submitted 2026-07-06 · cs.LG · cs.AI· cs.MA

Relational Multi-Agent Reinforcement Learning for Dynamic Pricing in High-Speed Railway Markets

Enrique Adrian Villarrubia-Martin , David Mu\~noz-Valero , Luis Rodriguez-Benitez , Giovanni Montana , Luis Jimenez-Linares This is my paper

pith:2BWX66IFreviewed 2026-07-08 00:52 UTCmodel glm-5.2open to challenge →

classification cs.LG cs.AIcs.MA
keywords dynamic pricingmulti-agent reinforcement learninggraph neural networksrelational graph convolutional networkrailway systemscentralised training decentralised executionentity graphattention mechanism
0
0 comments X

The pith

Entity graphs beat flat vectors in multi-agent rail pricing

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes RACHE, a framework that models a railway pricing environment as a graph of individual train services (not operators or stations) connected by three typed edges: competition (services vying for the same passengers), coordination (services run by the same operator), and connectivity (services that can form multi-leg passenger journeys). Each agent independently processes this graph through a relational graph convolutional network with attention-based pooling, producing a compact state vector that captures which competing and connecting services matter most for pricing decisions. This representation feeds into an actor-critic reinforcement learning algorithm (MATD3) under centralised training with decentralised execution, meaning agents learn from global information during training but price independently at runtime. The key design choice is that graph nodes are operational units (train services) rather than decision-makers (operators), which preserves fine-grained competitive and cooperative structure that agent-level graphs collapse. A gradient-stopping mechanism decouples representation learning from policy optimisation, stabilising per-agent training at a potential cost to total revenue. In two simulated railway markets of increasing complexity, RACHE achieves higher total revenue than both relational and non-relational baselines while using smaller price increases and retaining more passengers.

Core claim

Representing the market as an entity graph where train services are nodes and competition, coordination, and connectivity are heterogeneous edge types, processed through a multi-layer relational graph convolutional network with learned attention pooling, yields higher revenue and training stability than treating observations as flat vectors or modelling operators as graph nodes. The ablation studies show that competition edges contribute most to performance, followed by connectivity, while coordination (shared ownership) contributes least, indicating that market relationships drive pricing strategy more than organisational structure. Two message-passing layers outperform one or three, and a

What carries the argument

R-GCN (Relational Graph Convolutional Network) with attention pooling integrated into MATD3 under CTDE

If this is right

  • If the entity-graph approach generalises, it could apply to other partially observable, multi-agent markets with relational structure, such as airline pricing, ride-hailing surge pricing, or electricity market bidding, where competitors cannot communicate but must infer strategic interactions from observable market data.
  • The finding that coordination edges (same-operator services) contribute least while competition and connectivity edges contribute most suggests that pricing agents benefit primarily from modelling market-level interdependencies rather than internal portfolio management, which has implications for how operators should structure their own decision-support systems.
  • The scalability property, where model parameters are independent of the number of services due to weight sharing in message passing, means the framework could handle markets with hundreds or thousands of services without proportional growth in model size.
  • The gradient-stopping mechanism that decouples representation learning from policy optimisation presents a general technique for stabilising multi-agent training when shared representation modules receive conflicting gradient signals from actors and critics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The evaluation is conducted entirely in a simulator developed by the same research group, with passenger demand modelled by fixed Random Utility Models across four segments. If real passenger demand elasticity differs substantially from these models, the revenue improvements may not transfer to live railway markets.
  • The baseline comparison uses default (untuned) hyperparameters for all algorithms, but MADDPG operates with a different discount factor (0.95 vs 0.99) and learning rate (0.01 vs 0.001) than the shared configuration, which could systematically disadvantage it and inflate RACHE's relative performance.
  • The entity graph edge types are defined by static rules (same date, overlapping markets, station connectivity). In real markets, competitive relationships may be more fluid, for instance a service might compete with a flight or bus route, which the current graph does not capture.
  • The two-layer GNN outperforming three layers suggests the framework may struggle in markets where strategic dependencies extend beyond two hops in the service graph, such as complex multi-leg journeys with two or more transfers.

Load-bearing premise

All revenue improvements are measured in a railway pricing simulator whose passenger demand model uses four fixed traveller segments with predefined utility functions, and the results have not been validated against real pricing data or an independent simulator.

What would settle it

If replacing the flat observation encoding in a standard MARL algorithm with the entity-graph representation does not improve revenue or stability in an independent railway pricing environment with different demand models, the core claim that service-level relational structure is the key driver of performance would not hold.

Figures

Figures reproduced from arXiv: 2607.05179 by David Mu\~noz-Valero, Enrique Adrian Villarrubia-Martin, Giovanni Montana, Luis Jimenez-Linares, Luis Rodriguez-Benitez.

Figure 1
Figure 1. Figure 1: MARL interaction in a partially observable MG. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Entity graph representation for a railway network with 15 services scheduled [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Architecture of the relational state representation module. For each agent, [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Entity graph representations for the experimental scenarios. Each node corre [PITH_FULL_IMAGE:figures/full_fig_p021_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Average total revenue obtained at training for the two scenarios. [PITH_FULL_IMAGE:figures/full_fig_p026_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Average revenue obtained at evaluation using the last checkpoint for each agent. [PITH_FULL_IMAGE:figures/full_fig_p027_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Probability of improvement matrix using the Mann–Whitney [PITH_FULL_IMAGE:figures/full_fig_p027_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Distribution of the final price change per seat type relative to the initial fare. [PITH_FULL_IMAGE:figures/full_fig_p028_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Ablation study comparing average total revenue with each relation type removed. [PITH_FULL_IMAGE:figures/full_fig_p030_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Ablation study comparing average total revenue for one, two, and three layers. [PITH_FULL_IMAGE:figures/full_fig_p031_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Ablation study comparing average total revenue of learnt attention weights [PITH_FULL_IMAGE:figures/full_fig_p032_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Normalised mean cross-attention weights between agents at training step 200K. [PITH_FULL_IMAGE:figures/full_fig_p033_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Ablation study comparing average total revenue of detach and no detach vari [PITH_FULL_IMAGE:figures/full_fig_p033_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: t-SNE visualisation of learnt agent observations coloured by agent and reward. [PITH_FULL_IMAGE:figures/full_fig_p036_14.png] view at source ↗
read the original abstract

In liberalised railway systems, operators must set prices dynamically in an environment with partial observability, as they retain private information about their objectives and performance, where regulatory constraints prohibit communication or direct information exchange between competitors to prevent explicit collusion. Consequently, agents must learn to infer strategic interactions only from observable market data which presents a significant challenge for multi-agent reinforcement learning, where standard approaches typically treat observations as unstructured vectors, ignoring the underlying market topology that governs strategic interactions. To address this, an entity graph modelling approach is proposed, which represents the environment as a graph of operational units, rather than decision-making agents or static infrastructure, encoding competition, coordination, and connectivity relations between entities. Then, an extension of the multi-agent twin delayed deep deterministic policy gradient algorithm with graph-based representation learning processes the features of the entities through a multi-layer relational graph convolutional network and aggregates them via a learnt attention mechanism. Experimental results in a rail pricing reinforcement learning environment show that this novel framework achieves higher revenue and stability in two different settings of increasing market complexity compared to a representative selection of relational and non-relational baselines. The code is publicly available at: https://github.com/Kinrre/RelationalRailPricing-RL

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 8 minor

Summary. The paper proposes RACHE, a relational actor-critic framework for multi-agent reinforcement learning in dynamic railway pricing. The core idea is to model the environment as an entity graph where nodes are train services (not agents or stations) and edges encode competition, coordination, and connectivity relations. Each agent maintains its own R-GCN with attention pooling to produce a fixed-size relational state representation, integrated into MATD3 under CTDE. A gradient-detach mechanism decouples representation learning from policy optimization. Experiments in the RailPricing-RL simulator across two scenarios (18 services/3 agents and 63 services/4 agents) show RACHE achieving higher total revenue than MADDPG, MATD3, MAAC, and GA-AC baselines, with bootstrap confidence intervals over three seeds and a pairwise probability-of-improvement analysis. Ablation studies examine edge-type contributions, network depth, attention vs. uniform pooling, the detach mechanism, and learned embeddings.

Significance. The entity-graph modelling approach—where operational units rather than agents or infrastructure are graph nodes—is a reasonable conceptual contribution to graph-based MARL. The ablation studies are thorough and individually informative: the edge-type removal (Section 5.6.1), depth analysis (Section 5.6.2), attention comparison (Section 5.6.3), and detach analysis (Section 5.6.4) each isolate a specific design choice. The gradient-detach mechanism and its stability analysis (Table 5) provide a useful empirical finding about the trade-off between training stability and total revenue. Public code availability and reproducible hyperparameters are appreciated. The pairwise probability-of-improvement matrix (Figure 7) is a sound statistical treatment. However, the central comparative claim is undermined by unresolved hyperparameter disparities among baselines (see major comments).

major comments (2)
  1. Section 5.4, Tables A.1 and A.3: The paper states that 'the default hyperparameters of each algorithm were adopted without additional tuning,' but Table A.3 shows MADDPG uses γ=0.95 and learning rate=0.01, while the shared defaults (Table A.1) specify γ=0.99 and lr=0.001. A 10× higher learning rate and a materially different discount factor could disadvantage MADDPG independently of the architectural contribution being evaluated. Since RACHE inherits the shared defaults via MATD3, the revenue gap between RACHE and MADDPG (41036 vs. 32688 in Base; 53831 vs. 43902 in Large) cannot be cleanly attributed to the relational architecture. The paper should either (a) run MADDPG with matched hyperparameters as a control, or (b) explicitly acknowledge this confound and reframe the comparison accordingly. The RACHE-vs-MATD3 comparison (both using γ=0.99, lr=0.001) is more controlled and supports a
  2. Section 5.5, Table 2: Only three seeds are used for all experiments. While the bootstrap confidence intervals and probability-of-improvement analysis are reasonable given this sample size, three seeds is at the low end for MARL evaluation. The confidence intervals for MATD3 in the Base scenario [36312, 37739] and Large scenario [46930, 49425] are wide relative to the RACHE intervals, suggesting the comparison could be underpowered. The paper should either acknowledge this limitation more prominently or provide additional seeds for the closest competitor (MATD3) to strengthen the central claim.
minor comments (8)
  1. Section 4.2.2: The R-GCN update equation uses notation f^(ℓ) for the activation function but does not specify which activation is used in intermediate layers (ReLU is mentioned in the text but not in the equation). Please clarify.
  2. Table 1: The percentages of edge types (e.g., 21%, 61%, 18% for Base) appear to be proportions of total edges, but this is not explicitly stated. Please label.
  3. Section 5.6.4, Figure 13: In the Large scenario, the no-detach variant appears to achieve higher final revenue than the detach variant, which contradicts the paper's recommendation of the detach mechanism. The text acknowledges this but the framing could be clearer about when each variant is preferable.
  4. Section 5.6.5, Figure 14: The t-SNE visualizations are qualitative and the claim that embeddings 'capture aspects of agent identity and market profitability' would be strengthened by a quantitative measure (e.g., silhouette score or a simple classifier on the embeddings).
  5. Section 5.1: The reward function assumes no costs for providing services, which is acknowledged but could be more prominently discussed as a limitation on the realism of the learned policies.
  6. Table A.3: MADDPG uses Ornstein-Uhlenbeck exploration noise while MATD3/RACHE use Gaussian noise. This is an additional uncontrolled difference that should be noted.
  7. The paper would benefit from a brief discussion of how the entity-graph approach would handle dynamic graph topologies where services are added or removed at runtime, which is common in real-time pricing decisions.
  8. Section 2.2: The related work on graph-based MARL could more explicitly position the entity-graph contribution against InforMARL [9], which also uses agent-entity graphs, to clarify what is novel beyond the railway domain application.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for a careful and constructive report. The two major comments are addressed below. On the hyperparameter confound for MADDPG, we agree the point is valid and will run a matched-hyperparameter control experiment in revision. On the seed count, we will add additional seeds for MATD3 and strengthen the limitations discussion.

read point-by-point responses

  1. Referee: Section 5.4, Tables A.1 and A.3: MADDPG uses γ=0.95 and lr=0.01 while shared defaults specify γ=0.99 and lr=0.001. The revenue gap between RACHE and MADDPG cannot be cleanly attributed to the relational architecture. The paper should either (a) run MADDPG with matched hyperparameters as a control, or (b) explicitly acknowledge this confound and reframe the comparison accordingly. The RACHE-vs-MATD3 comparison is more controlled.

    Authors: The referee is correct that the hyperparameter disparity between MADDPG (γ=0.95, lr=0.01) and the shared defaults (γ=0.99, lr=0.001) constitutes a confound for the RACHE-vs-MADDPG comparison. We appreciate this careful observation. We acknowledge that the current manuscript does not adequately address this issue. In revision, we will run MADDPG with matched hyperparameters (γ=0.99, lr=0.001) as an additional control in both scenarios and report the results. We will also add an explicit discussion of this confound in Section 5.4, noting that the RACHE-vs-MATD3 comparison—which uses matched hyperparameters and isolates the architectural contribution of the relational representation—is the cleaner and more directly relevant comparison for evaluating the entity-graph approach. The MADDPG comparison will be reframed as a reference to the broader MARL literature's default configuration rather than as a controlled architectural comparison. revision: yes

  2. Referee: Section 5.5, Table 2: Only three seeds are used for all experiments. While the bootstrap CIs and probability-of-improvement analysis are reasonable given this sample size, three seeds is at the low end for MARL evaluation. The CIs for MATD3 are wide relative to RACHE intervals, suggesting the comparison could be underpowered. The paper should either acknowledge this limitation more prominently or provide additional seeds for the closest competitor (MATD3).

    Authors: We agree that three seeds is at the low end for MARL evaluation and that the wide MATD3 confidence intervals suggest the comparison could benefit from additional statistical power. We will address this in two ways. First, we will run additional seeds (targeting at least five total) for MATD3, the closest competitor and the most controlled comparison, in both scenarios. Second, we will add a more prominent and explicit limitations paragraph in Section 5.5 acknowledging that three seeds is a small sample size, that this affects the statistical power of the comparisons particularly for baselines with high variance, and that the bootstrap CIs and probability-of-improvement analysis should be interpreted with this caveat in mind. We note that even with the current wide MATD3 intervals, the RACHE-vs-MATD3 probability-of-improvement values (0.94 in Base, 0.98 in Large) and the non-overlapping confidence intervals provide reasonable evidence for the central claim, but we agree the case is strengthened by additional seeds. revision: yes

Circularity Check

0 steps flagged

No significant circularity; the one self-citation is to the evaluation simulator, not to a theoretical premise.

full rationale

The paper's derivation chain is self-contained. The entity graph construction rules (Section 4.1) are defined by domain semantics—competition edges connect services sharing a market operated by different agents, coordination edges link same-agent services, connectivity edges capture potential passenger transfers. These rules are not defined in terms of the output metric (revenue), so there is no self-definitional circularity. The R-GCN message passing (Eq. in Section 4.2.2), attention pooling (Eqs. 1–2), and MATD3 critic/actor losses (Eqs. 3–5) are standard architectural components assembled independently of the evaluation results. No parameter is fitted to a subset of data and then 'predicted' on a closely related quantity. The one self-citation is to RailPricing-RL [14], the simulator used for evaluation. This is a self-citation, but it is not load-bearing in the circularity sense: the simulator encodes domain assumptions (RUM-based passenger demand, market structure), not the target result that RACHE should outperform baselines. The simulator does not force RACHE's architecture to win by construction. The hyperparameter disparity concern (MADDPG using γ=0.95 and lr=0.01 vs shared defaults of γ=0.99 and lr=0.001, per Tables A.1 and A.3) is a fairness/correctness risk, not a circularity issue—the paper does not define MADDPG's hyperparameters in terms of RACHE's revenue advantage. No uniqueness theorem is invoked, no ansatz is smuggled through self-citation, and no known result is merely renamed. The score of 1 reflects the minor self-citation to the simulator environment, which is standard practice and does not undermine the independence of the architectural contribution.

Axiom & Free-Parameter Ledger

8 free parameters · 4 axioms · 0 invented entities

The framework introduces no new physical entities or postulated objects. All components (R-GCN, attention, MATD3) are standard neural network modules. The free parameters are standard hyperparameters selected by convention or ablation, not fitted to force the result. The primary risk is in the domain assumptions about simulator fidelity and fair baseline comparison, not in invented entities.

free parameters (8)
  • R-GCN learning rate = 0.001
    Set to default, not tuned per Section 5.4
  • R-GCN hidden dim = 128
    Chosen architecture parameter
  • Number of R-GCN layers = 2
    Selected via ablation (Section 5.6.2)
  • Categorical embedding dim = 4
    Architecture choice
  • Continuous embedding dim = 128
    Architecture choice
  • Dropout rate = 0.1
    Regularization parameter
  • Price scaling factor beta = 25
    Default from simulator, controls action magnitude
  • Reward scaling factor = 1000
    Chosen for numerical stability
axioms (4)
  • domain assumption The RailPricing-RL simulator accurately models passenger behaviour via Random Utility Models with four segments
    Section 5.1: all experiments depend on this simulator's fidelity to real markets
  • domain assumption Default hyperparameters of each baseline algorithm provide a fair comparison
    Section 5.4: 'the default hyperparameters of each algorithm were adopted without additional tuning'
  • domain assumption Three random seeds are sufficient for statistical claims about MARL performance
    Section 5.5: all results use seeds 0, 41, 73 only
  • domain assumption The entity graph edge definitions (competition, coordination, connectivity) capture the strategically relevant relations
    Section 4.1: these three edge types are postulated as sufficient for capturing market structure

pith-pipeline@v1.1.0-glm · 27858 in / 2515 out tokens · 189040 ms · 2026-07-08T00:52:49.078958+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages · 5 internal anchors

  1. [1]

    Jiang, C

    J. Jiang, C. Dun, T. Huang, Z. Lu, Graph Convolutional Reinforcement Learning, in: 8th International Conference on Learning Representations, ICLR 2020, 2020

    work page 2020

  2. [2]

    Goeckner, Y

    A. Goeckner, Y. Sui, N. Martinet, X. Li, Q. Zhu, Graph neural network- based multi-agent reinforcement learning for resilient distributed co- ordination of multi-robot systems, in: 2024 IEEE/RSJ International 40 Conference on Intelligent Robots and Systems (IROS), IEEE, 2024, pp. 5732–5739

    work page 2024

  3. [3]

    S. Utke, J. Houssineau, G. Montana, Investigating relational state ab- straction in collaborative marl, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, 2025, pp. 20947–20955

    work page 2025

  4. [4]

    Ardjmand, E

    E. Ardjmand, E. Izadi, A. Tavasoli, B. Moradi-Jamei, H. Shakeri, Graph-embedded reinforcement learning for dynamic pricing and ad- vertising under network effects, Applied Soft Computing 186 (2026). doi:10.1016/j.asoc.2025.114056

    work page doi:10.1016/j.asoc.2025.114056 2026

  5. [5]

    Q. Liu, L. Zhang, M. J. Skibniewski, Network extension plan- ning towards resilient urban critical infrastructures using deep reinforcement learning, Applied Soft Computing 176 (2025). doi:10.1016/j.asoc.2025.113163

    work page doi:10.1016/j.asoc.2025.113163 2025

  6. [6]

    T. Li, Y. Qin, M. Xu, Y. Zhou, L. Rong, Spatio-temporal vulnerability ofhigh-speedraillinenetworkin china, TransportationResearchPart D: Transport and Environment 134 (2024). doi:10.1016/j.trd.2024.104338

    work page doi:10.1016/j.trd.2024.104338 2024

  7. [7]

    Gutiérrez-Hita, O

    C. Gutiérrez-Hita, O. de la Cruz, R. Ramos-Melero, Infrastructure ac- cess charges, service differentiation, and strategic competition in the eu railway passenger market., Transportation Research Part B: Method- ological 164 (2022). doi:10.1016/j.trb.2022.08.004

    work page doi:10.1016/j.trb.2022.08.004 2022

  8. [8]

    Agarwal, S

    A. Agarwal, S. Kumar, K. Sycara, M. Lewis, Learning transferable co- operative behavior in multi-agent team, in: Proceedings of the Interna- tionalJointConferenceonAutonomousAgentsandMultiagentSystems, AAMAS, Vol. 2020-May, 2020. doi:10.65109/edcb3795

    work page doi:10.65109/edcb3795 2020

  9. [9]

    Nayak, K

    S. Nayak, K. Choi, W. Ding, S. Dolan, K. Gopalakrishnan, H. Balakr- ishnan, Scalable multi-agent reinforcement learning through intelligent information aggregation, in: International conference on machine learn- ing, PMLR, 2023, pp. 25817–25833

    work page 2023

  10. [10]

    J. N. Foerster, G. Farquhar, T. Afouras, N. Nardelli, S. Whiteson, Coun- terfactual multi-agent policy gradients, in: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, 2018. doi:10.1609/aaai.v32i1.11794. 41

    work page doi:10.1609/aaai.v32i1.11794 2018

  11. [11]

    2017-December, 2017

    R.Lowe, Y.Wu, A.Tamar, J.Harb, P.Abbeel, I.Mordatch, Multi-agent actor-critic for mixed cooperative-competitive environments, in: Ad- vances in Neural Information Processing Systems, Vol. 2017-December, 2017

    work page 2017

  12. [12]

    Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Critics

    J. Ackermann, V. Gabler, T. Osa, M. Sugiyama, Reducing overestima- tion bias in multi-agent domains using double centralized critics, arXiv preprint arXiv:1910.01465 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  13. [13]

    Zambaldi, D

    V. Zambaldi, D. Raposo, A. Santoro, V. Bapst, Y. Li, I. Babuschkin, K. Tuyls, D. Reichert, T. Lillicrap, E. Lockhart, M. Shanahan, V. Langston, R. Pascanu, M. Botvinick, O. Vinyals, P. Battaglia, Deep reinforcement learning with relational inductive biases, in: 7th Interna- tional Conference on Learning Representations, ICLR 2019, 2019

    work page 2019

  14. [14]

    E. A. Villarrubia-Martin, L. Rodriguez-Benitez, D. Muñoz-Valero, G. Montana, L. Jimenez-Linares, Dynamic Pricing in High-Speed Railways Using Multi-Agent Reinforcement Learning, arXiv preprint arXiv:2501.08234 (2025)

    work page arXiv 2025

  15. [15]

    S. V. Albrecht, F. Christianos, L. Schäfer, Multi-Agent Reinforcement Learning: Foundations and Modern Approaches, MIT Press, 2024. URLhttps://www.marl-book.com

    work page 2024

  16. [16]

    D. Pang, J. Zhu, H. Meng, J. Cao, S. Liu, Data-driven optimal bipartite containment control for heterogeneous multi-agent systems over coopeti- tion networks, Journal of the Franklin Institute 362 (16) (2025) 108057. doi:10.1016/J.JFRANKLIN.2025.108057

    work page doi:10.1016/j.jfranklin.2025.108057 2025

  17. [17]

    C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, Y. Wu, The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games, in: Advances in Neural Information Processing Systems, Vol. 35, 2022

    work page 2022

  18. [18]

    Iqbal, F

    S. Iqbal, F. Sha, Actor-attention-critic for multi-agent reinforcement learning, in: 36th International Conference on Machine Learning, ICML 2019, Vol. 2019-June, 2019, pp. 5261–5270

    work page 2019

  19. [19]

    L. Hou, Y. Li, J. Yan, C. Wang, L. Wang, B. Wang, Multi-agent re- inforcement mechanism design for dynamic pricing-based demand re- sponse in charging network, International Journal of Electrical Power and Energy Systems 147 (2023). doi:10.1016/j.ijepes.2022.108843. 42

    work page doi:10.1016/j.ijepes.2022.108843 2023

  20. [20]

    Fraija, N

    A. Fraija, N. Henao, K. Agbossou, S. Kelouwani, M. Fournier, S. Na- garsheth, Deep reinforcement learning based dynamic pricing for de- mand response considering market and supply constraints, Smart En- ergy 14 (2024). doi:10.1016/j.segy.2024.100139

    work page doi:10.1016/j.segy.2024.100139 2024

  21. [21]

    G. Sun, G. Boateng, L. Luo, H. Chen, D. Mensah, G. Liu, Competitive Pricing for Resource Trading in Sliced Mobile Networks: A Multi-Agent Reinforcement Learning Approach, IEEE Transactions on Mobile Com- puting (2023). doi:10.1109/TMC.2023.3281203

    work page doi:10.1109/tmc.2023.3281203 2023

  22. [22]

    H. Ma, H. Zhang, D. Tian, D. Yue, G. P. Hancke, Optimal demand response based dynamic pricing strategy via Multi-Agent Federated Twin Delayed Deep Deterministic policy gradient algo- rithm, Engineering Applications of Artificial Intelligence 133 (2024). doi:10.1016/j.engappai.2024.108012

    work page doi:10.1016/j.engappai.2024.108012 2024

  23. [23]

    T. N. Kipf, M. Welling, Semi-supervised classification with graph con- volutional networks, in: 5th International Conference on Learning Rep- resentations, ICLR 2017 - Conference Track Proceedings, 2017

    work page 2017

  24. [24]

    Y. Liu, W. Wang, Y. Hu, J. Hao, X. Chen, Y. Gao, Multi- Agent game abstraction via graph attention neural network, in: AAAI 2020 - 34th AAAI Conference on Artificial Intelligence, 2020. doi:10.1609/aaai.v34i05.6211

    work page doi:10.1609/aaai.v34i05.6211 2020

  25. [25]

    Hamilton, Z

    W. Hamilton, Z. Ying, J. Leskovec, Inductive representation learning on large graphs, in: Advances in neural information processing systems, Vol. 30, 2017

    work page 2017

  26. [26]

    J. Yoon, K. Ahn, J. Park, H. Yeo, Transferable traffic signal con- trol: Reinforcement learning with graph centric state representation, Transportation Research Part C: Emerging Technologies 130 (2021). doi:10.1016/j.trc.2021.103321

    work page doi:10.1016/j.trc.2021.103321 2021

  27. [27]

    S.Yang, Hierarchicalgraphmulti-agentreinforcementlearningfortraffic signal control, Information Sciences 634 (2023) 55–72

    work page 2023

  28. [28]

    Z. Pang, X. Li, L. Gao, X. Zhang, J. Wu, Multi-relational graph reinforcement learning for dynamic flexible job-shop schedul- ing under machine breakdowns, Applied Soft Computing 190 (2026). doi:10.1016/j.asoc.2026.114587. 43

    work page doi:10.1016/j.asoc.2026.114587 2026

  29. [29]

    L. Ning, M. Zhou, Z. Hou, R. Goverde, F.-Y. Wang, H. Dong, Deep De- terministic Policy Gradient for High-Speed Train Trajectory Optimiza- tion, IEEE Transactions on Intelligent Transportation Systems 23 (8) (2022) 11562–11574. doi:10.1109/TITS.2021.3105380

    work page doi:10.1109/tits.2021.3105380 2022

  30. [30]

    P. Yue, Y. Jin, X. Dai, Z. Feng, D. Cui, Reinforcement Learning for Scalable Train Timetable Rescheduling With Graph Representa- tion, IEEE Transactions on Intelligent Transportation Systems (2024). doi:10.1109/TITS.2023.3344468

    work page doi:10.1109/tits.2023.3344468 2024

  31. [31]

    Flatland-RL : Multi-Agent Reinforcement Learning on Trains

    S. Mohanty, E. Nygren, F. Laurent, M. Schneider, C. Scheller, N. Bhat- tacharya, J. Watson, A. Egli, C. Eichenberger, C. Baumberger, others, Flatland-rl: Multi-agentreinforcementlearningontrains, arXivpreprint arXiv:2012.05893 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  32. [32]

    Mohammadi, Q

    R. Mohammadi, Q. He, A deep reinforcement learning approach for rail renewal and maintenance planning, Reliability Engineering and System Safety 225 (2022). doi:10.1016/j.ress.2022.108615

    work page doi:10.1016/j.ress.2022.108615 2022

  33. [33]

    Arcieri, C

    G. Arcieri, C. Hoelzl, O. Schwery, D. Straub, K. Papakonstantinou, E. Chatzi, POMDP inference and robust solution via deep reinforce- ment learning: an application to railway optimal maintenance, Machine Learning (2024). doi:10.1007/s10994-024-06559-2

    work page doi:10.1007/s10994-024-06559-2 2024

  34. [34]

    W. Feng, W. Dong, S. Zhai, G. Zhang, X. Sun, Y. Ji, A deep reinforce- ment learning method for freight train driving based on domain knowl- edge and mass estimation network, in: ACM International Conference Proceeding Series, 2021, pp. 41–46. doi:10.1145/3468891.3468898

    work page doi:10.1145/3468891.3468898 2021

  35. [35]

    Y. Cui, G. Zhang, W. Dong, X. Sun, W. Yang, Knowledge-based Deep Reinforcement Learning for Train Automatic Stop Control of High- Speed Railway, in: ACM International Conference Proceeding Series, 2020, pp. 31–36. doi:10.1145/3426826.3426833

    work page doi:10.1145/3426826.3426833 2020

  36. [36]

    doi:10.1016/j.engappai.2024.109226

    J.Lin, X.Qiu, M.Li, Trackingintervalcontrolforurbanrailtrainsbased on safe reinforcement learning, Engineering Applications of Artificial Intelligence 137 (2024). doi:10.1016/j.engappai.2024.109226

    work page doi:10.1016/j.engappai.2024.109226 2024

  37. [37]

    J. Li, Z. Chen, Y. Zheng, S. H. Chan, D. Harabor, P. J. Stuckey, H. Ma, S. Koenig, Scalable Rail Planning and Replanning: Winning the 2020 44 FlatlandChallenge, in: 14thInternationalSymposiumonCombinatorial Search, SoCS 2021, 2021, pp. 477–485. doi:10.1609/socs.v12i1.18576

    work page doi:10.1609/socs.v12i1.18576 2020

  38. [38]

    Littman, Markov games as a framework for multi-agent reinforce- ment learning, in: Proceedings of the 11th International Conference on Machine Learning, ICML 1994, 1994, pp

    M. Littman, Markov games as a framework for multi-agent reinforce- ment learning, in: Proceedings of the 11th International Conference on Machine Learning, ICML 1994, 1994, pp. 157–163. doi:10.1016/B978-1- 55860-335-6.50027-1

    work page doi:10.1016/b978-1- 1994

  39. [39]

    F. A. Oliehoek, C. Amato, A Concise Introduction to Decentralized POMDPs, 1st Edition, Springer Publishing Company, Incorporated, 2016

    work page 2016

  40. [40]

    Relational inductive biases, deep learning, and graph networks

    P.W.Battaglia, J.B.Hamrick, V.Bapst, A.Sanchez-Gonzalez, V.Zam- baldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, others, Relational inductive biases, deep learning, and graph networks, arXiv preprint arXiv:1806.01261 (2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  41. [41]

    Ugadiarov, V

    L. Ugadiarov, V. Vorobyov, A. Panov, Relational Object-Centric Actor- Critic, in: Proceedings of Machine Learning Research, Vol. 275, 2025, p. 1450 – 1476

    work page 2025

  42. [42]

    Schlichtkrull, T

    M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, M.Welling, Modelingrelationaldatawithgraphconvolutionalnetworks, in: European semantic web conference, Springer, 2018, pp. 593–607

    work page 2018

  43. [43]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recog- nition, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2016-December, 2016. doi:10.1109/CVPR.2016.90

    work page doi:10.1109/cvpr.2016.90 2016

  44. [44]

    J. L. Ba, J. R. Kiros, G. E. Hinton, Layer normalization, arXiv preprint arXiv:1607.06450 (2016)

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  45. [45]

    Zaheer, S

    M. Zaheer, S. Kottur, S. Ravanbhakhsh, B. Póczos, R. Salakhutdinov, A. J. Smola, Deep sets, in: Advances in Neural Information Processing Systems, Vol. 2017-December, 2017

    work page 2017

  46. [46]

    Fujimoto, H

    S. Fujimoto, H. Van Hoof, D. Meger, Addressing Function Approxima- tion Error in Actor-Critic Methods, in: 35th International Conference on Machine Learning, ICML 2018, Vol. 4, 2018, pp. 1587–1596. 45

    work page 2018

  47. [47]

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Sil- ver, D. Wierstra, Continuous control with deep reinforcement learning, in: 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings, 2016

    work page 2016

  48. [48]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, in: 35th International Conference on Machine Learning, ICML 2018, Vol. 5, 2018, pp. 2976–2989

    work page 2018

  49. [49]

    Agarwal, M

    R. Agarwal, M. Schwarzer, P. S. Castro, A. Courville, M. G. Bellemare, Deep reinforcement learning at the edge of the statistical precipice, in: Advances in Neural Information Processing Systems, Vol. 35, 2021

    work page 2021

  50. [50]

    Q. Li, Z. Han, X. M. Wu, Deeper insights into graph convolutional networks for semi-supervised learning, in: 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, 2018. doi:10.1609/aaai.v32i1.11604

    work page doi:10.1609/aaai.v32i1.11604 2018

  51. [51]

    Van Der Maaten, G

    L. Van Der Maaten, G. Hinton, Visualizing data using t-SNE, Journal of Machine Learning Research 9 (2008)

    work page 2008

  52. [52]

    E.Rossi, B.Chamberlain, F.Frasca, D.Eynard, F.Monti, M.Bronstein, Temporal graph networks for deep learning on dynamic graphs, arXiv preprint arXiv:2006.10637 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2006

  53. [53]

    Kingma, J

    D. Kingma, J. Ba, Adam: A method for stochastic optimization, in: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 2015

    work page 2015

  54. [54]

    G. E. Uhlenbeck, L. S. Ornstein, On the theory of the Brownian motion, Physical Review 36 (5) (1930). doi:10.1103/PhysRev.36.823. 46

    work page doi:10.1103/physrev.36.823 1930