Pith. sign in

REVIEW 5 major objections 5 minor 69 references

Constrained Optimization of Charged Particle Tracking with Multi-Agent Reinforcement Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that collaborative multi-agent reinforcement learning with a unique-hit safety layer and cost-margin gradients reconstructs particle tracks in simulated proton-CT detector data better than unconstrained MARL and…

desk verdict Solid incremental MARL tracking paper with a genuinely useful cost-margin trick; the headline numbers are upper bounds due to ground-truth seeding, but the authors are honest about it and the work is reproducible. read the letter →

arxiv 2501.05113 v1 pith:TPHNB2I3 submitted 2025-01-09 physics.comp-ph cs.AIcs.LG

Tobias Kortus , Ralf Keidel , Nicolas R. Gauger , Jan Kieseler (for the Bergen pCT Collaboration) This is my paper · ORCID
classification physics.comp-phcs.AIcs.LG
keywords multi-agentreinforcementlearningparticletrackinglinearsumassignmentproblemsafetylayerblackboxdifferentiationcostmarginsprotoncomputedtomographyMATD3
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that charged-particle tracking—deciding which detector hits belong to which proton—can be posed as a multi-agent reinforcement learning problem with assignment constraints, and that enforcing those constraints during training is what makes the learned tracker accurate. On simulated readouts from a pixelated tracking calorimeter for proton computed tomography, a MATD3-trained team of agents whose joint actions pass through a linear-assignment safety layer, plus an extra cost-margin gradient, reaches about 96.3% purity and 84.0% efficiency at 50 primary particles per frame. That clearly beats an unconstrained MAPPO baseline and matches or slightly exceeds single-agent RL combined with the same safety layer. This matters because the assignment problem becomes combinatorially harder as particle density rises, and conventional and unconstrained-learned trackers both lose accuracy there. If correct, the result offers an end-to-end trainable, constraint-aware alternative for particle reconstruction that does not need labeled tracks during training.

What carries the argument

The central object is a decentralized partially observable Markov decision process whose environment is a directed acyclic hit graph: each vertex is a detector hit, each directed edge connects hits in adjacent layers, and agents choose edges through a pointer-network policy. The constraint is enforced by a centralized safety layer that solves a linear sum assignment problem at every joint action, projecting the agents' policy scores onto a feasible unique assignment with costs $c_{ij} = \|\mu_i(a_j|o) - \mathbf{1}(a_j)\|_2^2$ for unseeded hits and $\infty$ for seeded ones. Because the solver output is piecewise constant, training uses blackbox differentiation $\nabla^{\mathrm{BB}}_C f_\lambda(\hat{C}) = -\frac{1}{\lambda}[y(\hat{C}) - y_\lambda(C')]$, augmented by the proposed cost-margin term $\nabla^{\leftrightarrow}_C f(\hat{C}) = y(\hat{C})$, which pushes predictions toward lower assignment costs and away from decision boundaries. A factored centralized critic with self-attention stabilizes off-policy MATD3 training.

What would settle it

Run the same MATD3+LSA(BB↔ν) policy on the same simulated readouts but with seeds produced by a realistic seed-finding algorithm instead of ground truth, and measure purity and efficiency of the full pipeline; if the constrained multi-agent method no longer exceeds PPO+LSA or the track follower, the central claim is falsified at pipeline level. A cheaper check is to inject noise into the initial segments and compare the drop in performance across methods.

Watch

Extended reading notes

Core claim

The central claim is that constrained multi-agent reinforcement learning is the right formulation for RL-based particle tracking: decentralized agents propose next-hit assignments, and a centralized safety layer resolves the joint proposal by solving a linear sum assignment problem, guaranteeing each hit is used at most once while keeping the projection differentiable through blackbox differentiation. The paper's added mechanism is a cost-margin gradient term that explicitly pushes the policy's predicted assignment costs away from decision boundaries. On simulated detector data, this combination reaches about 96.3% purity and 84.0% efficiency at 50 primary particles per frame with a 100 mm water phantom, compared with 80.1% and 70.3% for MAPPO and 56.6% and 48.6% for the constrained multi-agent scheme without cost margins. At 200 particles per frame, the constrained multi-agent method exceeds the post-hoc constrained single-agent baseline by 0.75 percentage points in purity (p = 0.03) and 1.12 percentage points in efficiency (p = 0.02). The paper further argues that the safety layer is necessary: unconstrained multi-agent training often converges to high team reward while producing many incorrect tracks.

Load-bearing premise

The load-bearing premise is that each tracker starts every candidate track from the true initial segment (ground-truth seeding), which the paper itself calls a performance upper bound for RL-based tracking; with imperfect seeds from a real detector, the reported margins could shrink or disappear.

Editorial extensions

If this is right

  • The uniqueness safety layer makes duplicate-hit assignments impossible at both training and inference, removing by construction the combinatorial problem of assigning the same hit to multiple tracks.
  • The cost-margin gradient widens the plateau of high reconstruction performance in the policy landscape, so the learned tracker is less sensitive to weight perturbations and random restarts.
  • At high particle multiplicity (200 primary particles per frame), the constrained multi-agent policy outperforms even the post-hoc constrained single-agent tracker, suggesting the collaborative formulation helps exactly where assignment conflicts are hardest.
  • Because rewards are per-transition scatter angles rather than full-track rewards, the method is compatible with off-policy replay and can be trained end-to-end without labeled truth tracks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the ground-truth seeding assumption were relaxed, the measured advantage could shrink, since all compared methods are evaluated from perfect initial segments; testing with a learned seed finder would show how much of the gap is intrinsic to the assignment policy rather than to seed quality.
  • Editorial inference: the same safety-layer-plus-cost-margin design should apply to other unique-assignment reconstruction problems, such as vertex finding, matching in high-pileup collisions, or multi-object tracking, because those problems share the same assignment-polytope structure.
  • Editorial inference: the reward-surface analysis implies a falsifiable prediction—policies trained with cost margins should fine-tune more gracefully to detector changes such as sensor aging, because their parameters sit in wide, flat optima.
  • Editorial inference: the approach's constraint expressiveness is limited by the linear assignment polytope; constraints that couple tracks globally or over time would require a different projection layer, and the reported gains may not carry over.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a multi-agent reinforcement learning (MARL) approach for charged-particle tracking in a pixelated detector, building on a prior single-agent RL tracker. The method uses a centralized safety layer that solves a linear sum assignment problem at every step to enforce unique assignment of hits, a factored centralized critic, and an extension of blackbox combinatorial differentiation with an additional cost-margin gradient term. The authors compare several variants, including MATD3 with the safety layer and margin gradients, against a conventional track follower, single-agent PPO, PPO with the safety layer, MAPPO, and MATD3 without margin gradients. On simulated Bergen pCT DTC data, they report that MATD3+LSA with margin gradients achieves roughly 96–97% purity and 84–87% efficiency for water phantoms of 100–200 mm and particle densities of 50–200 p+/F, outperforming the baselines, with a statistically significant advantage over PPO+LSA at 200 p+/F. They also analyze policy entropy, reward surfaces, and prediction instabilities to support the claim that cost margins improve optimization and generalization.

Significance. If the claims hold, the paper makes a useful methodological contribution: it demonstrates a practical way to combine centralized safety layers, multi-agent off-policy training, and blackbox differentiation for a discrete assignment problem in physics reconstruction. The safety-layer formulation and the cost-margin gradient are generally applicable ideas beyond this specific tracking application. The empirical study is relatively extensive, with several baselines, multiple phantom configurations, confidence intervals, and a public code/data release, which are strengths. The paper is also transparent in stating that ground-truth seeding makes the reported reconstruction performance an upper bound. However, the central advantage claims rest on a few load-bearing points that are either confounded or not fully controlled, in particular the ground-truth seeding oracle and the change of a regularization hyperparameter between the compared margin and no-margin variants. These issues need to be addressed before the main claims can be accepted as stated.

major comments (5)
  1. [§IV-A-b, Tables II/III] The paper explicitly states in Section IV-A-b that ground-truth seeding 'provid[es] a performance upper bound of RL-based tracking,' and every compared tracker is initialized from true track segments. Therefore the purity and efficiency values in Tables II and III measure the downstream assignment policy under oracle initialization, not full reconstruction from detector readouts. This is not merely a scope caveat: the infinite-cost mask in Eq. (4) is exact only because of oracle seeds, and a realistic seed finder would feed incorrect, missing, or duplicated seeds into the safety layer. The abstract and conclusion present the method as reconstructing particle tracks without this qualification. Please either add a heuristic-seed robustness check, or rewrite the central claims as applying to the downstream assignment stage given known seeds.
  2. [Table I, §IV-B-g, Eq. (11)] The comparison between MATD3+LSA(BB) and MATD3+LSA(BB↔ν) changes two things at once: the presence of the cost-margin gradient term and the value of the regularization/smoothing weight γ in Eq. (11). Table I lists γ=0.75 for MATD3+LSA(BB) and γ=0.25 for MATD3+LSA(BB↔ν). Since γ controls the strength of the cross-state regularization term in Eq. (11), the entropy reduction and performance improvements attributed to cost margins in Section V-B and Figures 5–6 could be partly due to the change in γ. Please provide a matched-γ ablation, or otherwise demonstrate that the margin gradient alone is responsible for the observed improvement.
  3. [§V-A, section on off-policy optimization] The paper states that no stable MATD3 configuration without a safety layer was found and therefore excludes it from the experiments. As a result, the advantage of constrained over unconstrained MARL is not measured within the same off-policy algorithm family; the comparison against MAPPO varies both the constraint mechanism and the optimization algorithm (on-policy stochastic vs. off-policy deterministic). The claim that unconstrained approaches 'fail to converge consistently' is thus based on a single on-policy baseline. A best-effort unconstrained MATD3 baseline, or a more careful discussion of why its failure is attributable to the absence of constraints rather than to off-policy instability, is needed to support the conclusion that the safety layer is the decisive factor.
  4. [§V-A, Tables II/III] The confidence intervals for MAPPO and MATD3+LSA(BB) in Table II are extremely wide (roughly ±15 to ±25 percentage points), indicating large variance across the five random seeds. In several rows the intervals overlap with those of MATD3+LSA(BB↔ν), so the statement that the proposed method 'outperforms all baseline and MARL variants by a significant margin' is not supported for these baselines. The only formal significance test reported is against PPO+LSA at 200 p+/F (p=0.03 for purity, p=0.02 for efficiency). Please report per-seed results and pairwise statistical tests across all configurations, or soften the blanket performance claim.
  5. [§IV-B-f, Eq. (7)] The cost-margin gradient term ∇↔_C f(Ĉ) = y(Ĉ) is introduced heuristically without derivation. The statement that it 'forces the assignments of the joint policy µ in the direction of lower assignment costs' is not immediate, because y(Ĉ) is the discrete solution of an LSAP and does not carry an obvious directional interpretation for the continuous policy parameters. Given that this term is one of the main contributions, please provide a derivation or intuitive explanation, and ideally a controlled ablation with matched γ, to isolate its effect.
minor comments (5)
  1. [Eqs. (5)–(6)] The definition of C' in Eq. (6) uses the clip operation with lower and upper bounds (0, ∞); it would be clearer to state explicitly that the clipping is applied element-wise to the cost matrix, and to define the notation dL/dy at the discrete solution.
  2. [Eq. (7)] The symbol ∇↔ is not standard and is not defined formally. Please define the operator before using it in Eq. (7).
  3. [§V-D, Eq. (18)] The prediction instability measure is defined for classifiers and then adapted to the sequential RL setting by evaluating only on manually constructed correctly assigned states. This adaptation should be described more explicitly, in particular how the disagreement is aggregated over the sequential decisions.
  4. [§V-A-a] The text says that PPO and PPO+LSA results use models from [9], but it is not specified whether the same train/test split, seeding procedure, and evaluation metrics were used exactly as for the new MATD3 variants; please state this explicitly.
  5. [Table I] The table caption lists column abbreviations (SL(T), SL(E), SL-grad.) but the caption does not define them; the definitions appear in the text but including them in the caption would improve readability.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the claimed tracking gains are measured against external baselines and an external perfect-matching metric; the main caveat (ground-truth seeding) is an acknowledged upper-bound limitation, not a fitted prediction.

full rationale

The paper's optimization target (negative average scatter angle reward) is not the same as the reported metrics (track purity and efficiency under a perfect-matching criterion), so the headline numbers are not encoded into the objective by construction. MATD3+LSA, MAPPO, PPO, and PPO+LSA are compared on simulated readout frames, and the PPO/PPO+LSA baselines are prior published models from [9] rather than quantities fitted in this paper. The safety layer (Eq. 4) and blackbox/cost-margin gradients (Eqs. 5-7) are training and architecture components; their effect on purity and efficiency is established empirically, not by definition. The paper explicitly discloses in Section IV-A-b that track candidates start from ground-truth seeds, calling this 'a performance upper bound of RL-based tracking'; this is an evaluative oracle that limits the real-world claim but does not make the downstream assignment result equal to its input. Self-citations of [9] for graph parametrization, PE-ARF, feature normalization, and the seeding convention are inheritance of an independently published framework, not load-bearing circular justification: no uniqueness theorem from [9] is invoked to forbid alternatives, and the central comparison with external baselines remains self-contained. The only reason for a nonzero score is the presence of minor, non-load-bearing self-citation; no specific circular step satisfies the requirement of being exhibited as an equation-level reduction or a fitted parameter renamed as prediction.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central empirical claim rests on simulated data fidelity, ground-truth seeding, and several hand-set hyperparameters; the margin-gradient component is heuristic.

free parameters (3)
  • nu (cost-margin weight) = 0.01 and 0.1
    Hand-selected hyperparameter controlling the extra gradient term in Eq (7); authors test two values and find similar results.
  • lambda (blackbox differentiation interpolant) = not reported
    Tunable hyperparameter in Eq (5) controlling truthfulness versus informativeness of gradients; value not given in the text.
  • gamma (regularization/smoothing weight) = 0.75 (MATD3+LSA BB) and 0.25 (MATD3+LSA BB<->nu)
    From Table I; hand-set hyperparameter in the centralized critic's cross-state regularization in Eq (11).
assumptions (6)
  • standard math The linear sum assignment problem in Eq (3) is solved exactly at every joint action, and the resulting projection yields a valid constrained policy.
    The safety layer relies on the Hungarian/LSAP solver; the paper does not prove properties of the projected policy beyond uniqueness.
  • domain assumption GATE/Geant4 simulations of the Bergen pCT detector accurately model particle interactions relevant for the study.
    All results are on simulated readouts; fidelity of the simulation to the real detector is assumed.
  • domain assumption Negative average scatter angle is an appropriate reward proxy for reconstruction quality.
    The team reward is the negative mean transition scattering angle, simplifying the Highland model used in [9]; the paper relies on this proxy aligning with purity and efficiency.
  • domain assumption Ground-truth seeding is available for all algorithms, providing an initial partial track segment.
    Stated in Section IV-A b; this is an upper-bound assumption and a load-bearing condition for the reported absolute performance.
  • ad hoc to paper The extra gradient term in Eq (7) provides a useful learning signal for the policy.
    The term is proposed as a heuristic to enforce cost margins; no formal convergence or bias analysis is offered.
  • domain assumption Blackbox differentiation from [13] (Eqs 5 and 6) gives usable gradient information for the LSAP solver.
    The method is taken from Vlastelica et al. without re-derivation; the paper's training depends on it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Constrained Optimization of Charged Particle Tracking with Multi-Agent Reinforcement Learning." pith.science (2026). https://pith.science/paper/TPHNB2I3

@misc{pith2026250105113,
  author       = {Pith},
  title        = {Pith review of: Constrained Optimization of Charged Particle Tracking with Multi-Agent Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TPHNB2I3}},
  note         = {Machine review of arXiv:2501.05113}
}
read the original abstract

Reinforcement learning demonstrated immense success in modelling complex physics-driven systems, providing end-to-end trainable solutions by interacting with a simulated or real environment, maximizing a scalar reward signal. In this work, we propose, building upon previous work, a multi-agent reinforcement learning approach with assignment constraints for reconstructing particle tracks in pixelated particle detectors. Our approach optimizes collaboratively a parametrized policy, functioning as a heuristic to a multidimensional assignment problem, by jointly minimizing the total amount of particle scattering over the reconstructed tracks in a readout frame. To satisfy constraints, guaranteeing a unique assignment of particle hits, we propose a safety layer solving a linear assignment problem for every joint action. Further, to enforce cost margins, increasing the distance of the local policies predictions to the decision boundaries of the optimizer mappings, we recommend the use of an additional component in the blackbox gradient estimation, forcing the policy to solutions with lower total assignment costs. We empirically show on simulated data, generated for a particle detector developed for proton imaging, the effectiveness of our approach, compared to multiple single- and multi-agent baselines. We further demonstrate the effectiveness of constraints with cost margins for both optimization and generalization, introduced by wider regions with high reconstruction performance as well as reduced predictive instabilities. Our results form the basis for further developments in RL-based tracking, offering both enhanced performance with constrained policies and greater flexibility in optimizing tracking algorithms through the option for individual and team rewards.

Figures

Figures reproduced from arXiv: 2501.05113 by the authors.

Figure 1
Figure 1. General description of charged particle tracking framework for single- or multi-agent reinforcement learning. The agent (right) learns by iterated [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Interaction loop between environment description containing particle readouts in the form of a directed acyclic graph based on [9]. The agent (network [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Average return obtained by the agents over time during training, plotted as a function of performed updates (for MAPPO: iteration over all epochs [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Particle tracks generated using MATD3+LSA (BB [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Distributions of the uncertainties in local policy predictions, measured as the predictive entropy for various water phantoms and particle densities. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Two-dimensional reward and performance surfaces of multi-agent framework with (MATD3+LSA(BB [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Two-dimensional reward and performance (purity and efficiency) surfaces of multi-agent framework with (MATD3+LSA(BB [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Prediction instabilities of trained reconstruction policies generated for [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Distributions of prediction instabilities on a readout frame level [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 53 canonical work pages

  1. [1]

    Playing Atari with Deep Reinforcement Learning,

    V . Mnih et al., “Playing Atari with Deep Reinforcement Learning,” pp. 1–9, 2013. [Online]. Available: http://arxiv.org/abs/1312.5602

    arXiv 2013

  2. [2]

    A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play,

    D. Silver et al., “A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play,” Science, vol. 362, no. 6419, pp. 1140–1144, 2018

    work page 2018

  3. [3]

    Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,

    S. Gu et al., “Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates,” Proceedings - IEEE International Conference on Robotics and Automation , pp. 3389–3396, 2017

    work page 2017

  4. [4]

    Learning dexterous in-hand manipu- lation,

    O. A. M. Andrychowicz et al. , “Learning dexterous in-hand manipu- lation,” International Journal of Robotics Research , vol. 39, no. 1, pp. 3–20, 2020

    work page 2020

  5. [5]

    Learning to drive in a day,

    A. Kendall et al. , “Learning to drive in a day,” Proceedings - IEEE International Conference on Robotics and Automation , vol. 2019-May, pp. 8248–8254, 2019

    work page 2019

  6. [6]

    Magnetic control of tokamak plasmas through deep reinforcement learning,

    J. Degrave et al., “Magnetic control of tokamak plasmas through deep reinforcement learning,” Nature, vol. 602, no. 7897, pp. 414–419, 2022

    work page 2022

  7. [7]

    Sample-efficient reinforcement learning for CERN accelerator control,

    V . Kain et al. , “Sample-efficient reinforcement learning for CERN accelerator control,” Physical Review Accelerators and Beams , vol. 23, no. 12, p. 124801, 2020

    work page 2020

  8. [8]

    Reinforcement learning for charged-particle tracking Reinforcement learning,

    L. H. V ˚age, “Reinforcement learning for charged-particle tracking Reinforcement learning,” Proceedings of the CTD 2022 , 2022

    work page 2022

Show all 69 references
  1. [9]

    Towards Neural Charged Particle Tracking in Digital Tracking Calorimeters with Reinforcement Learning,

    T. Kortus et al., “Towards Neural Charged Particle Tracking in Digital Tracking Calorimeters with Reinforcement Learning,” IEEE Transac- tions on Pattern Analysis and Machine Intelligence , vol. 45, no. 12, pp. 15 820–15 833, 2023

    2023

  2. [10]

    R. S. Sutton et al. , Reinforcement Learning: An Introduction . Cam- bridge, MA, USA: A Bradford Book, 2018

    2018

  3. [11]

    Markov games as a framework for multi-agent rein- forcement learning,

    M. L. Littman, “Markov games as a framework for multi-agent rein- forcement learning,” Machine Learning Proceedings 1994, pp. 157–163, 1994

    1994

  4. [12]

    Learning tsp requires rethinking generalization,

    C. K. Joshi et al., “Learning tsp requires rethinking generalization,” pp. 33:1–33:0, 2021

    2021

  5. [13]

    Differentiation of Blackbox Combinatorial Solvers,

    M. Vlastelica et al. , “Differentiation of Blackbox Combinatorial Solvers,” 8th International Conference on Learning Representations, ICLR 2020, pp. 1–19, 2020

    2020

  6. [14]

    Proton tracking algorithm in a pixel-based range telescope for proton computed tomography,

    H. E. Pettersen et al., “Proton tracking algorithm in a pixel-based range telescope for proton computed tomography,” arXiv, 2020

    2020

  7. [15]

    Cliff diving: Exploring reward surfaces in reinforce- ment learning environments,

    R. Sullivan et al., “Cliff diving: Exploring reward surfaces in reinforce- ment learning environments,” in Proceedings of the 39th International Conference on Machine Learning . PMLR, 2022, pp. 20 744–20 776, ISSN: 2640-3498

    2022

  8. [16]

    Launch and iterate: Reducing prediction churn,

    M. M. Fard et al. , “Launch and iterate: Reducing prediction churn,” Advances in Neural Information Processing Systems , vol. 29, 2016

    2016

  9. [17]

    A High-Granularity Digital Tracking Calorimeter Optimized for Proton CT,

    J. Alme et al. , “A High-Granularity Digital Tracking Calorimeter Optimized for Proton CT,” Frontiers in Physics , vol. 8, no. October, pp. 1–20, 2020

    2020

  10. [18]

    The bergen proton CT system,

    M. Aehle et al. , “The bergen proton CT system,” Journal of Instrumentation, vol. 18, no. 2, p. C02051, 2023. [Online]. Available: https://iopscience.iop.org/article/10.1088/1748-0221/18/02/C02051

    2023 doi

  11. [19]

    ALPIDE, the Monolithic Active Pixel Sensor for the ALICE ITS upgrade,

    M. Mager, “ALPIDE, the Monolithic Active Pixel Sensor for the ALICE ITS upgrade,” Nuclear Instruments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, vol. 824, no. 2016, pp. 434–438, 2016. [Online]. Available: http:/...

    2016 doi

  12. [20]

    The ALPIDE pixel sensor chip for the upgrade of the ALICE Inner Tracking System,

    G. Aglieri Rinella, “The ALPIDE pixel sensor chip for the upgrade of the ALICE Inner Tracking System,” Nuclear Instruments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment , vol. 845, pp. 583–587, 2017. [Online]. Avail...

    2017 doi

  13. [21]

    Passage of particles through matter,

    D. Groom et al. , “Passage of particles through matter,” European Physical Journal C — EUR PHYS J C , vol. 15, pp. 163–173, 2000

    2000

  14. [22]

    Radiotherapy Proton Interactions in Matter,

    B. Gottschalk, “Radiotherapy Proton Interactions in Matter,” arXiv, 2018

    2018

  15. [23]

    Application of Kalman filtering to track and vertex fitting,

    R. Fr ¨uhwirth, “Application of Kalman filtering to track and vertex fitting,” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment , vol. 262, no. 2, pp. 444–450, 1987

    1987

  16. [24]

    A concurrent track evolution algorithm for pattern recog- nition in the HERA-B main tracking system,

    R. Mankel, “A concurrent track evolution algorithm for pattern recog- nition in the HERA-B main tracking system,” Nuclear Instruments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, vol. 395, no. 2, pp. 169–184, 1997

    1997

  17. [25]

    Tracking elementary particles near their primary vertex: A combinatorial approach,

    J. F. Pusztaszeri et al., “Tracking elementary particles near their primary vertex: A combinatorial approach,” Journal of Global Optimization , vol. 9, pp. 41–64, 1996

    1996

  18. [26]

    Object condensation: one-stage grid-free multi-object reconstruction in physics detectors, graph, and image data,

    J. Kieseler, “Object condensation: one-stage grid-free multi-object reconstruction in physics detectors, graph, and image data,” European Physical Journal C , vol. 80, pp. 1–12, 2020. [Online]. Available: https://doi.org/10.1140/epjc/s10052-020-08461-2

    2020 doi

  19. [27]

    High pileup particle tracking with object condensation,

    K. Lieret et al. , “High pileup particle tracking with object condensation,” p. 2023, 12 2023. [Online]. Available: https://arxiv.org/ abs/2312.03823v1

    2023 arXiv

  20. [28]

    Charged particle tracking via edge-classifying interaction networks,

    G. DeZoort et al. , “Charged particle tracking via edge-classifying interaction networks,” Computing and Software for Big Science , vol. 5, pp. 1–13, 2021. [Online]. Available: https://doi.org/10.1007/ s41781-021-00073-z

    2021

  21. [29]

    Exploring end-to-end differentiable neural charged particle tracking - a loss landscape perspective,

    T. Kortus et al. , “Exploring end-to-end differentiable neural charged particle tracking - a loss landscape perspective,” arXiv:2407.13420 [physics.comp-ph], 2024. [Online]. Available: https://arxiv.org/abs/2407. 13420

    2024

  22. [30]

    OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World,

    T. H. Pham et al. , “OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World,” Proceedings - IEEE International Conference on Robotics and Automation , pp. 6236–6243, 2018

    2018

  23. [31]

    Safe Exploration in Continuous Action Spaces,

    G. Dalal et al., “Safe Exploration in Continuous Action Spaces,” 2018. [Online]. Available: http://arxiv.org/abs/1801.08757

    2018 arXiv

  24. [32]

    Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spaces,

    Z. Sheebaelhamd et al. , “Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spaces,” 2021. [Online]. Available: http://arxiv.org/abs/2108.03952

    2021 arXiv

  25. [33]

    Safe multi-agent reinforcement learning via shielding,

    I. ElSayed-Aly et al. , “Safe multi-agent reinforcement learning via shielding,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS , vol. 1, pp. 483– 491, 1 2021. [Online]. Available: https://arxiv.org/abs/2101.11196v2

    2021 arXiv

  26. [34]

    Safe reinforcement learning via shielding,

    M. Alshiekh et al. , “Safe reinforcement learning via shielding,” 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 , pp. 2669–2678, 8 2017. [Online]. Available: https://arxiv.org/abs/1708.08611v2 PRE-PRINT (2025-JAN-09) / THIS PAPER IS CURRENTLY UNDER REVIEW 12

    2018 arXiv

  27. [35]

    Optimal and approximate Q-value functions for decentralized POMDPs,

    F. A. Oliehoek et al. , “Optimal and approximate Q-value functions for decentralized POMDPs,” Journal of Artificial Intelligence Research , vol. 32, pp. 289–353, 2008

    2008

  28. [36]

    Policy iteration for decentralized control of markov decision processes,

    D. S. Bernstein et al. , “Policy iteration for decentralized control of markov decision processes,” Journal of Artificial Intelligence Research , vol. 34, pp. 89–132, 2009

    2009

  29. [37]

    Some practical remarks on multiple scattering,

    V . L. Highland, “Some practical remarks on multiple scattering,”Nuclear Instruments and Methods , vol. 129, no. 2, pp. 497–499, 1975

    1975

  30. [38]

    Backpropagation through combinatorial algorithms: Identity with projection works,

    S. Sahoo et al. , “Backpropagation through combinatorial algorithms: Identity with projection works,” in Proceedings of the Eleventh Interna- tional Conference on Learning Representations , May 2023

    2023

  31. [39]

    Pointer networks,

    O. Vinyals et al. , “Pointer networks,” Advances in Neural Information Processing Systems, vol. 2015-Janua, pp. 2692–2700, 2015

    2015

  32. [40]

    Neural machine translation by jointly learning to align and translate,

    D. Bahdanau et al. , “Neural machine translation by jointly learning to align and translate,” in 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings , 2015, pp. 1–15

    2015

  33. [41]

    Noisy networks for exploration,

    M. Fortunato et al., “Noisy networks for exploration,” 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, pp. 1–21, 2018

    2018

  34. [42]

    Parameter space noise for exploration,

    M. Plappert et al., “Parameter space noise for exploration,” 6th Interna- tional Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, pp. 1–18, 2018

    2018

  35. [43]

    Multi-Agent Reinforcement Learning: Independent vs. Cooper- ative Agents,

    M. Tan, “Multi-Agent Reinforcement Learning: Independent vs. Cooper- ative Agents,” Machine Learning Proceedings 1993, pp. 330–337, 1993

    1993

  36. [44]

    Value-decomposition networks for cooperative multi- agent learning based on team reward,

    P. Sunehag et al., “Value-decomposition networks for cooperative multi- agent learning based on team reward,” Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AA- MAS, vol. 3, pp. 2085–2087, 2018

    2018

  37. [45]

    Actor-attention-critic for multi-agent reinforcement learning,

    S. Iqbal et al. , “Actor-attention-critic for multi-agent reinforcement learning,” 36th International Conference on Machine Learning, ICML 2019, vol. 2019-June, pp. 5261–5270, 2019

    2019

  38. [46]

    Layer normalization,

    J. L. Ba et al. , “Layer normalization,” 2016. [Online]. Available: http://arxiv.org/abs/1607.06450

    2016 arXiv

  39. [47]

    Deep residual learning for image recognition,

    K. He et al., “Deep residual learning for image recognition,”Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-Decem, pp. 770–778, 2016

    2016

  40. [48]

    On the use and misuse of absorbing states in multi-agent reinforcement learning,

    A. Cohen et al. , “On the use and misuse of absorbing states in multi-agent reinforcement learning,” 11 2021. [Online]. Available: https://arxiv.org/abs/2111.05992v2

    2021 arXiv

  41. [49]

    The surprising effectiveness of ppo in cooperative multi- agent games,

    C. Yu et al., “The surprising effectiveness of ppo in cooperative multi- agent games,” in Advances in Neural Information Processing Systems , S. Koyejo et al. , Eds., vol. 35. Curran Associates, Inc., 2022, pp. 24 611–24 624

    2022

  42. [50]

    Value-decomposition multi-agent proximal policy op- timization,

    Y . Ma et al. , “Value-decomposition multi-agent proximal policy op- timization,” Proceedings - 2022 Chinese Automation Congress, CAC 2022, vol. 2022-January, pp. 3460–3464, 2022

    2022

  43. [51]

    High-dimensional continuous control using gener- alized advantage estimation,

    J. Schulman et al., “High-dimensional continuous control using gener- alized advantage estimation,” 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings , pp. 1–14, 2016

    2016

  44. [52]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap et al. , “Continuous control with deep reinforcement learning,” 4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings , 2016

    2016

  45. [53]

    Multi-agent actor-critic for mixed cooperative- competitive environments,

    R. Lowe et al. , “Multi-agent actor-critic for mixed cooperative- competitive environments,” Advances in Neural Information Processing Systems, vol. 2017-Decem, pp. 6380–6391, 2017

    2017

  46. [54]

    Reducing overestimation bias in multi-agent domains using double centralized critics,

    J. Ackermann et al. , “Reducing overestimation bias in multi-agent domains using double centralized critics,” 10 2019. [Online]. Available: https://arxiv.org/abs/1910.01465v2

    2019 arXiv

  47. [55]

    Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning,

    J. Hu et al. , “Rethinking the Implementation Tricks and Monotonicity Constraint in Cooperative Multi-Agent Reinforcement Learning,” 2021. [Online]. Available: http://arxiv.org/abs/2102.03479

    2021 arXiv

  48. [56]

    Proximal Policy Optimization Algorithms,

    J. Schulman et al. , “Proximal Policy Optimization Algorithms,” pp. 1–12, 2017. [Online]. Available: http://arxiv.org/abs/1707.06347

    2017 arXiv

  49. [57]

    Particle Tracking Data: Bergen DTC Prototype,

    T. Kortus et al., “Particle Tracking Data: Bergen DTC Prototype,” dec

  50. [58]

    GATE -Geant4 Application for Tomographic Emission: a simulation toolkit for PET and SPECT,

    S. Jan et al. , “GATE -Geant4 Application for Tomographic Emission: a simulation toolkit for PET and SPECT,” Phys Med Biol. Phys Med Biol, vol. 49, no. 19, pp. 4543–4561, 2004

    2004

  51. [59]

    GATE V6: A major enhancement of the GATE simula- tion platform enabling modelling of CT and radiotherapy,

    S. Jan et al. , “GATE V6: A major enhancement of the GATE simula- tion platform enabling modelling of CT and radiotherapy,” Physics in Medicine and Biology , vol. 56, no. 4, pp. 881–901, 2011

    2011

  52. [60]

    GEANT4 - A simulation toolkit,

    S. Agostinelli et al. , “GEANT4 - A simulation toolkit,” Nuclear In- struments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, vol. 506, no. 3, pp. 250–303, 2003

    2003

  53. [61]

    Geant4 developments and applications,

    J. Allison et al., “Geant4 developments and applications,” IEEE Trans- actions on Nuclear Science , vol. 53, no. 1, pp. 270–278, 2006

    2006

  54. [62]

    Recent developments in GEANT4,

    J. Allison et al. , “Recent developments in GEANT4,” Nuclear In- struments and Methods in Physics Research, Section A: Accelerators, Spectrometers, Detectors and Associated Equipment , vol. 835, pp. 186– 225, 2016

    2016

  55. [63]

    Investigating particle track topology for range telescopes in particle radiography using convolutional neural networks,

    H. E. S. Pettersen et al., “Investigating particle track topology for range telescopes in particle radiography using convolutional neural networks,” Acta Oncologica, vol. 60, pp. 1413–1418, 2021

    2021

  56. [64]

    The generalisation of student’s problems when several different population variances are involved

    B. L. Welch, “The generalisation of student’s problems when several different population variances are involved.” Biometrika, vol. 34, no. 1-2, pp. 28–35, 1947

    1947

  57. [65]

    Visualizing the loss landscape of neural nets,

    H. Li et al. , “Visualizing the loss landscape of neural nets,” Advances in Neural Information Processing Systems , vol. 2018-Decem, pp. 6389– 6399, 2018

    2018

  58. [66]

    Similarity of neural network models: A survey of functional and representational measures,

    M. Klabunde et al., “Similarity of neural network models: A survey of functional and representational measures,” 5 2023. [Online]. Available: https://arxiv.org/abs/2305.06329v2

    2023 arXiv

  59. [67]

    On the prediction instability of graph neural net- works,

    M. Klabunde et al., “On the prediction instability of graph neural net- works,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13715 LNAI, pp. 187–202, 2023. Tobias Kortus received the ...

    2023

  60. [2021]

    His research interests include machine learning and reinforcement learning, with focus on applications in high energy and medical physics

    He is currently working towards his PhD at the University of Kaiserslautern-Landau. His research interests include machine learning and reinforcement learning, with focus on applications in high energy and medical physics. Ralf Keidel is Senior Professor at the University of A...

  61. [2022]

    Available: https://doi.org/10.5281/zenodo.7426388

    [Online]. Available: https://doi.org/10.5281/zenodo.7426388

    doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.