Pith. sign in

REVIEW 3 major objections 5 minor 28 references

Multi Agent Reinforcement Learning for Sequential Satellite Assignment Problems

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A hybrid of per-agent Q-learning and an optimal assignment solver is proposed for sequential satellite tasking, with reported 20-50% gains over IQL, IPPO, COMA, and HAAL at 324 satellites and 450 tasks.

desk verdict A genuinely useful MARL-assignment architecture with a strong large-scale experiment, but the theoretical justification overreaches: the Lemma 1 fixed-point proof does not survive partial observability, and Equation (4) rests on an unexamined slow-policy assumption. read the letter →

arxiv 2412.15573 v1 pith:KCZB733Y submitted 2024-12-20 cs.MA cs.LG

Joshua Holder , Natasha Jaques , Mehran Mesbahi This is my paper · ORCID
classification cs.MAcs.LG
keywords multi-agentreinforcementlearningsequentialassignmentproblemsatelliteconstellationsQ-functiondecompositionoptimalmechanismtaskallocationpartialobservabilityvalue-basedRL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that sequential assignment problems — where the value of assigning a given agent to a given task depends on the evolving state of the whole system — can be solved by combining per-agent reinforcement learning with a classical optimal-assignment solver. The proposed method, REDA, trains each agent to learn a Q-function estimating the future reward it would earn from each candidate task, then treats the matrix of these learned values as the benefit matrix for the optimal assignment mechanism alpha, which returns a feasible joint assignment that maximizes total value. Because the solver enforces that each agent gets exactly one task and each task at most one agent, REDA avoids the duplicate-assignment conflicts that plague independent MARL methods while continuing to scale to hundreds of agents. On a high-fidelity satellite constellation simulation with 324 satellites and 450 tasks, the authors report that REDA outperforms IQL, IPPO, COMA, and HAAL by 20-50%, and they give a contraction argument intended to show the per-agent value estimates converge to the Q-function of the joint policy.

What carries the argument

The load-bearing pair is (Q^pi_i, $\alpha$). Each agent's learned value function Q^pi_i(o_i, j) estimates the total discounted future reward agent i would obtain if assigned task j and the group then follows the joint policy; $\alpha$ is the optimal assignment mechanism — the Hungarian/auction algorithm — that accepts an n by m benefit matrix and returns a feasible joint assignment maximizing the sum of benefits. The identity carrying the argument is Theorem 1, which states that for a constant deterministic joint policy the joint Q-function equals the sum of the per-agent Q-functions, so choosing by $\alpha$(Q^pi_k) is the natural analogue of choosing by argmax over joint assignments. Lemma 1 supplies convergence: the REDA target update is a gamma-contraction on the space of Q-functions with Q^pi_i as its unique fixed point, invoking the SARSA-style proof, and exploration is a Gaussian perturbation scaled by the average magnitude of the current Q-matrix.

What would settle it

A small partially observable environment with two hidden states that give the same observation but different optimal assignments would settle it: train REDA in that environment and compare the selected joint assignment against the true joint-optimal action found by exhaustive enumeration of state-specific values; if the solver driven by the learned observation-conditioned values systematically chooses the assignment that is optimal for the wrong hidden state, the paper's key approximation is contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the joint action-selection step in a sequential assignment problem can be approximated by applying an optimal assignment solver to learned per-agent values: x_k = $\alpha$(Q^pi_k) approximately equals argmax over feasible joint assignments x* of Q^pi(s_k, x*). In REDA, each agent's Q-network learns the expected future reward for being assigned to each task, and the solver $\alpha$ — the polynomial-time optimal assignment mechanism — selects the joint assignment that maximizes the sum of these learned values, perturbed by scaled noise for exploration. The authors argue the approach is theoretically justified by Theorem 1, which decomposes the joint Q-function into a sum of per-agent Q-functions when the policy is fixed, and by Lemma 1, which shows the per-agent target update is a gamma-contraction whose unique fixed point is the true Q^pi_i. From this they infer that acting through $\alpha$(Q^pi_k + xi) is approximately epsilon-greedy with respect to the joint Q-function, so REDA approximates DQN on the joint Q-function and inherits its convergence behavior.

Load-bearing premise

The argument assumes that a satellite's learned value based only on its partial observation is the same value it would learn from the full hidden state, and that the assignment policy changes slowly enough that the solver's output stays close to the policy whose values are being learned.

Editorial extensions

If this is right

  • REDA's joint assignments are always feasible by construction — the solver enforces one task per agent and at most one agent per task — so duplicate-assignment conflicts that hurt independent learners cannot occur.
  • Scaling to hundreds of agents and tasks is possible because agents learn only their own Q-functions; no centralized critic that evaluates all joint assignments (as COMA requires) is needed.
  • In deployment, alpha(Q^pi) can be computed by each agent through market-based bidding, so the optimal joint assignment can be reached without a central controller.
  • If Lemma 1 holds, agents cannot profit from reporting false values to the assignment mechanism, because training drives Q_i to the true Q^pi_i; the learned values are truthful inputs to the mechanism.
  • Bootstrapping from the greedy assignment policy and using noise scaled by the average magnitude of Q provides exploration that does not require hand-tuning to the reward scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The slow-policy assumption is testable: track how far alpha(Q^pi_k) drifts from the policy actually deployed at each update; if the drift is large during training, the learned values may be a fixed point of a different policy than the one being executed.
  • The learn-values-then-solve template is domain-general; the same decomposition should transfer to power-grid dispatch, ride-pooling, or mobile-robot scheduling whenever the global reward is the sum of per-agent rewards.
  • Because the theory identifies observation-conditioned Q_i with the full-state Q^pi_i, perceptual aliasing (different hidden states with the same observation) is the likely failure mode; adding recurrence or state estimation to the Q-network and testing in a POMDP with hidden-state confusion would show whether the gap is real.
  • REDA's action selection through a constrained optimization oracle over learned atomic values suggests a recipe for other combinatorial action spaces: learn values for feasible pieces, then let an optimization solver assemble the joint action.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces REDA (RL-Enabled Distributed Assignment), a multi-agent reinforcement learning algorithm for sequential assignment problems. REDA learns per-agent Q-functions and feeds them as a benefit matrix into the classical optimal assignment mechanism α, which selects joint assignments. Training bootstraps from a greedy policy and uses SARSA-style target updates. The authors claim a theoretical justification (Theorem 1, Lemma 1, and Equation 4), and report empirical results on a small 'dictator' environment and a realistic 324-satellite, 450-task constellation, where REDA is said to outperform IQL, IPPO, COMA, and HAAL by 20–50%. The paper includes a limitations section noting that the reward must decompose as a sum of per-agent rewards.

Significance. If the empirical results and the algorithmic design hold together, this is a practically valuable contribution: it offers a way to incorporate a classical polynomial-time assignment solver into a MARL loop, which could plausibly scale to large constellations and other assignment domains. The paper provides code, uses a realistic orbital-mechanics simulator, and reports comparisons against several strong baselines. However, the theoretical justification—which the paper explicitly leans on to explain the method's behavior—has a load-bearing gap: Lemma 1's fixed-point claim is made in a partially observable setting by identifying observations with states, and Equation 4 relies on an unproven 'slow policy change' approximation. These gaps do not necessarily invalidate the empirical results, but they undercut the paper's central claim that REDA is 'theoretically justified.' The significance of the contribution is therefore real but currently overstated.

major comments (3)
  1. [Theoretical justification, Lemma 1 and Appendix proof] The proof of Lemma 1 in the appendix proves that the operator F is a contraction on observation-conditioned Q-functions, but then identifies the fixed point with the state-defined Q^π_i by writing 'We abuse notation by writing s_k ∼ o_i_k.' In a partially observable MDP, the observation-conditioned Bellman operator has a fixed point that is the expectation of Q^π_i over states consistent with the observation, not Q^π_i itself. Consequently, the learned value Q_i(o_i, j) that is fed into α(Q^π_k) in Equation (4) is an observation-averaged quantity, and the identity Q^π(s_k, x) = Σ_i Q^π_i(s_k, x_i) from Theorem 1 does not transfer automatically to the observation-conditioned values. This is a central gap: it is the only bridge from per-agent Q-learning to the claim that α(Q^π_k) approximately maximizes the joint Q-function. The empirical results may still be valid, but the theory as written does not establish that bridge.
  2. [Method, Equation (4) and surrounding text] Equation (4) states that x_k = α(Q^π_k) ≈ argmax_{x*∈X} Q^π(s_k, x*), justified by the phrase 'if we assume that policies change slowly during the learning process such that α(Q^π_k) ≈ π(s).' This assumption is not proven and is not a standard convergence assumption. Theorem 1 holds only for a constant policy π, while REDA updates the Q-network at every step, so the policy implicit in α(Q^π_k) changes continuously. Lemma 1 separately requires a constant policy and infinite visits to each observation-action pair. The paper does not provide a rate or a bound showing that the changing policy is close enough to a constant policy for the decomposition to hold. Without this, the theoretical justification of REDA as an approximate DQN on the joint Q-function is incomplete.
  3. [Empirical Experiments, Figure 3] The paper states that REDA 'significantly outperforms' baselines by 20–50%, but no statistical significance tests or confidence intervals are reported; the only evidence is the mean and shaded standard deviation over 5 runs shown in Figure 3. Given the small number of seeds and the variance visible in the figure, a quantitative assessment of whether the improvement is distinguishable from noise (e.g., a paired test across seeds, or per-seed learning curves) is needed to support the strength of the claim. This is load-bearing for the paper's central empirical contribution.
minor comments (5)
  1. [Introduction] There is a typo: 'it is often necessarily to solve optimization problems' should be 'it is often necessary to solve.'
  2. [Algorithm 1 and Exploration paragraph] The noise scale is defined as ξ_ij ~ N(0, 2Q_avg ε), but the text does not specify whether ε is the same epsilon-decay probability used for the greedy guide policy. Clarifying this would help reproducibility.
  3. [Related Work] The reference to 'HAAL' is cited as 'Holder, Kraisler, and Mesbahi 2024' but the bibliographic entry says '(Manuscript under review)'. Please update the status or provide a preprint identifier.
  4. [Appendix, Hyperparameters] Table 2 has inconsistent spacing in the learning rate column ('0.0005' vs '0 .0005') and would benefit from formatting corrections.
  5. [Limitations] The limitations paragraph correctly notes that the method assumes r(s,x) = Σ_i r_i(s,x_i). It would strengthen the paper to state explicitly that this also excludes settings with inter-agent interference, as the discussion of frequency interference already hints, and to connect this to the theoretical decomposition.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: REDA's Bellman-style target update and external benchmark comparisons are self-contained; the only self-citation (HAAL) is used as a baseline, not as load-bearing justification.

full rationale

The central derivation is not circular. REDA trains per-agent value functions with a SARSA-style target (Algorithm 1, lines 21-23: y_i^t = r_i^t + gamma Q_i(o_{t+1}, x_{t+1}; theta_bar), with x_{t+1} = alpha(Q_{t+1})), and this update is a self-consistent Bellman equation rather than a fit to the reported return metric. Theorem 1 is a linearity identity for additive rewards, and Equation 4 is explicitly an approximation, flagged by the paper's own assumptions that "policies change slowly during the learning process such that alpha(Q^pi_k) approx pi(s)" and that "we make the approximation Q^pi_i(s,j) approx Q^pi_i(O_i(s),j)". The "abuse of notation" in the Lemma 1 proof (writing s_k ~ o_i^k) is a correctness gap under partial observability, but it is not a circular reduction: the proof attempts to identify a fixed point of an operator with Q^pi_i rather than defining Q^pi_i in terms of the conclusion. Empirically, REDA is compared against IQL, IPPO, COMA, HAAL, and the greedy policy alpha(beta(s)); using the greedy policy as both bootstrap and baseline does not force the reported 20-50% improvement, since the learned Q-values must generalize beyond greedy rollouts. The only self-citation is the HAAL baseline (Holder, Kraisler, and Mesbahi 2024), which is not load-bearing and does not import a uniqueness theorem or ansatz. No circular step is therefore recorded.

Assumptions & free parameters 10 free parameters · 5 assumptions · 0 invented entities

The algorithm introduces no new entities or fitted physical parameters. The burden is carried by hand-chosen environment constants and unproven approximations connecting partial-observation Q-learning to the centralized assignment mechanism.

free parameters (10)
  • Switching penalty = 0.5
    In the state-dependent benefit formula, an assignment that differs from the previous time step loses 0.5 benefit. This constant shapes the trade-off between stability and responsiveness and was chosen by hand.
  • Power drain per visible assignment = 0.2
    Each assignment to a visible task consumes 0.2 power; chosen by hand in the environment model.
  • Power recharge per out-of-view assignment = 0.1
    Assignments to invisible tasks recharge 0.1 power; chosen by hand.
  • Initial satellite power = 1.0
    All satellites start with 1.0 power; this normalizes the depletion dynamics.
  • Task priority values = {1, 1, 1, 5}
    Randomly assigned priorities; high-priority tasks provide five times the baseline benefit.
  • Observation lookahead L = 3
    Baseline benefits are summed over the next three time steps to select the top 10 tasks and neighbors; chosen by hand.
  • Exploration noise scale = 2 * Q_avg * epsilon
    Algorithm line 10: Gaussian perturbation with standard deviation 2 * Q_avg * epsilon; chosen by hand and not tuned per reward scale.
  • Discount factor gamma = 0.99
    Standard RL discount used in all algorithms.
  • Epsilon decay schedule = 0 after 10k steps (dictator), 300k steps (constellation)
    Exploration schedule chosen by hand; affects how long the greedy bootstrap guides learning.
  • Observation truncation width = 10 tasks and 10 neighbors
    The observation and action spaces are restricted to the 10 best tasks and 10 nearest satellites; chosen by hand and central to scalability.
assumptions (5)
  • domain assumption Reward decomposability r(s,x) = sum_i r_i(s,x_i)
    Used in Theorem 1 and throughout the method; acknowledged in the Discussion as failing when satellite beams interfere.
  • domain assumption Q_i(O_i(s),j) approximates Q_i(s,j)
    Method section states: 'in practice we make the approximation Q_i^pi(s,j) ≈ Q_i^pi(O_i(s),j)'. This is invoked by Lemma 1 and not proven.
  • standard math Constant policy and infinite visitation in Lemma 1
    Lemma 1 assumes a constant policy and that each observation-assignment pair is visited infinitely often; REDA's policy changes during learning, so the convergence is approximate.
  • ad hoc to paper Slow policy change for Eq. 4
    The method assumes 'policies change slowly during the learning process such that alpha(Q^pi_k) ≈ pi(s)'. No formal bound is given.
  • standard math Hungarian assignment alpha solves Eq. 1 in polynomial time
    The optimal assignment mechanism alpha is a standard polynomial-time result from Kuhn 1955, treated as background.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi Agent Reinforcement Learning for Sequential Satellite Assignment Problems." pith.science (2026). https://pith.science/paper/KCZB733Y

@misc{pith2026241215573,
  author       = {Pith},
  title        = {Pith review of: Multi Agent Reinforcement Learning for Sequential Satellite Assignment Problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KCZB733Y}},
  note         = {Machine review of arXiv:2412.15573}
}
read the original abstract

Assignment problems are a classic combinatorial optimization problem in which a group of agents must be assigned to a group of tasks such that maximum utility is achieved while satisfying assignment constraints. Given the utility of each agent completing each task, polynomial-time algorithms exist to solve a single assignment problem in its simplest form. However, in many modern-day applications such as satellite constellations, power grids, and mobile robot scheduling, assignment problems unfold over time, with the utility for a given assignment depending heavily on the state of the system. We apply multi-agent reinforcement learning to this problem, learning the value of assignments by bootstrapping from a known polynomial-time greedy solver and then learning from further experience. We then choose assignments using a distributed optimal assignment mechanism rather than by selecting them directly. We demonstrate that this algorithm is theoretically justified and avoids pitfalls experienced by other RL algorithms in this setting. Finally, we show that our algorithm significantly outperforms other methods in the literature, even while scaling to realistic scenarios with hundreds of agents and tasks.

Figures

Figures reproduced from arXiv: 2412.15573 by the authors.

Figure 1
Figure 1. Architecture of REDA. 1) Calculate independent estimates of future utility for each agent, combine into a matrix. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Performance over 5 runs of various algorithms in a realistic constellation environment with 324 satellites and 450 tasks, shown with standard deviation shaded. ϵ is de￾cayed to zero over 300k time steps. REDA consistently con￾verges and obtains more reward than all other tested algo￾rithms. Rather than requiring x ∈ X, which cannot be satisfied by existing algorithms, we specify that when agents assign themselves to… view at source ↗
Figure 4
Figure 4. Performance of tested algorithms on various metrics; percentage of satellites without charge at the end of the episode [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 14 canonical work pages

  1. [1]

    V.; Christianos, F.; and Sch\"afer, L

    Albrecht, S. V.; Christianos, F.; and Sch\"afer, L. 2024. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press

    2024

  2. [2]

    Azagirre, X.; Balwally, A.; Candeli, G.; Chamandy, N.; Han, B.; King, A.; Lee, H.; Loncaric, M.; Martin, S.; Narasiman, V.; et al. 2024. A Better Match for Drivers and Riders: Reinforcement Learning at L yft. INFORMS Journal on Applied Analytics, 54(1): 71--83

    work page 2024

  3. [3]

    Berner, C.; Brockman, G.; Chan, B.; Cheung, V.; Debiak, P.; Dennison, C.; Farhi, D.; Fischer, Q.; Hashme, S.; Hesse, C.; et al. 2019. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680

    arXiv 2019

  4. [4]

    Bui, V.-P.; Van Chien, T.; Lagunas, E.; Grotz, J.; Chatzinotas, S.; and Ottersten, B. 2022. Joint Beam Placement and Load Balancing Optimization for Non-Geostationary Satellite Systems. In 2022 IEEE International Mediterranean Conference on Communications and Networking (MeditCom), 316--321

    work page 2022

  5. [5]

    M.; Griffiths, T.; and Levine, S

    Chang, M.; Kaushik, S.; Weinberg, S. M.; Griffiths, T.; and Levine, S. 2020. Decentralized reinforcement learning: Global decision-making via local economic transactions. In International Conference on Machine Learning, 1437--1447. PMLR

    work page 2020

  6. [6]

    S.; Gupta, T.; Makoviichuk, D.; Makoviychuk, V.; Torr, P

    De Witt, C. S.; Gupta, T.; Makoviichuk, D.; Makoviychuk, V.; Torr, P. H.; Sun, M.; and Whiteson, S. 2020. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533

    arXiv 2020

  7. [7]

    Foerster, J.; Farquhar, G.; Afouras, T.; Nardelli, N.; and Whiteson, S. 2018. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI conference on artificial intelligence, volume 32

    2018

  8. [8]

    P.; and Matari \'c , M

    Gerkey, B. P.; and Matari \'c , M. J. 2004. A formal analysis and taxonomy of task allocation in multi-robot systems. The International journal of robotics research, 23(9): 939--954

    2004

Show all 28 references
  1. [9]

    Giovanelli, C.; Kilkki, O.; Sierla, S.; Seilonen, I.; and Vyatkin, V. 2019. Task Allocation Algorithm for Energy Resources Providing Frequency Containment Reserves. IEEE Transactions on Industrial Informatics, 15: 677--688

    2019

  2. [10]

    Holder, J.; Kraisler, S.; and Mesbahi, M. 2024. Centralized and Distributed Strategies for Handover-Aware Task Allocation in Satellite Constellations. Journal of Guidance Control and Dynamics (Manuscript under review)

    2024

  3. [11]

    Hwang, K.-S.; Chen, Y.-J.; and Lee, C.-H. 2007. Reinforcement Learning in Strategy Selection for a Coordinated Multirobot System. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 37(6): 1151--1157

    2007

  4. [12]

    Kuhn, H. W. 1955. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2): 83--97

    1955

  5. [13]

    Lin, Z.; Ni, Z.; Kuang, L.; Jiang, C.; and Huang, Z. 2022. Dynamic Beam Pattern and Bandwidth Allocation Based on Multi-Agent Deep Reinforcement Learning for Beam Hopping Satellite Systems. IEEE Transactions on Vehicular Technology, 71: 3917--3930

    2022

  6. [14]

    J.; and Le Fort-Piat, N

    Matignon, L.; Laurent, G. J.; and Le Fort-Piat, N. 2012. Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems. The Knowledge Engineering Review, 27(1): 1--31

    2012

  7. [15]

    Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; and Riedmiller, M. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602

    2013 arXiv

  8. [16]

    Pachler de la Osa, N.; Guerster, M.; del Portillo Barrios, I.; Crawley, E.; and Cameron, B. 2021. Static beam placement and frequency plan algorithms for LEO constellations. International Journal of Satellite Communications and Networking, 39(1): 65--77

    2021

  9. [17]

    Papoudakis, G.; Christianos, F.; Sch \"a fer, L.; and Albrecht, S. V. 2020. Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. arXiv preprint arXiv:2006.07869

    2020 arXiv

  10. [18]

    Qin, Z.; Zhu, H.; and Ye, J. 2022. Reinforcement Learning for Ridesharing : An Extended Survey . Transportation Research Part C: Emerging Technologies, 144

    2022

  11. [19]

    S.; Farquhar, G.; Foerster, J.; and Whiteson, S

    Rashid, T.; Samvelyan, M.; De Witt, C. S.; Farquhar, G.; Foerster, J.; and Whiteson, S. 2020. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21(178): 1--51

    2020

  12. [20]

    Shah, S.; Lowalekar, M.; and Varakantham, P. 2020. Neural Approximate Dynamic Programming for On-Demand Ride-Pooling. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01): 507--515

    2020

  13. [21]

    L.; and Szepesv \'a ri, C

    Singh, S.; Jaakkola, T.; Littman, M. L.; and Szepesv \'a ri, C. 2000. Convergence results for single-step on-policy reinforcement-learning algorithms. Machine learning, 38: 287--308

    2000

  14. [22]

    M.; Zambaldi, V.; Jaderberg, M.; Lanctot, M.; Sonnerat, N.; Leibo, J

    Sunehag, P.; Lever, G.; Gruslys, A.; Czarnecki, W. M.; Zambaldi, V.; Jaderberg, M.; Lanctot, M.; Sonnerat, N.; Leibo, J. Z.; Tuyls, K.; et al. 2017. Value-decomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296

    2017 arXiv

  15. [23]

    Tampuu, A.; Matiisen, T.; Kodelja, D.; Kuzovkin, I.; Korjus, K.; Aru, J.; Aru, J.; and Vicente, R. 2017. Multiagent cooperation and competition with deep reinforcement learning. PloS one, 12(4): e0172395

    2017

  16. [24]

    Wang, X.; Wu, G.; Xing, L.; and Pedrycz, W. 2021. Agile Earth Observation Satellite Scheduling Over 20 Years: Formulations, Methods, and Future Directions. IEEE Systems Journal, 15(3): 3881--3892

    2021

  17. [25]

    Yu, C.; Velu, A.; Vinitsky, E.; Gao, J.; Wang, Y.; Bayen, A.; and Wu, Y. 2022. The surprising effectiveness of PPO in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35: 24611--24624

    2022

  18. [26]

    M.; Spesivtsev, L.; and Pappas, G

    Zavlanos, M. M.; Spesivtsev, L.; and Pappas, G. J. 2008. A distributed auction algorithm for the assignment problem. In 2008 47th IEEE Conference on Decision and Control, 1212--1217. IEEE

    2008

  19. [27]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  20. [28]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.