Pith. sign in

REVIEW 4 major objections 5 minor 56 references

Learning To Communicate Over An Unknown Shared Network

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Trained only on a one-parameter queue simulation, the query policy QNet transfers zero-shot to real WiFi and cellular networks and matches or beats fixed policies under contention.

desk verdict Useful first sim-to-real result for DRL query control over wireless: credible WiFi/cellular transfer but the 'any number of agents' claim and uncalibrated max-Q selection need fixing. read the letter →

arxiv 2507.06499 v1 pith:MIIZ7X57 submitted 2025-07-09 cs.MA cs.NI

Shivangi Agarwal , Adi Asija , Sanjit K. Kaul , Arani Bhattacharya , Saket Anand This is my paper
classification cs.MAcs.NI
keywords multi-agentreinforcementlearningdeepquerycontroledge-cloudcommunicationageofinformationsim-to-realtransferdomainrandomizationwirelessnetworks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks when an agent sharing a wireless network with an unknown number of other agents should query its edge-cloud for fresh information. Its answer is a deep reinforcement learning policy, Query Net (QNet), that decides whether to query from the age of its latest received measurement and its current estimate of the source. The policy is trained entirely on a deliberately simple simulation in which the network and edge-cloud form a single first-come-first-served queue whose service time is geometric with one parameter q, randomized over (0.05, 1) during training. The central empirical claim is that this simulation-only policy transfers zero-shot to real WiFi networks with 5 to 50 agents and to real cellular traces. Under contention it matches or beats Always Query, threshold, and probabilistic baselines while querying less; only in very lightly loaded networks does Always Query win.

What carries the argument

The load-bearing object is the single-parameter service-facility simulator: the wireless network plus the edge-cloud is one first-come-first-served queue, and a packet in service finishes in a given slot with probability q, so service times are geometric(q). That one number q stands in for all other traffic, packet errors, and congestion, and randomizing it over (0.05, 1) exposes training to conditions from near-idle to heavily queued. Around this simulator sit three QNet instances, each with an estimator network that forms the agent's estimate from received measurements and an actor-critic pair that sees the estimate plus the age of the most recent measurement; deployment takes the maximum Q-value across the three critics. The age signal carries the argument across the simulation-to-real gap: it is a technology-agnostic summary of how the shared network treats the agent's own queries, and it is the only network information the policy is allowed to use.

What would settle it

Deploy QNet on a network with enough agents that the agents' own querying measurably slows the network, for example a WiFi testbed with 100 agents or with cross-traffic that reacts to query load, and compare the measured average-age versus estimation-error scatter with the simulation envelope. If the scatter leaves the envelope at a fixed apparent q, or if the policy's query rate fails to track rising contention, the single-parameter sufficiency assumption is falsified.

Watch

Extended reading notes

Core claim

QNet is a three-network deep RL pipeline: an estimator reconstructs the source state from sparse delayed measurements, and actor-critic networks score Query versus Don't Query given the current estimate and the age of the newest measurement. Because a single policy trained over the full q range fails to vary its query rate with network speed, the paper trains one QNet per q range, (0.05, 0.1), (0.1, 0.3), and (0.3, 1.0), and at deployment selects the action with the largest Q-value among the three critics. The core finding is that this simulator-trained policy generalizes to real networks without fine-tuning: on a real WiFi testbed the average-age versus estimation-error scatter overlaps the simulation scatter, and as contention rises the policy lowers its query rate. Concretely, at 50 agents QNet reports mean estimation error 25.0 with standard deviation 2.3, versus 31.3 with standard deviation 9.2 for Always Query, while at 5 agents Always Query reports 0.6 versus QNet's 1.0. On cellular traces the same pattern holds: QNet is competitive in good conditions and has lower mean error than Always Query when baseline round-trip times exceed 0.2 seconds.

Load-bearing premise

The whole transfer claim rests on assuming that the effect of every other user and channel condition can be summarized by one number, the per-slot probability that a packet finishes service, and that this number does not change as a result of how often agents query.

Editorial extensions

If this is right

  • An agent can be deployed on an unfamiliar shared network with no per-network training and no knowledge of how many other agents are present; the same simulator-trained model is used without modification on WiFi and cellular.
  • At 25 WiFi agents, Always Query and QNet have similar mean estimation error but Always Query queries about 1.5 times as often, so QNet achieves equal tracking accuracy with a materially lower load on the network.
  • Fixed threshold and probabilistic policies require choosing a threshold or scaling per agent count, whereas QNet adapts its query rate automatically as contention rises.
  • The overlap between simulated and real age-versus-error scatter suggests the low-fidelity queue can serve as a cheap screening tool: configurations whose average age and error fall inside the simulated envelope are expected to transfer to real networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is to let q vary within an episode or depend on recent query load, modeling non-stationary contention and congestion rather than a fixed per-episode service rate.
  • Because the policy's input is an age and a learned estimate rather than raw measurements, the same decision layer may transfer across source processes with similar estimation-error dynamics if the estimator is retrained for the new source.
  • The three-model max-Q selection suggests a probabilistic generalization: instead of three discrete q ranges, one could maintain a belief over q from observed delays and compute query values by posterior-weighted averaging, making the same idea work continuously.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes QNet, a deep RL policy that decides, at each decision instant, whether an edge-device should query an edge-cloud over a shared wireless network whose state is unknown. QNet is trained exclusively on a low-fidelity single-server queue simulation with a single parameter q (geometric service time), using domain randomization over q in (0.05, 1). Three separate RL models are trained for disjoint q-ranges and are combined at deployment by taking the action with the maximum Q-value among the six Q-values output by the three critics. The resulting policy is evaluated zero-shot on real WiFi networks (ORBIT testbed, 5 to 50 agents) and on cellular traces replayed with Mahimahi, and it is compared against Always Query, threshold-based, and probabilistic querying baselines. The central empirical claim is that QNet transfers from simulation to real networks, adapts its query rate to unknown contention, and matches or outperforms the baselines except at very low contention (5 agents), where Always Query is better.

Significance. If the claims hold, the paper is a useful practical contribution: it demonstrates that a one-parameter queue model with domain randomization can produce a communication policy that transfers zero-shot to real WiFi and cellular networks, and it provides a head-to-head comparison with several baselines. The real-network experiments, especially the ORBIT WiFi experiments with 5 to 50 agents and packet error rates up to 60%, are a genuine strength, as is the explicit attempt to validate simulation-to-real transfer through overlapping scatter plots and age-binned error tables. The main limitations are that the abstract's 'any number of other agents' claim is extrapolated beyond the tested range, and the deployment-time combination of the three trained critics via an uncalibrated maximum over Q-values is not justified.

major comments (4)
  1. [Section 5.3, Figure 7] The deployment rule picks the action corresponding to the maximum of six Q-values produced by three critics trained on different q-ranges with different n-step returns (60, 20, 10) and different target entropies (0.09, 0.3, 0.6). Q-values from different SAC critics are not on a common scale; an arbitrary offset or scaling in one critic can dominate the maximum and make the selected action depend on training artifacts rather than on the current network state. The paper gives no justification for this max-over-critics rule and no ablation showing that it beats alternatives. Please either justify the calibration of the Q-values, replace the rule with a principled arbitration (for example, an estimate of q, a weighted average of the policies, or a hierarchical selector), or provide empirical evidence that the six-Q-value maximum is necessary and robust.
  2. [Abstract, Section 3, Section 4] The claim that the policy 'generalizes well to any number of other agents' is not supported by the training or the experiments. The simulation model of Section 4 contains exactly one agent: the only arrivals to the queue are the agent's own queries, and all contention from other agents is collapsed into the exogenous per-episode service parameter q. The WiFi experiments do run multiple QNet agents and show encouraging transfer for 5 to 50 agents, but that is interpolation over a tested range, not evidence for arbitrary N, and the training distribution never exposes the policy to the endogenous feedback in which its own querying and other QNet agents' querying jointly determine the congestion. A concrete test would be a multi-agent simulation sweep in which all agents use QNet and N is varied (for example, N = 100, 200), or training with N as part of the randomized domain. At minimum, the abstract and Section 3 should be rephrased to claim generalization over the tested range rather than 'any number'.
  3. [Section 4, Figure 2, Section 6.2] The simulator never drops packets: every enqueued packet is eventually served, and the agent receives a response to every successful query. In the real WiFi experiments, however, the packet error rate reaches 60% (Figure 10b), meaning that a substantial fraction of queries or responses are permanently lost, an event class never experienced during training. Additionally, q is described both as the probability that a packet finishes service in a time slot and as the probability that a packet is successfully transmitted in an attempt (Section 4, paragraph beginning 'The probability q models...'), which are not the same quantity. The paper should clarify what q represents, state explicitly whether permanent query loss is part of the simulation model, and explain why the learned policy still performs well in a deployment regime with significant loss. Without this clarification, the mechanism behind the claimed sim-to-real transfer for high-contention WiFi scenarios is incomplete.
  4. [Section 7.1] The cellular experiments use two agents, each with its own uplink and downlink traces replayed by Mahimahi. This setup models time-varying channel conditions, but the two agents do not actually contend for a shared network resource. Consequently, the cellular results validate transfer across a range of network conditions, but they do not test the shared-network aspect of the problem, and they should not be presented as evidence for multi-agent generalization. Please state this limitation explicitly and, if possible, add a cellular or emulated experiment in which agents share a bottleneck.
minor comments (5)
  1. [Figure 2] The text 'Ground Trurth' in Figure 2 is a typo and should read 'Ground Truth'.
  2. [Section 6.2] The sentence 'Varying the number of agents has us observe close to no retires to a significant fraction of retries' contains a typo ('retires' should be 'retries') and would benefit from rewording.
  3. [Section 6.4, Table 2] The table reports means and standard deviations but does not state the number of repeated experiments or runs behind each entry, and no statistical significance tests are provided. Since many reported differences are within one standard deviation (for example, the 25-agent row, where QNet and Always Query both report 1.3), the claim that QNet 'does at least as well or better' would be strengthened by confidence intervals or pairwise tests.
  4. [Figures 9 and 12, Tables 1 and 3] The claim that simulated and real scatter plots 'overlap quite well' is qualitative; please add a quantitative agreement metric (for example, root-mean-square difference or a correlation coefficient) and report the number of simulation episodes and real experiments used in each age bin.
  5. [Section 5.1] The reward scaling parameters (r_s = 5, r = 8e4), the per-range n-step values, and the target entropy values are stated but no sensitivity analysis is provided. Since these are central to making training work, a brief ablation or a statement of how they were selected would help reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: QNet is an empirically validated sim-to-real pipeline tested against external real WiFi and cellular networks, with no self-citations and no fitted parameter renamed as a prediction.

full rationale

The paper makes no analytic derivation that could reduce to its own inputs; the load-bearing claim is that a policy trained in a one-parameter single-server queue simulation transfers zero-shot to real wireless networks. The simulation parameter q is randomized over (0.05, 1) during training, and the trained QNet is then evaluated on external real-world data: ORBIT WiFi experiments with 5 to 50 agents and Mahimahi-replayed 4G LTE traces from NYC and Ghent. No real-network measurement used in the evaluation is fitted into the training procedure, so the transfer result is not forced by construction. The baselines (Always Query, threshold-based, and probabilistic policies) are compared on the same real data, and the 'best' baseline variants are selected post hoc per agent count, but QNet itself is not selected on those data; this is a comparison choice, not a fitted-input-called-prediction. The reference list contains no self-citations by the authors, and the cited techniques (domain randomization, SUMO, IDM, Mahimahi, Saturator) are independent external tools and datasets. The manuscript does contain an inconsistency in the meaning of q (service-completion probability in Section 4 versus successful-transmission probability later), and the training simulator contains only one agent with exogenous contention, so the abstract's claim that the policy 'generalizes well to any number of other agents' is an extrapolation beyond the tested range of 5 to 50 agents and is not mechanistically demonstrated for the endogenous multi-agent feedback regime. These are correctness and generalization risks, not circularity: the claim is not true by definition, by construction, or by self-citation. Therefore, the honest finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The pipeline rests on the queue abstraction and the hand-selected training regimes; no new physical entities are introduced. The free parameters are training/hyperparameter choices, not fitted to the real-network transfer target.

free parameters (3)
  • simulation service parameter q = randomized in (0.05, 1); split into (0.05,0.1), (0.1,0.3), (0.3,1.0)
    The central training abstraction. The range and split are chosen by hand; a single model over the full range fails (QNet-one), so three models with per-range n-step (60/20/10) and target entropy (0.09/0.3/0.6) are used.
  • reward scaling r_s and error cap r = r_s=5.0, r=8e4
    Reward = r_s * (1 - error^2 / r). Values are fixed without sensitivity analysis. They shape the RL objective but are not fitted to real data.
  • discount factor gamma = not explicitly specified in text (0<gamma<1)
    The objective uses a discounted sum; the exact value is not reported, affecting re-implementation.
assumptions (3)
  • domain assumption A shared wireless network and edge-cloud can be modeled as a single-server FCFS queue with i.i.d. geometric service times.
    Section 4 and Figure 2. This abstracts away multi-user contention, losses and correlated delays; the agent's own queries are the only arrivals, and q is fixed per episode.
  • domain assumption The age of the last received measurement and the current estimate are a sufficient statistic for the query decision.
    Section 3 defines policy pi_i(X_i(tau), Delta_i(tau)); no POMDP belief-state argument is given, yet the actor-critic uses only (estimate, age).
  • domain assumption The LSTM estimator can learn an unbiased estimate of the source state from sparse, delayed samples.
    Section 5.1 uses an LSTM+FC estimator pre-trained with random q/2 querying; no error bound for the learned estimator is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning To Communicate Over An Unknown Shared Network." pith.science (2026). https://pith.science/paper/MIIZ7X57

@misc{pith2026250706499,
  author       = {Pith},
  title        = {Pith review of: Learning To Communicate Over An Unknown Shared Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MIIZ7X57}},
  note         = {Machine review of arXiv:2507.06499}
}
read the original abstract

As robots (edge-devices, agents) find uses in an increasing number of settings and edge-cloud resources become pervasive, wireless networks will often be shared by flows of data traffic that result from communication between agents and corresponding edge-cloud. In such settings, agent communicating with the edge-cloud is unaware of state of network resource, which evolves in response to not just agent's own communication at any given time but also to communication by other agents, which stays unknown to the agent. We address challenge of an agent learning a policy that allows it to decide whether or not to communicate with its cloud node, using limited feedback it obtains from its own attempts to communicate, to optimize its utility. The policy generalizes well to any number of other agents sharing the network and must not be trained for any particular network configuration. Our proposed policy is a DRL model Query Net (QNet) that we train using a proposed simulation-to-real framework. Our simulation model has just one parameter and is agnostic to specific configurations of any wireless network. It allows training an agent's policy over a wide range of outcomes that an agent's communication with its edge-cloud node may face when using a shared network, by suitably randomizing the simulation parameter. We propose a learning algorithm that addresses challenges observed in training QNet. We validate our simulation-to-real driven approach through experiments conducted on real wireless networks including WiFi and cellular. We compare QNet with other policies to demonstrate its efficacy. WiFi experiments involved as few as five agents, resulting in barely any contention for the network, to as many as fifty agents, resulting in severe contention. The cellular experiments spanned a broad range of network conditions, with baseline RTT ranging from a low of 0.07 second to a high of 0.83 second.

Figures

Figures reproduced from arXiv: 2507.06499 by the authors.

Figure 1
Figure 1. At the top is a sequence diagram of communication between agent [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. A low-cost low-fidelity simulation model to generate experiences for the agent to train its policy. The wireless network and [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Average age as a function of utilization [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Training Pipeline of Estimator, Actor and Critic NN. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparing QNet and QNet-one. Figure (a) shows Query Rate and Average Age and (b) the estimation error, for different [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparing training performances of QNet with QNet-One and other neural architectures with varying input configurations. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Selecting actions during execution in an unknown network environment using three neural networks. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: ORBIT Setup: Source 𝑖 uses SUMO trace 𝑖 to generate measurement packets and set the query bit to 0 or 1 in the packet. It uses WiFi client 𝑖 to send packets over the WiFi network to the AP. The AP then sends responses to queries. In our experiments, we vary the number …
Figure 9
Figure 9. Figure 9: A scatter plot of average age and average estimation error for experiments using real WiFi networks overlaid with a scatter [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: (a) Received Signal Strength in dBm, (b) Packet Error Rate (%) , (c) RTT (s), and (d) Retry Rate (%), with (b), (c) and (d) plotted [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Comparison of (a) mean estimation error, (b) query rates, and (c) average age across all agent configurations. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]
Figure 12
Figure 12. Figure 12: A scatter plot of average age and average estimation error for experiments using real Cellular networks overlaid with a scatter [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Comparison of policies over not-stationary experiments with bRTT [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Comparison of policies over not-stationary experiments with bRTT [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]
Figure 15
Figure 15. Figure 15: Comparison of policies over stationary experiments. [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 45 canonical work pages

  1. [1]

    Jonathan Chao

    Soheil Abbasloo, Chen-Yu Yen, and H. Jonathan Chao. 2020. Classic Meets Modern: a Pragmatic Learning-Based Congestion Control for the Internet. In Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication (Virtual Event, USA) (SIGC...

    arXiv 2020

  2. [2]

    Subramaniam, Zuriati Ahmad Zukarnain, and Normalia Samian

    Ghaihab Hassan Adday, Shamala K. Subramaniam, Zuriati Ahmad Zukarnain, and Normalia Samian. 2024. Investigating and Analyzing Simulation Tools of Wireless Sensor Networks: A Comprehensive Survey. IEEE Access 12 (2024), 22938–22977. doi:10.1109/ACCESS.2024.3362889

    arXiv 2024

  3. [3]

    Eduardo Candela, Leandro Parada, Luis Marques, Tiberiu-Andrei Georgescu, Yiannis Demiris, and Panagiotis Angeloudis. 2022. Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, Kyoto,Japan, 8814–8820. doi:10.1109/IROS4761...

    arXiv 2022

  4. [4]

    Wyrwal, and Marco Caccamo

    Hongpeng Cao, Mirco Theile, Federico G. Wyrwal, and Marco Caccamo. 2022. Cloud-Edge Training Architecture for Sim-to-Real Deep Reinforcement Learning. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, Kyoto, Japan, 9363–9370. doi:10.1109/iros47612. 2022.9981565

    work page arXiv 2022

  5. [5]

    Marwa Chafii, Salmane Naoumi, Reda Alami, Ebtesam Almazrouei, Mehdi Bennis, and Merouane Debbah. 2023. Emergent Communication in Multi-Agent Reinforcement Learning for Future Wireless Networks. IEEE Internet of Things Magazine 6, 4 (2023), 18–24

    work page 2023

  6. [6]

    Shutong Chen, Guanjun Liu, Ziyuan Zhou, Kaiwen Zhang, and Jiacun Wang. 2023. Robust multi-agent reinforcement learning method based on adversarial domain randomization for real-world dual-uav cooperation. IEEE Transactions on Intelligent Vehicles 1 (2023), 1615 – 1627

    work page 2023

  7. [7]

    Xiaoyu Chen, Jiachen Hu, Chi Jin, Lihong Li, and Liwei Wang. 2022. Understanding Domain Randomization for Sim-to-real Transfer. InInternational Conference on Learning Representations . iclr, virtual, . https://openreview.net/forum?id=T8vZHIRTrY

    work page 2022

  8. [8]

    Petros Christodoulou. 2019. Soft actor-critic for discrete action settings. arXiv preprint arXiv:1910.07207 -, - (2019), –

    arXiv 2019

Show all 56 references
  1. [9]

    Abhishek Das, Théophile Gervet, Joshua Romoff, Dhruv Batra, Devi Parikh, Mike Rabbat, and Joelle Pineau. 2019. Tarmac: Targeted multi-agent communication. In International Conference on Machine Learning . PMLR, Long Beach, California„ 1538–1546

    2019

  2. [10]

    Ziluo Ding, Tiejun Huang, and Zongqing Lu. 2020. Learning individually inferred communication for multi-agent cooperation. Advances in Neural Information Processing Systems 33 (2020), 22069–22079

    2020

  3. [11]

    Jakob Foerster, Ioannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson. 2016. Learning to communicate with deep multi-agent reinforcement learning. Advances in neural information processing systems 29 (2016), 2145–2153

    2016

  4. [12]

    Jennifer Gielis, Ajay Shankar, and Amanda Prorok. 2022. A critical review of communications in multi-robot systems. Current Robotics Reports 3, 4 (2022), 213–225

    2022

  5. [13]

    Sven Gronauer and Klaus Diepold. 2022. Multi-agent deep reinforcement learning: a survey. Artificial Intelligence Review 55, 2 (2022), 1–49

    2022

  6. [14]

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning . PMLR, Stockholm,Sweden, 1861–1870

    2018

  7. [15]

    Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al. 2018. Soft actor-critic algorithms and applications. arXiv preprint arXiv:1812.05905 -, - (2018), –

    2018 arXiv

  8. [16]

    Shuai Han, Mehdi Dastani, and Shihan Wang. 2023. Model-based Sparse Communication in Multi-agent Reinforcement Learning. InProceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems (London, United Kingdom). International Foundation for Autono...

    2023

  9. [17]

    Dániel Horváth, Gábor Erdős, Zoltán Istenes, Tomáš Horváth, and Sándor Földi. 2022. Object detection using sim2real domain randomization for robotic applications. IEEE Transactions on Robotics 39, 2 (2022), 1225–1243

    2022

  10. [18]

    Diyi Hu, Chi Zhang, Viktor Prasanna, and Bhaskar Krishnamachari. 2023. Learning Practical Communication Strategies in Cooperative Multi-Agent Reinforcement Learning. In Asian Conference on Machine Learning . PMLR, Istanbul,Turkey, 467–482

    2023

  11. [19]

    Guangzheng Hu, Yuanheng Zhu, Dongbin Zhao, Mengchen Zhao, and Jianye Hao. 2020. Event-triggered multi-agent reinforcement learning with communication under limited-bandwidth constraint. arXiv preprint arXiv:2010.04978 -, - (2020), –

    2020 arXiv

  12. [20]

    Kathy Jang, Eugene Vinitsky, Behdad Chalaki, Ben Remer, Logan Beaver, Andreas A Malikopoulos, and Alexandre Bayen. 2019. Simulation to scaled city: zero-shot policy transfer for traffic control via autonomous vehicles. In Proceedings of the 10th ACM/IEEE International Conferen...

    2019

  13. [21]

    Abhishek Kadian, Joanne Truong, Aaron Gokaslan, Alexander Clegg, Erik Wijmans, Stefan Lee, Manolis Savva, Sonia Chernova, and Dhruv Batra

  14. [22]

    Lukas Kesper, Sebastian Trimpe, and Dominik Baumann. 2023. Toward Multi-Agent Reinforcement Learning for Distributed Event-Triggered Control. In Proceedings of The 5th Annual Learning for Dynamics and Control Conference (Proceedings of Machine Learning Research, Vol. 211) , Ni...

    2023

  15. [23]

    Daewoo Kim, Sangwoo Moon, David Hostallero, Wan Ju Kang, Taeyoung Lee, Kyunghwan Son, and Yung Yi. 2019. Learning to schedule communication in multi-agent reinforcement learning. arXiv preprint arXiv:1902.01554 -, - (2019), –

    2019 arXiv

  16. [24]

    Kota Kondo, Reinaldo Figueroa, Juan Rached, Jesus Tordesillas, Parker C Lusk, and Jonathan P How. 2023. Robust mader: Decentralized multiagent trajectory planner robust to communication delay in dynamic environments. IEEE ROBOTICS AND AUTOMATION LETTERS 9 (2023), 1476–1483

    2023

  17. [25]

    Daniel Krajzewicz, Jakob Erdmann, Michael Behrisch, and Laura Bieker. 2012. Recent development and applications of SUMO-Simulation of Urban MObility. International journal on advances in systems and measurements 5, 3&4 (2012)

    2012

  18. [26]

    Papadopoulos, Antoine Gallais, Periklis Chatzimisios, and Fabrice Théoleyre

    Kosmas Kritsis, Georgios Z. Papadopoulos, Antoine Gallais, Periklis Chatzimisios, and Fabrice Théoleyre. 2018. A Tutorial on Performance Evaluation and Validation Methodology for Low-Power and Lossy Networks. IEEE Communications Surveys & Tutorials 20, 3 (2018), 1799–1825. doi...

    2018

  19. [27]

    Shaoshan Liu, Liangkai Liu, Jie Tang, Bo Yu, Yifan Wang, and Weisong Shi. 2019. Edge Computing for Autonomous Driving: Opportunities and Challenges. Proc. IEEE 107, 8 (2019), 1697–1716. doi:10.1109/JPROC.2019.2915983

    2019

  20. [28]

    Hangyu Mao, Zhengchao Zhang, Zhen Xiao, Zhibo Gong, and Yan Ni. 2020. Learning Agent Communication under Limited Bandwidth by Message Pruning. Proceedings of the AAAI Conference on Artificial Intelligence 34, 04 (Apr. 2020), 5142–5149. doi:10.1609/aaai.v34i04.5957

    2020 doi

  21. [29]

    Federico Mason, Federico Chiariotti, Andrea Zanella, and Petar Popovski. 2023. Multi-Agent Reinforcement Learning for Pragmatic Communication and Control. arXiv preprint arXiv:2302.14399 - (2023), –

    2023 arXiv

  22. [30]

    Ravi Netravali, Anirudh Sivaraman, Somak Das, Ameesh Goyal, Keith Winstein, James Mickens, and Hari Balakrishnan. 2015. Mahimahi: accurate record-and-replay for HTTP. In Proceedings of the 2015 USENIX Conference on Usenix Annual Technical Conference (Santa Clara, CA) (USENIX A...

    2015

  23. [31]

    Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. 2018. Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA) . IEEE, Brisbane,Australia, 3803–3810

    2018

  24. [32]

    Amanda Prorok, Jan Blumenkamp, Qingbiao Li, Ryan Kortvelesy, Zhe Liu, and Ethan Stump. 2022. The Holy Grail of Multi-Robot Planning: Learning to Generate Online-Scalable Solutions from Offline-Optimal Experts. In Proceedings of the 21st International Conference on Autonomous A...

    2022

  25. [33]

    Haoran Shi, Guanjun Liu, Kaiwen Zhang, Ziyuan Zhou, and Jiacun Wang. 2022. Marl sim2real transfer: Merging physical reality with digital virtuality in metaverse. IEEE Transactions on Systems, Man, and Cybernetics: Systems 53, 4 (2022), 2107–2117

    2022

  26. [34]

    Jonah Siekmann, Srikar Valluri, Jeremy Dao, Lorenzo Bermillo, Helei Duan, Alan Fern, and Jonathan Hurst. 2020. Learning Memory-Based Control for Human-Scale Bipedal Locomotion. Robotics: Science and Systems x, x (2020), x. arXiv:2006.02402 [cs.RO] https://doi.org/10.48550/arXi...

    2020 doi arXiv

  27. [35]

    Amanpreet Singh, Tushar Jain, and Sainbayar Sukhbaatar. 2018. Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks. arXiv preprint arXiv:1812.09755 -, - (2018), –

    2018 arXiv

  28. [36]

    Sainbayar Sukhbaatar, Arthur Szlam, and Rob Fergus. 2016. Learning multiagent communication with backpropagation. In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS’16). Curran Associates Inc., Red Hook, NY, US...

    2016

  29. [37]

    Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press

    2018

  30. [38]

    Gabriele Tiboni, Andrea Protopapa, Tatiana Tommasi, and Giuseppe Averta. 2023. Domain Randomization for Robust, Affordable and Effective Closed-loop Control of Soft Robots. x x, x (2023), x. arXiv:2303.04136 [cs.RO] https://arxiv.org/abs/2303.04136

    2023 arXiv

  31. [39]

    Martin Treiber, Ansgar Hennecke, and Dirk Helbing. 2000. Congested traffic states in empirical observations and microscopic simulations. Physical review E 62, 2 (2000), 1805

    2000

  32. [40]

    Eugene Valassakis, Zihan Ding, and Edward Johns. 2020. Crossing the gap: A deep dive into zero-shot sim-to-real transfer for dynamics. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, Las Vegas, USA, 5372–5379

    2020

  33. [41]

    Jeroen Van Der Hooft, Stefano Petrangeli, Tim Wauters, Rafael Huysegems, Patrice Rondao Alface, Tom Bostoen, and Filip De Turck. 2016. HTTP/2-based adaptive streaming of HEVC video over 4G/LTE networks. IEEE Communications Letters 20, 11 (2016), 2177–2180

    2016

  34. [42]

    Rundong Wang, Xu He, Runsheng Yu, Wei Qiu, Bo An, and Zinovi Rabinovich. 2020. Learning efficient multi-agent communication: An information bottleneck approach. In International Conference on Machine Learning . PMLR, Vienna, Austria, 9908–9918

    2020

  35. [43]

    Keith Winstein, Anirudh Sivaraman, and Hari Balakrishnan. 2013. Stochastic forecasts achieve high throughput and low delay over cellular networks. In Proceedings of the 10th USENIX Conference on Networked Systems Design and Implementation (Lombard, IL) (nsdi’13). USENIX Associ...

    2013

  36. [44]

    Zhaoming Xie, Patrick Clary, Jeremy Dao, Pedro Morais, Jonanthan Hurst, and Michiel Panne. 2020. Learning locomotion skills for cassie: Iterative design and sim-to-real. In Conference on Robot Learning . PMLR, Massachusetts Institute of Technology, 317–329

    2020

  37. [45]

    Zhaoming Xie, Xingye Da, Michiel van de Panne, Buck Babich, and Animesh Garg. 2021. Dynamics Randomization Revisited: A Case Study for Quadrupedal Locomotion. In 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE Press, Xi’an, China, 4955–4961. doi:10....

    2021

  38. [46]

    Chao Yu, Xinyi Yang, Jiaxuan Gao, Jiayu Chen, Yunfei Li, Jijia Liu, Yunfei Xiang, Ruixin Huang, Huazhong Yang, Yi Wu, and Yu Wang. 2023. Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Exploration. In Proceedings of the 2023 Inte...

    2023

  39. [47]

    Lei Yuan, Ziqian Zhang, Lihe Li, Cong Guan, and Yang Yu. 2023. A Survey of Progress on Cooperative Multi-agent Reinforcement Learning in Open Environment. arXiv preprint arXiv:2312.01058 -, - (2023), 62 pages

    2023 arXiv

  40. [48]

    Mingfeng Yuan, Jinjun Shan, and Kevin Mi. 2023. From Naturalistic Traffic Data to Learning-Based Driving Policy: A Sim-to-Real Study. IEEE Transactions on Vehicular Technology -, - (2023), 1–14. doi:10.1109/TVT.2023.3307409

    2023

  41. [49]

    Tingting Yuan, Hwei-Ming Chung, Jie Yuan, and Xiaoming Fu. 2023. DACOM: learning delay-aware communication for multi-agent reinforcement learning. In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applicat...

    2023 doi

  42. [50]

    Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. 2021. Multi-agent reinforcement learning: A selective overview of theories and algorithms. Handbook of reinforcement learning and control - (2021), 321–384

    2021

  43. [51]

    Sai Qian Zhang, Qi Zhang, and Jieyu Lin. 2019. Efficient communication in multi-agent reinforcement learning via variance based control . Curran Associates Inc., Red Hook, NY, USA, Chapter -, –

    2019

  44. [52]

    Sai Qian Zhang, Qi Zhang, and Jieyu Lin. 2020. Succinct and robust multi-agent communication with temporal message control. Advances in Neural Information Processing Systems 33 (2020), 17271–17282

    2020

  45. [53]

    Tan Zhang, Kefang Zhang, Jiatao Lin, Wing-Yue Geoffrey Louie, and Hui Huang. 2022. Sim2real Learning of Obstacle Avoidance for Robotic Manipulators in Uncertain Environments. IEEE Robotics and Automation Letters 7, 1 (2022), 65–72. doi:10.1109/LRA.2021.3116700

    2022

  46. [54]

    Ziyuan Zhou, Guanjun Liu, and Ying Tang. 2023. Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challenges. arXiv preprint arXiv:2305.10091 - (2023), –

    2023 arXiv

  47. [55]

    Changxi Zhu, Mehdi Dastani, and Shihan Wang. 2024. A Survey of Multi-Agent Deep Reinforcement Learning with Communication. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems (Auckland, New Zealand) (AAMAS ’24). International Foundat...

    2024

  48. [2020]

    Sim2real predictivity: Does evaluation in simulation predict real-world performance? IEEE Robotics and Automation Letters 5, 4 (2020), Manuscript submitted to ACM Learning To Communicate Over An Unknown Shared Network 21 6670–6677

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.