Pith. sign in

REVIEW 2 major objections 6 minor 2 cited by

Actions Speak Louder Than Words: Rate-Reward Trade-off in Markov Decision Processes

T0 review · 2 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that when an MDP controller embeds messages into its actions and a receiver reads them from the observed state sequence, the infinite-horizon rate-reward trade-off is exactly the value of a convex program over occupation…

desk verdict The convex-program formulation and Act2Comm are real contributions, but the proof of Theorem 1 as written covers only the no-feedback channel and leaves the flagship capacity formula under-proved for the paper's own model. read the letter →

arxiv 2502.03335 v3 pith:YD3I2JNT submitted 2025-02-05 cs.IT math.IT

Haotian Wu , Gongpu Chen , Deniz Gündüz This is my paper · ORCID
classification cs.ITmath.IT MSC 94A1594A1790C4090C25
keywords action-statechannelfinite-stateimplicitcommunicationcapacity-rewardtrade-offoccupationmeasureMarkovdecisionprocesscodingtransformer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper treats the state transitions of a Markov decision process as a communication channel: the controller's actions are the channel input and the states observed by another agent are the output. It claims that the maximum rate at which messages can be reliably sent this way, in the infinite-horizon regime, is the conditional mutual information $I(X; S^+ \mid S)$ maximized over stationary randomized policies—a single-letter formula. Under a constraint that the long-run average reward stay above $V$, the same rate becomes the value of a convex program over occupation measures, so the full rate-reward trade-off is a concave curve that can be computed numerically. The paper also introduces Act2Comm, a transformer-based feedback coding scheme for finite blocklengths that learns to embed messages into actions while approximating a target reward-achieving policy, and shows in three small MDPs that it trades reward for reliability in the predicted direction.

What carries the argument

The central objects are the action-state channel, where the MDP transition kernel $T(s' \mid s,x)$ acts as the channel law with the action as input and next state as output; the occupation measure $w(s,x) = \rho_\pi(s)\pi(x \mid s)$, which linearizes the reward constraint; and the function $I(w,T)$, a concave mutual-information functional on the occupation-measure polytope whose tangent lines are given explicitly by Lemma 2, yielding a closed-form gradient for ascent. The proof chain converts the channel to Shannon's extended action-state (EAS) channel, reduces history-dependent encoders to Markov encoders via Puterman's theorem, then invokes average-reward MDP theory to argue that a stationary deterministic policy attains the optimum. Act2Comm's machinery is a transformer encoder-decoder pair with block-attention feedback, a hard quantizer mapping beliefs to decision rules, and a critic network trained on noisy neighbors of the belief map to provide gradients through the non-differentiable channel.

What would settle it

Take a small MDP, for example a two-state, two-action variant of the paper's Lucky Wheel environment, and compare the maximum average mutual information over all history-dependent encoders, computed by dynamic programming over a long finite horizon, against the value of the convex program in Theorem 2. If any history-dependent or randomized time-varying encoder strictly exceeds the stationary deterministic optimum, or if the convex-program value cannot be approached by any code sequence with vanishing error probability, then Theorem 1's single-letter formula is an upper bound rather than the exact capacity.

Watch

Extended reading notes

Core claim

The central claim is that for a finite-state MDP observed by a receiver, the capacity of the action-state channel with reward constraint $V$ is $C(V) = \max_{w \in W} I(w,T)$ subject to $\sum_{s,x} w(s,x) r(s,x) \ge V$, where $W$ is the polytope of occupation measures and $I(w,T)$ is a concave function, making this a convex optimization (Theorem 2). Without the reward constraint, capacity equals $\max_{\pi} I(X; S^+ \mid S)$ over stationary policies (Theorem 1). The paper highlights that this implies history does not increase the infinite-horizon rate: a stationary randomized policy suffices to achieve capacity. The trade-off function $C(V)$ is concave, so the achievable region of rate-reward pairs is convex. The authors also claim that Act2Comm is a practical finite-blocklength policy that approaches this trade-off, as validated by bit-error-rate versus reward curves in three environments.

Load-bearing premise

The exact capacity formula collapses if the unichain assumption fails or if a history-dependent or randomized policy strictly outperforms every stationary deterministic policy for the communication objective—a real risk because the per-step reward in the dynamic-programming reduction is concave, not linear, in the action distribution.

Editorial extensions

If this is right

  • The rate-reward trade-off for communication through actions is computable: for any finite MDP the capacity under reward constraint $V$ can be found by convex optimization, so system designers can know the fundamental limit before choosing a coding scheme.
  • Because a stationary randomized policy achieves unconstrained capacity, infinite-horizon communication through actions requires no memory of past states or actions, a structural simplification relative to general finite-state channels.
  • The achievable region of rate-reward pairs is convex, so convex combinations of control-optimal and communication-oriented policies fill in intermediate points of the trade-off.
  • Act2Comm provides a finite-blocklength construction that empirically approaches the trade-off: lower reward constraints yield lower bit error rates at fixed rate, and lower rates yield higher reward at fixed bit error rate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the receiver could also observe the controller's actions, the same setting would become a source-coding problem in which messages are compressed into action sequences; the gap between that rate and the action-state channel capacity derived here would measure the value of action observability.
  • The convex program in Theorem 2 has a natural dual: its Lagrange multiplier prices the reward constraint, giving a marginal cost of communication that a system designer could use to decide how much control performance to sacrifice for each additional unit of rate.
  • Beyond the unichain assumption, one could formulate the same trade-off over convex hulls of recurrent-class occupation measures; the structure would likely persist, but the single-letter capacity formula would need re-derivation because the limiting state distribution need not be unique.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper studies implicit communication through actions in an MDP: the controller embeds messages into actions, and a receiver observes only the resulting state sequence. The MDP is modeled as a finite-state channel (specifically a POST channel) with output feedback and with the state available to the encoder. The main theoretical claims are Theorem 1, that the unconstrained capacity of this action-state channel is C = max over stationary randomized policies of I(X; S+ | S), and Theorem 2, that the capacity under an average-reward constraint V equals the optimal value of a convex program over occupation measures, C(V) = max_{w in W} I(w,T) subject to the linear reward constraint. The paper also proposes Act2Comm, a transformer-based finite-blocklength coding scheme with a critic network and iterative training, and reports BER-versus-reward experiments on three MDPs. Appendix B contains the proofs: Theorem 1 is derived through an extended action-state channel and a no-feedback FSC capacity formula, with reductions from history-dependent to Markov and stationary encoders; Theorem 2 is derived using occupation measures, a log-sum inequality, and a tangent-line lemma.

Significance. If the theoretical claims are fully established, the single-letter expression for the capacity of action-state channels and the convex characterization of the rate-reward trade-off would be a valuable information-theoretic result, providing a fundamental benchmark for implicit communication through actions in MDPs. The convexity proof is self-contained, and Lemma 2 gives a closed-form gradient that makes the convex program in Theorem 2 numerically tractable. Act2Comm is a substantial engineering contribution: it addresses a non-differentiable finite-state channel with a critic-based gradient estimator, supports block-feedback coding, and is evaluated across three environments with ablations on feedback, message length, and target policy. The main weakness is that the proof of Theorem 1 does not currently handle the output feedback explicitly present in the model, and the proof of Theorem 2 establishes only the upper-bound direction of the claimed equality. These gaps are load-bearing for the central claims and must be closed or made precise before the theoretical contribution can be accepted as stated.

major comments (2)
  1. [Appendix B.1, Eq. (14) and Section 3] The proof of Theorem 1 starts from Eq. (14), C(s1) = max_{p(u_i|u^{i-1})} lim (1/N) I(U^N; S_2^{N+1} | s1), which is the standard capacity formula for a finite-state channel without feedback. However, the system model in Section 3 explicitly includes output feedback: the encoder is defined as E_t : M x S^t x X^{t-1} x Y^{t-1} -> X, and Act2Comm's encoder is fed feedback blocks C^(tau). Under feedback, U_{i+1} may depend on S_{i+1}, so the equality labelled (a), H(S_{i+1} | S_2^i, U^N, X_i, s1) = H(S_{i+1} | S_i, U_i, s1), is not valid in general. The subsequent inequalities therefore bound the no-feedback capacity, not the capacity of the model studied, and the converse direction of Theorem 1 is missing. The paper cites Permuter et al. (2014) for the fact that this is a POST channel, but it never states which POST-channel theorem closes the gap. The authors must either prove that feedback does not increase capacity for this channel, invoke and state precisely a known result that does so, or revise the model and Theorem 1 to the no-feedback setting.
  2. [Theorem 2 and Appendix B.2] Theorem 2 asserts equality between C(V), the capacity under reward constraint V, and the value of the convex program max_{w in W} I(w,T) subject to sum w(s,x) r(s,x) >= V. Appendix B.2 proves only the unconstrained identity C = max_{w in W} I(w,T) and the concavity of I(w,T). It does not prove achievability for the constrained problem: for a fixed stationary occupation measure w with reward at least V, the paper never constructs a code family achieving rate arbitrarily close to I(w,T) with vanishing error probability and long-term average reward at least V. Since the reward constraint is linear in the occupation measure, the upper-bound direction is straightforward, but the reverse direction is essential for the claimed exact trade-off. The definition of C(V) itself, as an infinite-horizon limit of problem P3, is also never formally given; this should be stated before the equality in Theorem 2 can be assessed.
minor comments (6)
  1. [Section 3, Controller & Encoder paragraph] The domain of E_i is written as "M x S_i^i x X^{i-1}" (typeset as "M x S_i^i" or "M x S_i"), which appears to be a typo for the full state history S^i; in addition, to match the earlier FSC definition, the encoder domain should include the output feedback history Y^{i-1}.
  2. [Section 6 and Appendix D.3] For the "Catch the Ball" environment, the main text refers to the noisy case as p = 0.2, while Appendix D.3 sets the failure probability to p = 0.8; this inconsistency should be reconciled.
  3. [Appendix B.1, Eqs. (16)-(17)] The reduction from the time-varying DP in Eq. (16) to the stationary expression in Eq. (17) should state precisely which theorem of Fainberg or Hernandez-Lerma and Lasserre is being applied, and should verify its hypotheses explicitly, since the action space is the simplex of distributions and the per-step reward r(s_i, q_i(s_i)) is concave rather than linear in q_i.
  4. [Algorithm 2, line 9] There is a stray closing parenthesis in the line "x_t^(tau) = U^(tau)[s_t, t])"; this should be corrected.
  5. [Section 6, Figures 3-5] The experimental section does not plot the theoretical C(V) from Theorem 2; adding such a comparison would clarify how close Act2Comm's finite-blocklength curves come to the fundamental rate-reward bound.
  6. [Appendix D.4] The name "Act2cComm" appears in the first paragraph; this is a typo for Act2Comm.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the capacity-reward result is derived from standard FSC and MDP theory, with no fitted parameter or load-bearing self-citation.

full rationale

I walked the claimed derivation chain. Theorem 1 starts from the standard FSC multi-letter expression (eq. 14, cited to Gallager), then uses an entropy expansion, a reduction from history-dependent to Markov encoders via Puterman Theorem 5.5.1, and an average-reward DP (eq. 16) whose stationary-deterministic optimum is supplied by Fainberg and Hernandez-Lerma-Lasserre. The single-letter quantity I(X; S+|S) is derived, not assumed: it is the limit of the per-step mutual informations under the DP-optimal stationary policy. The unichain condition is an explicit modeling assumption, not a hidden duplicate of the capacity claim. Theorem 2 only changes coordinates: the one-to-one map between occupation measures W and stationary policies Pi_S rewrites the same objective, and the reward constraint enters as a linear inequality over W. Lemma 2's tangent line is a convex-analysis fact with no additional content smuggled in. Act2Comm is trained against an independently chosen target policy with a cross-entropy communication loss and an MSE control loss; its reported BER curves are not claimed to be predictions of Theorem 2, and no parameter of Theorem 2 is fitted to the experiments. The only references with overlapping authorship (e.g., Tung et al. 2021) appear in related work and are not load-bearing. The skeptic's concerns—feedback in the entropy step, possible randomized-policy optimality, and the unproved achievability of C(V)—are correctness/completeness gaps, not circular reductions: even if the proof is incomplete, the theorem's content is not equivalent to its inputs by construction. The appended correction note fixes a typo in the proof and does not assert circular dependence. Therefore no circular step is identified.

Assumptions & free parameters 5 free parameters · 7 assumptions · 0 invented entities

The theory rests on standard FSC/MDP machinery (causal coding limits, Markovification, average-reward DP existence) plus stated domain assumptions (unichain MDP, finite alphabets, receiver observes states only). No constants are fitted in the theory; the numerical claims live in Act2Comm's hyperparameters, which are hand-chosen rather than learned. The most fragile entries are the omitted achievability details for Theorems 1 and 2 and the deterministic-optimum assertion in the DP step.

free parameters (5)
  • Control loss weight lambda = tuned in [0.01, 20]
    Selects the operating point on the rate-reward-BER frontier in Section 6; the achievable regions of Figs. 3-5 are traced by sweeping lambda.
  • Soft-count temperature gamma = 10, 50, 100, 200
    Controls the sharpness of the action-frequency estimate f_hat in eq. (13); hand-chosen per experiment (Appendix D.1).
  • Critic noise variance sigma_w^2 = 0.1
    Neighboring sampling for critic training (Algorithm 1); hand-chosen.
  • Inner critic steps s_in = 20
    Number of critic updates per encoder update; hand-chosen (Appendix D.1).
  • Message block size mu = 3 bits
    Block partition in block-attention coding; hand-chosen and swept in Table 6.
assumptions (7)
  • domain assumption The MDP is unichain: every deterministic policy induces a single recurrent class plus transient states.
    Stated in Section 3 before P1; used in Appendix B.1 to ensure every stationary policy has a unique equilibrium distribution rho_pi and that stationary policies suffice for the average-reward DP.
  • standard math The capacity of an FSC with feedback and encoder state knowledge is given by the causal-coding limit of eq. (14).
    Standard FSC capacity theory (Gallager 1968); the starting point of the Theorem 1 proof in Appendix B.1.
  • standard math Puterman Theorem 5.5.1 (Markovification): any history-dependent policy's state-action marginals can be realized by a (time-inhomogeneous) Markov policy.
    Used in step (c) of the Theorem 1 proof (Appendix B.1) to justify restricting to Markov encoders.
  • standard math Existence of stationary optimal policies for average-reward MDPs with compact action spaces and continuous reward and transition functions (Fainberg 1976; Hernandez-Lerma and Lasserre 2012).
    Invoked in Appendix B.1 to move from eq. (16) to eq. (17). The paper additionally asserts a deterministic optimum, which is not automatic for the concave per-step reward.
  • standard math Achievability of the single-letter rate by i.i.d. random codes with per-state input distributions for the POST (action-state) channel.
    Theorem 1's achievability claim is asserted rather than proved from first principles; it relies on the POST channel structure (Permuter et al. 2014, cited at eq. (4)).
  • domain assumption The decoder knows the transition kernel T and the codebook statistics.
    The decoder in Section 3 maps state sequences to messages; optimal decoding assumes knowledge of channel law and code.
  • standard math The reward-constrained capacity C(V) is achievable: for any V and R < C(V) a code exists with reward at least V and rate R.
    This achievability direction of Theorem 2 is asserted ('provides an upper bound...'), but a full characterization requires the converse; the paper sketches neither the random-coding argument nor the typicality details.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Actions Speak Louder Than Words: Rate-Reward Trade-off in Markov Decision Processes." pith.science (2026). https://pith.science/paper/YD3I2JNT

@misc{pith2026250203335,
  author       = {Pith},
  title        = {Pith review of: Actions Speak Louder Than Words: Rate-Reward Trade-off in Markov Decision Processes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YD3I2JNT}},
  note         = {Machine review of arXiv:2502.03335}
}
read the original abstract

The impact of communication on decision-making systems has been extensively studied under the assumption of dedicated communication channels. We instead consider communicating through actions, where the message is embedded into the actions of an agent which interacts with the environment in a Markov decision process (MDP) framework. We conceptualize the MDP environment as a finite-state channel (FSC), where the actions of the agent serve as the channel input, while the states of the MDP observed by another agent (i.e., receiver) serve as the channel output. Here, we treat the environment as a communication channel over which the agent communicates through its actions, while at the same time, trying to maximize its reward. We first characterize the optimal information theoretic trade-off between the average reward and the rate of reliable communication in the infinite-horizon regime. Then, we propose a novel framework to design a joint control/coding policy, termed \textit{Act2Comm}, which seamlessly embeds messages into actions. From a communication perspective, \textit{Act2Comm} functions as a learning-based channel coding scheme for non-differentiable FSCs under input-output constraints. From a control standpoint, \textit{Act2Comm} learns an MDP policy that incorporates communication capabilities, though at the cost of some control performance. Overall, \textit{Act2Comm} effectively balances the dual objectives of control and communication in this environment. Experimental results validate \textit{Act2Comm}'s capability to enable reliable communication while maintaining a certain level of control performance.

Figures

Figures reproduced from arXiv: 2502.03335 by the authors.

Figure 1
Figure 1. From a standard finite-state channel to an action-state channel. policies is denoted by ΠSD. It is worth noting that the set of admissible policies for an MDP is not restricted to ΠSD. In general, a policy can be history-dependent, determining xt using all historical states and actions up to time t. Let ΠS and ΠH denote the sets of stationary (possibly randomized) and history-dependent policies, respectively. It is … view at source ↗
Figure 2
Figure 2. (Left) Workflow diagram of the Act2Comm scheme, with the dashed line indicating the gradient flow. (Right) Illustration of the iterative training strategy, incorporating a critic network. 5 ACT2COMM: A PRACTICAL CODING SCHEME This section presents Act2Comm, a learning-based practical coding scheme that balances both con￾trol and communication objectives. This framework assumes a pre-determined control policy π that … view at source ↗
Figure 3
Figure 3. Control-communication trade-off of Act2Comm in “Lucky Wheel”. as an unknown environment. Before each update of the encoder, a critic network is trained over sin inner steps to predict the logits for the neighbor belief maps of a given Z. For each inner step k, the network is trained to predict the corresponding logits Mˆk as Mˆ ck based on the neighbor belief maps sampled from Zk = Z + Wk, where Wk ∈ R |S|× k R ∼ N … view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Control-communication trade-off of Act2Comm in “Catch the Ball” with p = 0. -0.5 0 0.5 1 1.5 Reward 10-4 10-3 10-2 10-1 100 BER Optimal reward Achievable R=1/2 R=1/3 R=1/4 R=1/5 R=1/6 R=1/8 (a) BER v.s. Reward 0 0.1 0.2 0.3 0.4 0.5 Coding rate 0 0.5 1 1.5 Reward Target…
Figure 5
Figure 5. Figure 5: Control-communication trade-off of Act2Comm in “Catch the Ball” with p = 0.2. (see Appendix D for details). When p = 0, the action-state channel is perfect as each action can be reliably inferred from the resulting state transition, but it becomes noisy for p > 0. We f…
Figure 6
Figure 6. Figure 6: The action-state channel and the equivalent extended action-state channel. B TECHNICAL PROOFS This section presents the proofs of Section 4. B.1 PROOF OF THEOREM 1 The proof of Theorem 1 relies on converting the action-state channel to an equivalent channel. This equiv…
Figure 7
Figure 7. Figure 7: Illustration of the iterative training process: (a) Gradient flow in the proposed method, where blue arrows indicate the gradient flow, and red arrows represent the forward process. (b) Diagram of the update steps, where the red block represents the component being upd…
Figure 8
Figure 8. Figure 8: (a). Training process comparison for scalar state and vector state representations, where [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: Illustrations of experimental MDP environments is further transformed by a fully-connected layer into Z(τ) . The decoding process begins with re￾shaping S into S k µ × µ R . This reshaped S is then processed symmetrically through fully-connected layers, positional enco…
Figure 10
Figure 10. Figure 10: Control-communication trade-off of Act2Comm in “Erratic Robot”. penalties for the player. After a ball disappears or is caught, a new ball appears randomly (with equal probability) at one of the three positions at the top of the grid. The player has three available ac…
Figure 11
Figure 11. Figure 11: Loss during the training process, where the loss value decreases significantly with each decoder update, while showing a slight increase with each encoder update [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]
Figure 12
Figure 12. Figure 12: (a) Performance for different coding schemes. (b) Approaching a given policy [PITH_FULL_IMAGE:figures/full_fig_p027_12.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. KGRAG-SC: Knowledge Graph RAG-Assisted Semantic Communication

    eess.SP 2025-09 conditional novelty 6.0 of 10

    KGRAG-SC transmits only knowledge-graph entity IDs and reconstructs text with an LLM, improving low-SNR semantic fidelity and cutting transmitted bits versus text-based baselines.

  2. SREC: Encrypted Semantic Super-Resolution Enhanced Communication

    eess.SP 2025-09 reject novelty 4.0 of 10

    An encrypted semantic image transmission scheme that appends modulo-256 encryption and a super-resolution post-processor to NTSCC, evaluated on 25 Urban100 images.

Reference graph

Works this paper leans on

49 extracted references · 38 canonical work pages · cited by 2 Pith papers

  1. [1]

    Data-driven neural polar codes for unknown channels with and without memory

    Ziv Aharoni, Bashar Huleihel, Henry D Pfister, and Haim H Permuter. Data-driven neural polar codes for unknown channels with and without memory. arXiv preprint arXiv:2309.03148, 2023

    arXiv 2023

  2. [2]

    Constrained Markov decision processes

    Eitan Altman. Constrained Markov decision processes. Routledge, 2021

    2021

  3. [3]

    Nonlinear programming

    Dimitri Bertsekas. Nonlinear programming. Athena Scientific, 2016

    work page 2016

  4. [4]

    Proof of shannon's transmission theorem for finite-state indecomposable channels

    David Blackwell, Leo Breiman, and Aram J Thomasian. Proof of shannon's transmission theorem for finite-state indecomposable channels. The Annals of Mathematical Statistics, pp.\ 1209--1220, 1958

    work page 1958

  5. [5]

    A review of the applications of deep learning-based emergent communication

    Brendon Boldt and David R Mortensen. A review of the applications of deep learning-based emergent communication. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=jesKcQxQ7j

    work page 2024

  6. [6]

    Rgmcomm: Return gap minimization via discrete communications in multi-agent reinforcement learning

    Jingdi Chen, Tian Lan, and Carlee Joe-Wong. Rgmcomm: Return gap minimization via discrete communications in multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 17327--17336, 2024

    work page 2024

  7. [7]

    Elements of information theory

    Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999

    1999

  8. [8]

    Kamak Ebadi, Lukas Bernreiter, Harel Biggie, Gavin Catt, Yun Chang, Arghya Chatterjee, Christopher E. Denniston, Simon-Pierre Deschênes, Kyle Harlow, Shehryar Khattak, Lucas Nogueira, Matteo Palieri, Pavel Petráček, Matěj Petrlík, Andrzej Reinke, Vít Krátký, Shibo Zhao, Ali-akbar Agha-mohammadi, Kostas Alexis, Christoffer Heckman, Kasra Khosoussi, Navinda...

    work page arXiv 2024

Show all 49 references
  1. [9]

    On controlled finite state markov processes with compact control sets

    EA Fainberg. On controlled finite state markov processes with compact control sets. Theory of Probability & Its Applications, 20 0 (4): 0 856--862, 1976

    1976

  2. [10]

    Learning to Communicate with Deep Multi - Agent Reinforcement Learning

    Jakob Foerster, Ioannis Alexandros Assael, Nando de Freitas, and Shimon Whiteson. Learning to Communicate with Deep Multi - Agent Reinforcement Learning . In Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc., 2016. URL https://papers.nips.c...

    2016

  3. [11]

    Information theory and reliable communication, volume 588

    Robert G Gallager. Information theory and reliable communication, volume 588. Springer, 1968

    1968

  4. [12]

    Goldsmith and P.P

    A.J. Goldsmith and P.P. Varaiya. Capacity, mutual information, and coding for finite-state markov channels. IEEE Transactions on Information Theory, 42 0 (3): 0 868--886, 1996. doi:10.1109/18.490551

    1996 doi

  5. [13]

    Deeppolar: Inventing nonlinear large-kernel polar codes via deep learning

    S Ashwin Hebbar, Sravan Kumar Ankireddy, Hyeji Kim, Sewoong Oh, and Pramod Viswanath. Deeppolar: Inventing nonlinear large-kernel polar codes via deep learning. arXiv preprint arXiv:2402.08864, 2024

    2024 arXiv

  6. [14]

    Discrete-time Markov control processes: basic optimality criteria, volume 30

    On \'e simo Hern \'a ndez-Lerma and Jean B Lasserre. Discrete-time Markov control processes: basic optimality criteria, volume 30. Springer Science & Business Media, 2012

    2012

  7. [15]

    Turbo autoencoder: Deep learning based channel codes for point-to-point communication channels

    Yihan Jiang, Hyeji Kim, Himanshu Asnani, Sreeram Kannan, Sewoong Oh, and Pramod Viswanath. Turbo autoencoder: Deep learning based channel codes for point-to-point communication channels. Advances in neural information processing systems, 32, 2019

    2019

  8. [16]

    Learn codes: Inventing low-latency codes via recurrent neural networks

    Yihan Jiang, Hyeji Kim, Himanshu Asnani, Sreeram Kannan, Sewoong Oh, and Pramod Viswanath. Learn codes: Inventing low-latency codes via recurrent neural networks. IEEE Journal on Selected Areas in Information Theory, 1 0 (1): 0 207--216, 2020

    2020

  9. [17]

    Karabag, Melkior Ornik, and Ufuk Topcu

    Mustafa O. Karabag, Melkior Ornik, and Ufuk Topcu. Least inferable policies for markov decision processes. In 2019 American Control Conference (ACC), pp.\ 1224--1231, 2019. doi:10.23919/ACC.2019.8815129

    2019

  10. [18]

    Deepcode: Feedback codes via deep learning

    Hyeji Kim, Yihan Jiang, Sreeram Kannan, Sewoong Oh, and Pramod Viswanath. Deepcode: Feedback codes via deep learning. Advances in neural information processing systems, 31, 2018

    2018

  11. [19]

    Deepcode: Feedback codes via deep learning

    Hyeji Kim, Yihan Jiang, Sreeram Kannan, Sewoong Oh, and Pramod Viswanath. Deepcode: Feedback codes via deep learning. IEEE Journal on Selected Areas in Information Theory, 1 0 (1): 0 194--206, 2020

    2020

  12. [20]

    Knepper, Christoforos I

    Ross A. Knepper, Christoforos I. Mavrogiannis, Julia Proft, and Claire Liang. Implicit communication in a joint action. In 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI, pp.\ 283--292, 2017

    2017

  13. [21]

    Joint source-channel coding with feedback

    Victoria Kostina, Yury Polyanskiy, and Sergio Verd. Joint source-channel coding with feedback. IEEE Transactions on Information Theory, 63 0 (6): 0 3502--3515, 2017

    2017

  14. [22]

    Ko codes: inventing nonlinear encoding and decoding for reliable wireless communication via deep-learning

    Ashok V Makkuva, Xiyang Liu, Mohammad Vahid Jamali, Hessam Mahdavifar, Sewoong Oh, and Pramod Viswanath. Ko codes: inventing nonlinear encoding and decoding for reliable wireless communication via deep-learning. In International Conference on Machine Learning, pp.\ 7368--7378....

    2021

  15. [23]

    Survey of unmanned subterranean exploration, navigation, and localisation

    Jeffrey Martz, Wesam Al-Sabban, and Ryan N Smith. Survey of unmanned subterranean exploration, navigation, and localisation. IET Cyber-Systems and Robotics, 2 0 (1): 0 1--13, 2020

    2020

  16. [24]

    Causality, feedback and directed information

    James Massey. Causality, feedback and directed information. In Proc. Int. Symp. Inf. Theory Applic.(ISITA-90), pp.\ 303--305, 1990

    1990

  17. [25]

    Capacity of a post channel with and without feedback

    Haim Henri Permuter, Himanshu Asnani, and Tsachy Weissman. Capacity of a post channel with and without feedback. IEEE Transactions on Information Theory, 60 0 (10): 0 6041--6057, 2014. doi:10.1109/TIT.2014.2343232

    2014

  18. [26]

    Goldsmith

    Haim Henry Permuter, Tsachy Weissman, and Andrea J. Goldsmith. Finite state channels with time-invariant deterministic feedback. IEEE Transactions on Information Theory, 55 0 (2): 0 644--662, 2009. doi:10.1109/TIT.2008.2009849

    2009

  19. [27]

    Jamming attacks and anti-jamming strategies in wireless networks: A comprehensive survey

    Hossein Pirayesh and Huacheng Zeng. Jamming attacks and anti-jamming strategies in wireless networks: A comprehensive survey. IEEE Communications Surveys & Tutorials, 24 0 (2): 0 767--809, 2022. doi:10.1109/COMST.2022.3159185

    2022

  20. [28]

    Vincent Poor and Rafael F

    H. Vincent Poor and Rafael F. Schaefer. Wireless physical layer security. Proceedings of the National Academy of Sciences, 114 0 (1): 0 19--26, 2017. doi:10.1073/pnas.1618130114. URL https://www.pnas.org/doi/abs/10.1073/pnas.1618130114

    2017 doi

  21. [29]

    Markov decision processes: discrete stochastic dynamic programming

    Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014

    2014

  22. [30]

    Permuter, and Henry D

    Oron Sabag, Haim H. Permuter, and Henry D. Pfister. A single-letter upper bound on the feedback capacity of unifilar finite-state channels. IEEE Transactions on Information Theory, 63 0 (3): 0 1392--1409, 2017. doi:10.1109/TIT.2016.2636851

    2017

  23. [31]

    Channels with side information at the transmitter

    Claude E Shannon. Channels with side information at the transmitter. IBM journal of Research and Development, 2 0 (4): 0 289--293, 1958

    1958

  24. [32]

    Finite-state channels with feedback and state known at the encoder

    Eli Shemuel, Oron Sabag, and Haim H Permuter. Finite-state channels with feedback and state known at the encoder. arXiv preprint arXiv:2212.12886, 2022

    2022 arXiv

  25. [33]

    Permuter

    Eli Shemuel, Oron Sabag, and Haim H. Permuter. Finite-state channels with feedback and state known at the encoder. IEEE Transactions on Information Theory, 70 0 (3): 0 1610--1628, 2024. doi:10.1109/TIT.2023.3336939

    2024

  26. [34]

    Communicating via markov decision processes

    Samuel Sokota, Christian A Schroeder De Witt, Maximilian Igl, Luisa M Zintgraf, Philip Torr, Martin Strohmeier, Zico Kolter, Shimon Whiteson, and Jakob Foerster. Communicating via markov decision processes. In International Conference on Machine Learning, pp.\ 20314--20328. PMLR, 2022

    2022

  27. [35]

    Learning multiagent communication with backpropagation

    Sainbayar Sukhbaatar, arthur szlam, and Rob Fergus. Learning multiagent communication with backpropagation. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 29. Curran Associates, Inc., 2016. URL htt...

    2016

  28. [36]

    Learning to communicate implicitly by actions

    Zheng Tian, Shihao Zou, Ian Davies, Tim Warr, Lisheng Wu, Haitham Bou Ammar, and Jun Wang. Learning to communicate implicitly by actions. arXiv:cs.AI:1810.04444, 2019. URL https://arxiv.org/abs/1810.04444

    2019 arXiv

  29. [37]

    Trenholm

    S. Trenholm. Thinking Through Communication: An Introduction to the Study of Human Communication (9th ed.). Routledge, 2020. doi:10.4324/9781003016366

    2020 doi

  30. [38]

    Effective Communications : A Joint Learning and Communication Framework for Multi - Agent Reinforcement Learning Over Noisy Channels

    Tze-Yang Tung, Szymon Kobus, Joan Pujol Roig, and Deniz Gündüz. Effective Communications : A Joint Learning and Communication Framework for Multi - Agent Reinforcement Learning Over Noisy Channels . IEEE Journal on Selected Areas in Communications, 39 0 (8): 0 2590--2603, Augu...

    2021

  31. [39]

    Verdu and Te Sun Han

    S. Verdu and Te Sun Han. A general formula for channel capacity. IEEE Transactions on Information Theory, 40 0 (4): 0 1147--1157, 1994. doi:10.1109/18.335960

    1994 doi

  32. [40]

    Pheromone communication in ants: a detailed analysis of concentration-dependent decisions in three species

    Wolfhard von Thienen, Dirk Metzler, Dong-Hwan Choe, and Volker Witte. Pheromone communication in ants: a detailed analysis of concentration-dependent decisions in three species. Behavioral ecology and sociobiology, 68: 0 1611--1627, 2014

    2014

  33. [41]

    Towards practical and scalable molecular networks

    Jiaming Wang, Sevda \" O g \" u t, Haitham Al Hassanieh, and Bhuvana Krishnaswamy. Towards practical and scalable molecular networks. In Proceedings of the ACM SIGCOMM 2023 Conference, ACM SIGCOMM '23, pp.\ 62–76. Association for Computing Machinery, 2023. doi:10.1145/3603269.3604881

    2023

  34. [42]

    Learning Efficient Multi -agent Communication : An Information Bottleneck Approach

    Rundong Wang, Xu He, Runsheng Yu, Wei Qiu, Bo An, and Zinovi Rabinovich. Learning Efficient Multi -agent Communication : An Information Bottleneck Approach . In Proceedings of the 37th International Conference on Machine Learning , pp.\ 9908--9918. PMLR, November 2020. URL htt...

    2020

  35. [43]

    Quorum sensing: cell-to-cell communication in bacteria

    Christopher M Waters and Bonnie L Bassler. Quorum sensing: cell-to-cell communication in bacteria. Annual Review of Cell and Developmental Biology, 21 0 (1): 0 319--346, 2005

    2005

  36. [44]

    Ron Weiss and Thomas F. Knight. Engineered communications for microbial robotics. In Anne Condon and Grzegorz Rozenberg (eds.), DNA Computing, pp.\ 1--16, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg. ISBN 978-3-540-44992-8

    2001

  37. [45]

    Lookahead optimizer: k steps forward, 1 step back

    Michael Zhang, James Lucas, Jimmy Ba, and Geoffrey E Hinton. Lookahead optimizer: k steps forward, 1 step back. Advances in neural information processing systems, 32, 2019

    2019

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  39. [47]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  40. [48]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  41. [49]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.