The reviewed record of science sign in
Pith

arxiv: 1811.04251 · v4 · pith:FOYMECHR · submitted 2018-11-10 · cs.IT · cs.LG· math.IT· stat.ML

Formal Limitations on the Measurement of Mutual Information

David McAllester , Karl Stratos This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:FOYMECHRrecord.jsonopen to challenge →

classification cs.IT cs.LGmath.ITstat.ML
keywords informationmutualboundlimitationslowermeasuringcannotconsidered
0
0 comments X
read the original abstract

Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confidence lower bound on mutual information estimated from N samples cannot be larger than O(ln N ).

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Dream to Control: Learning Behaviors by Latent Imagination

    cs.LG 2019-12 accept novelty 7.0

    Dreamer learns to control from images by imagining and optimizing behaviors in a learned latent world model, outperforming prior methods on 20 visual tasks in data efficiency and final performance.