Formal Limitations on the Measurement of Mutual Information

Code

• github.com/karlstratos/doe
  OfficialIn paperpytorch★ 0
• github.com/createamind/keras-cpcgan
  tf★ 0

Abstract

Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confidence lower bound on mutual information estimated from N samples cannot be larger than O(ln N ).

Reproductions