Follow
TIL (somewhat embarrassingly) that https://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence provides a connection of sorts between mutual information and KL divergence
TIL (somewhat embarrassingly) that https://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence provides a connection of sorts between mutual information and KL divergence
To be more precise: the interesting thing is that one can interpret D_JS(A||B) as a mutual information between something for any pair of distributions. (One can do the inverse with D_KL: I(A;B) for any two variables can be interpreted as D_KL between some two distributions.)