2019-09 Discrete entropy

Suppose that \( X \) is a discrete random variable on the set \( \{ a_1, a_2, \dots \} \) with \( P(X=a_i) = p_i \). Define the discrete entropy
\[
H(X) = -\sum_{n=1}^{\infty} p_i \log p_i.
\]
Find constants \( C_1, C_2 \geq 0 \) such that
\[
e^{2H(X)} \leq C_1 Var(X) + C_2
\]
holds for any \( X \).

GD Star Rating
loading...
2019-09 Discrete entropy, 2.9 out of 5 based on 9 ratings