PFR

The conditional mutual information step of sum_of_rdist_eq

Let $Y_1,Y_2,Y_3$ and $Y_4$ be independent $G$-valued random variables. Then $d[Y_1-Y_3; Y_2-Y_4] + d[Y_1|Y_1-Y_3; Y_2|Y_2-Y_4]$ $+ I[Y_1-Y_2 : Y_2 - Y_4 | Y_1-Y_2-Y_3+Y_4] = d[Y_1; Y_2] + d[Y_3; Y_4].$

The sum of $d[X_1+\tilde X_2;X_2+\tilde X_1] + d[X_1|X_1+\tilde X_2; X_2|X_2+\tilde X_1]$ and $I[X_1+ X_2 : \tilde X_1 + X_2 \,|\, X_1 + X_2 + \tilde X_1 + \tilde X_2]$ is equal to $2k$.

We have $I_1 \leq 2 \eta k$

If A is independent from B, then conditioning on an event given by B does not change the distribution of A.

If A is independent of B, then they remain independent when conditioning on an event of the form A ∈ s of positive probability.

If A is independent of B, then they remain independent when conditioning on an event of the form A ∈ s ∩ B ∈ t of positive probability.

Composing independent functions with a measurable embedding of conull range gives independent functions.

For X, Y random variables, there exist conditionally independent trials X_1, X_2, Y'.

For X, Y random variables, there exist conditionally independent trials X₁, X₂, Y'.

If X is uniformly distributed on H, then H[X] = log |H|.

If X is S-valued random variable, then H[X] = log |S| if and only if X is uniformly distributed.

If X is an S-valued random variable, then there exists s ∈ S such that P[X = s] ≥ \exp(- H[X]).

TODO: remove the probability measure hypothesis, which is unnecessary here.

If X is an S-valued random variable, then there exists s ∈ S such that P[X=s] ≥ \exp(-H[X]).

If X is an S-valued random variable of non-positive entropy, then X is almost surely constant.

Conditional entropy of a random variable is equal to the entropy of its conditional kernel.

Conditional entropy is at most the logarithm of the cardinality of the range.