Question

The RDA Corporation has a prison with many cells. Without justification, you're about to be randomly thrown into a cell with equal probability. Cells 1 to 3 have Toruks that eat prisoners. Cells 4 to 6 are safe. With sufficient bribe, the warden will answer your question "Will I be in cell 1?" What's the mutual information (we call it information gain) between the warden's answer and your encounter with the Toruks?

Answer 1

The information gain here for you is the change in probability fo encountering a Toruk and getting eaten.

Now without any information from the warden, you probability of getting eaten =

Pr(you are put in cell 1) + Pr(you are put in cell 2) + Pr(you are put in cell 3)

= 1/6 + 1/6 + 1/6 = 3/6 = 1/2 0.5

Now, if the warden answers the question as yes i.e. you will be put in cell 1, there is no uncertainty left and you know that you will encounter a Toruk

If the warden answers the question as no, then things are different. Now, the probability fo encountering a Toruk

=  Pr(you are put in cell 2) + Pr(you are put in cell 3)

= 1/5 + 1/5 = 2/5 = 0.4

Thus, the information gain here is that you know the probaility of encountering a Toruk and getting eaten has reduced from 0.5 to 0.4.