Question

2) In the game Scrabble, a player has a tray of 7 letter tiles. Consider the word 7-letter word SCIENCE

and answer the following questions.

a. Calculate the probability of randomly selecting each unique letter from this word.

b. Using those probabilities, calculate the entropy of this set of letters.

c. What would be the reduction in entropy (i.e., the information gain), if you split these

letters into two sets, one containing the vowels and one containing the consonants?

d. What is the maximum possible entropy in a set of 7 Scrabble tiles?

e. In general, which is preferable when you are playing Scrabble: a set of letters with high

entropy, or a set of letters with low entropy?

Answer 1

a. There are total 3 unique letters out of 7. So, probability will be:

b. Formula to calculate entropy using probability is:

So, here in our case we have only one probability, so:

Entropy =

c.) When we divide the letters in 2 set, we get 2 set as {SCNC) and {IEE}

So probability of choosing unique letters from each set individually is:

Now the overall entropy of the system becomes:

Entropy has increased.

e.) The one with low entropy is much preferable since entropy is the measurement of uncertainty. So, if the system will be less uncertain, the better overall performance it will give.