Quantification of Information

Shannon's Examples of Artificial Information Sources


1. FIVE LETTERS {A, B, C, D, E} WITH EQUAL PROBABILITIES
Typical message:
BDCBCECCCADCBDDAAECEEAABBDAEECACEEBAEECBCEAD
 
2. FIVE LETTERS {A, B, C, D, E} WITH PROBABILITIES {0.4, 0.1, 0.2, 0.2, 0.1}
Typical message:
AAACDCBDCEAADADACEDAEADCABEDADDCECAAAAAD
 
3. THREE LETTERS {A, B, C} SPECIFIED BY DIGRAM PROBABILITIES











 
              
a
 b
 c
a
 0 4/5 1/5
b
 1/2 1/2 0
c
 1/2 2/5 1/10


a
 9/27
b
 16/27
c
 2/17











 
             

a
 b
 c
a
 0 4/15 1/15
b
 8/27 8/27 0
c
 1/27 4/135 1/135






Typical message:
ABBABABABABABABBBABBBBBBABABABABABBBACACABBABBBBABBABACBBBABA