Quantification of Information

Entropy of English: Word Models reference


  1. First-order word model. (The words are chosen independent, but with frequencies found in English text.)
      2.  
      REPRESENTING AND SPEEDILY IS AN GOOD APT OR COME CAN DIFFERENT NATURAL
      HERE HE THE A IN CAME THE TO      OF TO EXPERT GRAY COME TO FURNISHES THE LINE
      MESSAGE HAD BE THESE.

 

  1. Second-order word model. (The word transition probabilities match those found in English text.)
      2.  
      THE HEAD AND IN FRONTAL ATTACK ON AN ENGLISH WRITER THAT THE CHARACTER
      OF THIS POINT IS       THEREFORE ANOTHER METHOD FOR THE LETTERS THAT
      THE       TIME OF WHO EVER TOLD THE PROBLEM FOR AN UNEXPECTED.

 

reference
Shanon examples taken from Elements of Information Theory, Thomas M. Cover and Joy A. Thomas, John Wiley & Sons, New York, (1991) ISBN 0-471-06259-6