Information AboutInformation Content |
| CATEGORIES ABOUT SELF-INFORMATION | |
| entropy and information | |
|
By definition, the amount of self-information contained in a probabilistic Event depends only on the Probability of that event: the smaller its probability, the larger the self-information associated with receiving the information that the event indeed occurred. Further, by definition, the Measure of self-information has the following property. If an event ''C'' is composed of two mutually Independent events ''A'' and ''B'', then the amount of information at the proclamation that ''C'' has happened, equals the sum of the amounts of information at proclamations of event ''A'' and event ''B'' respectively. Taking into account these properties, the self-information (measured in Bit s) associated with outcome is: : This definition, using the Binary Logarithm function, complies with the above conditions. This measure has also been called surprisal, as it represents the " Surprise " of seeing the outcome (a highly probable outcome is not surprising). This term was coined by Myron Tribus in his 1961 book ''Thermostatics and Thermodynamics''. The Information Entropy of a random event is the Expected Value of its self-information. Self-information is an example of a Proper Scoring Rule . EXAMPLES
: ''I''('tail') = log2 (1/0.5) = log2 2 = 1 bits of information.
I I REFERENCES EXTERNAL LINKS |
|
|