Self-information

By definition, the amount of self-information contained in a probabilistic Event depends only on the Probability

p

of that event. More specifically: the smaller this probability is, the larger is the self-information associated with receiving information that the event indeed occurred.

Further, by definition, the Measure of self-information has the following property. If an event ''C'' is composed of two mutually Independent events ''A'' and ''B'', then the amount of information at the proclamation that ''C'' has happened, equals the sum of the amounts of information at proclamations of event ''A'' and event ''B'' respectively.

Taking into account these properties, the self-information

I(A_n)

(measured in Bit s) associated with outcome

A_n

whose outcome has probability

p

is Defined as:

:

I(A_n) = \log_2 \left(rac{1}{p(A_n)} 
ight) = - \log_2(p(A_n))

This definition, using the Binary Logarithm function, complies with the above conditions.

This measure has also been called surprisal, as it represents the " Surprise " of seeing the outcome (a certain outcome is not surprising). This term was coined by Myron Tribus in his 1961 book ''Thermostatics and Thermodynamics''. Some claim it is more accurate than "self-information", but it has not been widely used.

INFORMATION ENTROPY

The concept is related to that of Information Entropy ; the information entropy of a random event

A

is the Expected Value of its self-information:

:

H(A)=\sum_{i=1}^np(A_i)\log_2 \left(rac{1}{p(A_i)}
ight)

EXAMPLES

On Tossing A Coin , the chance of 'tail' is 0.5. When it is proclaimed that indeed 'tail' occurred, this amounts to

I

When throwing a Die , the probability of 'four' is 1/6. When there is proclaimed that 'four' has been thrown, the amount of self-information is

I

When, independently, two dice are thrown, the amount of information associated with {throw 1 = 'two' & throw 2 = 'four'} equals

I

EXTERNAL LINKS

Surprisals, from excitation to code

Examples of surprisal measures

Entropy and surprisal

"Surprisal" entry in a glossary of molecular information theory

	APPAREL
	BABY
	BEAUTY
	BOOKS
	CAR TOYS
	CELL PHONES
	DVD'S
	ELECTRONICS
	GOURMET FOOD
	GROCERIES
	HEALTH & PERSONAL
	HOME & GARDEN
	JEWELRY
	MUSIC
	MUSIC INSTRUMENTS
	OFFICE PRODUCTS
	SOFTWARE
	SPORTING GOODS
	TOOLS & HARDWARE
	TOYS
	VIDEO GAMES
	SHOPPING HOME

	MORE SHOPPING...

	CATEGORIES ABOUT SELF-INFORMATION

	entropy and information

Information About

Self-information