Typical Set

This has great use in Compression theory as it provides a theoretical means for compressing data, allowing us to represent any sequence

X^n

using

nH(X)

bits on average, and, hence, justifying the use of entropy as a measure of information from a source.

The AEP can also be proven for a large class of Stationary Ergodic Process es, allowing typical set to be defined in more general cases.

(WEAKLY) TYPICAL SEQUENCES

If a sequence ''x''₁, ..., ''x''_''n'' is drawn from an I.i.d. Distribution

X

then the typical set,

{A_\epsilon}^{(n)}

is defined as those sequences which satisfy:

:

2^{-n(H(X)+\epsilon)} \leq p(x_1, x_2, ..., x_n) \leq 2^{-n(H(X)-\epsilon)}

The probability above need only be within a factor of

2^{n\epsilon}

.

It has the following properties if ''n'' is sufficiently large, ε can be chosen arbitrarily small so that:
#The probability of a sequence from

X

being drawn from

{A_\epsilon}^{(n)}

is greater than

1-\epsilon

	APPAREL
	BABY
	BEAUTY
	BOOKS
	CAR TOYS
	CELL PHONES
	DVD'S
	ELECTRONICS
	GOURMET FOOD
	GROCERIES
	HEALTH & PERSONAL
	HOME & GARDEN
	JEWELRY
	MUSIC
	MUSIC INSTRUMENTS
	OFFICE PRODUCTS
	SOFTWARE
	SPORTING GOODS
	TOOLS & HARDWARE
	TOYS
	VIDEO GAMES
	SHOPPING HOME

	MORE SHOPPING...

	CATEGORIES ABOUT TYPICAL SET

	information theory

	probability theory

Information About

Typical Set