Central Limit Theorem

Article Index for Central

Limousines in Central

Website Links For Central

Information About Central Limit Theorem

	CATEGORIES ABOUT CENTRAL LIMIT THEOREM
	probability theory
	mathematical theorems
	statistics
	articles containing proofs

	APPAREL
	BABY
	BEAUTY
	BOOKS
	CAR TOYS
	CELL PHONES
	DVD'S
	ELECTRONICS
	GOURMET FOOD
	GROCERIES
	HEALTH & PERSONAL
	HOME & GARDEN
	JEWELRY
	MUSIC
	MUSIC INSTRUMENTS
	OFFICE PRODUCTS
	SOFTWARE
	SPORTING GOODS
	TOOLS & HARDWARE
	TOYS
	VIDEO GAMES
	SHOPPING HOME
	MORE SHOPPING...

Since many real processes yield distributions with finite Variance , this explains the ubiquity of the normal probability distribution.

Then the distribution of ''Z''_''n'' Converges towards the Standard Normal Distribution N(0,1)
as ''n'' approaches ∞ (this is of N(0,1), then for every Real Number ''z'', we have

:$\backslash lim\_\{n\; o\; \backslash infty\}\; \backslash mbox\{P\}(Z\_n\; \backslash le\; z)\; =\; \backslash Phi(z),$
or, equivalently,

:
where
:$\backslash overline\{X\}\_n=S\_n/n=(X\_1+\backslash cdots+X\_n)/n$
is the Sample Mean .


Proof of the central limit theorem

For a theorem of such fundamental importance to Statistics and Applied Probability , the central limit theorem has a remarkably simple proof using Characteristic Functions . It is similar to the proof of a (weak) Law Of Large Numbers . For any random variable, ''Y'', with zero Mean and unit variance (var(''Y'') = 1), the characteristic function of ''Y'' is, by Taylor's Theorem ,

:

where ''o'' (''t²'' ) is " Little O Notation " for some function of ''t''  that goes to zero more rapidly than ''t²''. Letting ''Y''_''i'' be (''X''_''i'' − μ)/σ, the standardised value of ''X''_''i'', it is easy to see that the standardised mean of the observations ''X₁'', ''X₂'', ..., ''X_n'' is just

:

By simple properties of characteristic functions, the characteristic function of ''Z''_''n'' is

:
\over 2n} + o\left({t^2 \over n} ight) ight]^n \, ightarrow \, e^{-t^2/2}, \quad n ightarrow \infty.

But, this limit is just the characteristic function of a standard normal distribution, N(0,1), and the central limit theorem follows from the Lévy Continuity Theorem , which confirms that the Convergence of characteristic functions implies convergence in distribution.


Convergence to the limit

If the third central Moment E((''X''₁ − μ)³) exists and is finite, then the above convergence is Uniform and the speed of convergence is at least on the order of 1/''n''^½ (see Berry-Esséen Theorem ).

The convergence normal is monotonic, in the sense that the Entropy of $Z\_n$ increases Monotonically to that of the normal distribution, as proven by Artstein, Ball, Barthe and Naor.

Pictures of a distribution being "smoothed out" by Summation (showing original Density Of Distribution and three subsequent summations, obtained by Convolution of density functions):



An equivalent formulation of this limit theorem starts with ''A''_''n'' = (''X''₁ + ... + ''X''_''n'') / ''n'' which can be interpreted as the mean of a Random Sample of size ''n''. The expected value of ''A''_''n'' is μ and the standard deviation is σ / ''n''^½. If we standardize ''A''_''n'' by setting ''Z''_''n'' = (''A''_''n'' - μ) / (σ / ''n''^½), we obtain the same variable ''Z''_''n'' as above, and it approaches a standard normal distribution.

The Central Limit Theorem, as an approximation for a finite number of observations, provides a reasonable approximation only when close to the peak of the normal distribution; it requires a very large number of observations to stretch into the tails.

The Central Limit theorem also applies to sums of independent and identical s is still a Discrete Random Variable , so that we are confronted to a Series of Discrete Random Variable s whose probability distribution converges towards a Probability Density Function corresponding to a continuous variable (namely the Normal Distribution ). This means that if we build a Histogram of the realisations of the sum of ''n'' independent identical discrete variables, the curve that joins the centers of the upper faces of the rectangles forming the histogram converges toward a gaussian curve as ''n'' approaches $\backslash infty$. The Binomial Distribution article details such an application of the central limit theorem in the simple case of a discrete variable taking only two possible values.


Relation to the law of large numbers

is one of the most popular tools employed to approach such questions.

Suppose we have an asymptotic expansion of ''f(n)'':



dividing both parts by and taking the limit will produce $a\_\{1\}$ - the coefficient at the highest-order term in the expansion representing the rate at which $f(n)$ changes in its leading term.



Informally, one can say: "$f(n)$ grows approximately as ". Taking the difference between $f(n)$ and its approximation and then dividing by the next term in the expansion we arrive to a more refined statement about $f(n)$:



here one can say that: "the difference between the function and its approximation grows approximately as " The idea is that dividing the function by appropriate normalizing functions and looking at the limiting behavior of the result can tell us much about the limiting behavior of the original function itself.

Informally, something along these lines is happening when ''S''_''n'' is being studied in classical probability theory. Under certain regularity conditions, by The Law of Large Numbers, and by The Central Limit Theorem,
where $\backslash xi$ is distributed as $N(0,\backslash sigma^2)$ which provide values of first two constants in informal expansion:

$S\_n\; \backslash approx\; \backslash mu\; n+\backslash xi\; \backslash sqrt\{n\}.$