Errors And Residuals In Statistics

''Error'' is a misnomer; an '''error''' is the amount by which an observation differs from its Expected Value ; the latter being based on the whole population from which the statistical unit was chosen randomly. The expected value, being the average of the entire population, is typically unobservable. If the average height in a population of 21-year-old men is 5 feet 9 inches, and one randomly chosen man is 5 feet 11 inches tall, then the "error" is 2 inches; if the randomly chosen man is 5 feet 7 inches tall, then the "error" is −2 inches. The nomenclature arose from random Measurement Errors in Astronomy . It is as if the measurement of the man's height were an attempt to measure the population average, so that any difference between the man's height and the average would be a measurement error.

A residual, on the other hand, is an observable ''estimate'' of the unobservable error. The simplest case involves a random sample of ''n'' men whose heights are measured. The '' Sample '' average is used as an estimate of the '' Population '' average. Then we have:

The difference between the height of each man in the sample and the unobservable ''population'' average is an ''error'', and

The difference between the height of each man in the sample and the observable ''sample'' average is a ''residual''.

:''Residuals are observable; errors are not''.

Note that the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily ''not Independent ''. The sum of the errors need not be zero; the errors are independent Random Variable s if the individuals are chosen from the population independently.

:''Errors are often independent of each other; residuals are ''not'' independent of each other'' (at least in the simple situation described above, and in many others).

EXAMPLE WITH SOME MATHEMATICAL THEORY

If we assume a Normally Distributed population with mean μ and Standard Deviation σ, and choose individuals independently, then we have

:

X_1, \dots, X_n\sim N(\mu,\sigma^2)\,

and the Sample Mean

:

\overline{X}={X_1 + \cdots + X_n \over n}

is a random variable distributed thus:

:

\overline{X}\sim N(\mu, \sigma^2/n).

The ''errors'' are then

:

arepsilon_i=X_i-\mu,\,

whereas the ''residuals'' are

:

\widehat{arepsilon}_i=X_i-\overline{X}.

(As is often done, the "hat" over the letter ε indicates an observable ''estimate'' of an unobservable quantity called ε.)

The sum of squares of the errors, divided by σ², has a Chi-square Distribution with ''n'' degrees of freedom:

:

\sum_{i=1}^n \left(X_i-\mu
ight)^2/\sigma^2\sim\chi^2_n.

This quantity, however, is not observable. The sum of squares of the residuals, on the other hand, is observable. The quotient of that sum by σ² has a chi-square distribution with only ''n'' − 1 degrees of freedom:

:

\sum_{i=1}^n \left(\,X_i-\overline{X}\,
ight)^2/\sigma^2\sim\chi^2_{n-1}.

It is remarkable that the Sum Of Squares Of The Residuals and the sample mean can be shown to be independent of each other. That fact and the normal and chi-square distributions given above form the basis of Confidence Interval calculations relying on Student's T-distribution . In those calculations one encounters the quotient

:

{\overline{X}_n - \mu \over S_n/\sqrt{n}},

in which the σ appears in both the numerator and the denominator and cancels. That is fortunate because in practice one would not know the value of σ².

REFERENCES

''Residuals and Influence in Regression'', R. Dennis Cook, New York : Chapman And Hall , 1982.

	CATEGORIES ABOUT ERRORS AND RESIDUALS IN STATISTICS

	statistical theory

	statistical deviation and dispersion

	error

	measurement

	APPAREL
	BABY
	BEAUTY
	BOOKS
	CAR TOYS
	CELL PHONES
	DVD'S
	ELECTRONICS
	GOURMET FOOD
	GROCERIES
	HEALTH & PERSONAL
	HOME & GARDEN
	JEWELRY
	MUSIC
	MUSIC INSTRUMENTS
	OFFICE PRODUCTS
	SOFTWARE
	SPORTING GOODS
	TOOLS & HARDWARE
	TOYS
	VIDEO GAMES
	SHOPPING HOME

	MORE SHOPPING...

Information About

Errors And Residuals In Statistics