| Local Regression |
Article Index for Local |
Website Links For Local |
Information AboutLocal Regression |
| CATEGORIES ABOUT LOCAL REGRESSION | |
| regression analysis | |
|
The trade-off for these features is increased computation. Because it is so computationally intensive, LOESS would have been practically impossible to use in the era when least squares regression was being developed. Most other modern methods for process modeling are similar to LOESS in this respect. These methods have been consciously designed to use our current computational ability to the fullest possible advantage to achieve goals not easily achieved by traditional approaches. Plotting a smooth curve through a set of data points using this statistical technique is called a Loess Curve, particularly when each smoothed value is given by a weighted quadratic least squares regression over the span of values of the y-axis Scattergram criterion variable. When each smoothed value is given by a weighted linear least squares regression over the span, this is known as a '''Lowess curve'''; however, some authorities treat '''Lowess''' and Loess as synonyms. DEFINITION OF A LOESS MODEL LOESS, originally proposed by Cleveland (1979) and further developed by Cleveland and Devlin (1988), specifically denotes a method that is (somewhat) more descriptively known as locally weighted polynomial regression. At each point in the Data Set a low-degree Polynomial is fit to a subset of the data, with Explanatory Variable values near the point whose Response is being estimated. The polynomial is fit using Weighted Least Squares , giving more weight to points near the point whose response is being estimated and less weight to points further away. The value of the regression function for the point is then obtained by evaluating the local polynomial using the explanatory variable values for that data point. The LOESS fit is complete after regression function values have been computed for each of the data points. Many of the details of this method, such as the degree of the polynomial model and the weights, are flexible. The range of choices for each part of the method and typical defaults are briefly discussed next. LOCALIZED SUBSETS OF DATA The subsets of data used for each weighted least squares fit in LOESS are determined by a nearest neighbors algorithm. A user-specified input to the procedure called the "bandwidth" or "smoothing parameter" determines how much of the data is used to fit each local polynomial. The smoothing parameter, , is a number between and 1, with denoting the degree of the local polynomial. The value of is the proportion of data used in each fit. The subset of data used in each weighted least squares fit comprises the (rounded to the next largest integer) points whose explanatory variables values are closest to the point at which the response is being estimated. is called the smoothing parameter because it controls the flexibility of the LOESS regression function. Large values of produce the smoothest functions that wiggle the least in response to fluctuations in the data. The smaller is, the closer the regression function will conform to the data. Using too small a value of the smoothing parameter is not desirable, however, since the regression function will eventually start to capture the random error in the data. Useful values of the smoothing parameter typically lie in the range 0.25 to 0.5 for most LOESS applications. DEGREE OF LOCAL POLYNOMIALS The local polynomials fit to each subset of the data are almost always of first or second degree; that is, either locally linear (in the straight line sense) or locally quadratic. Using a zero degree polynomial turns LOESS into a weighted Moving Average . Such a simple local model might work well for some situations, but may not always approximate the underlying function well enough. Higher-degree polynomials would work in theory, but yield models that are not really in the spirit of LOESS. LOESS is based on the ideas that any function can be well approximated in a small neighborhood by a low-order polynomial and that simple models can be fit to data easily. High-degree polynomials would tend to overfit the data in each subset and are numerically unstable, making accurate computations difficult. WEIGHT FUNCTION As mentioned above, the weight function gives the most weight to the data points nearest the point of estimation and the least weight to the data points that are furthest away. The use of the weights is based on the idea that points near each other in the explanatory variable space are more likely to be related to each other in a simple way than points that are further apart. Following this logic, points that are likely to follow the local model best influence the local model parameter estimates the most. Points that are less likely to actually conform to the local model have less influence on the local model Parameter Estimates . The traditional weight function used for LOESS is the tri-cube weight function, |
|
|