### Smoothing Methods in Regression (Advanced Data Analysis from an Elementary Point of View)

The constructive alternative to complaining about linear regression is
non-parametric regression. There are many ways to do this, but we will focus
on the conceptually simplest one, which is smoothing; especially kernel
smoothing. All smoothers involve local averaging of the training data. The
bias-variance trade-off tells us that there is an optimal amount of smoothing,
which depends both on how rough the true regression curve is, and on how much
data we have; we should smooth less as we get more information about the true
curve. Knowing the truly optimal amount of smoothing is impossible, but we can
use cross-validation to select a good degree of smoothing, and adapt to the
unknown roughness of the true curve. Detailed examples. Analysis o how
quickly kernel regression converges on the truth. Using smoothing to
automatically discover interactions. Plots to help interpret multivariate
smoothing results. Average predictive comparisons.

*Readings*: Notes, chapter 4 (R); Faraway, section 11.1

*Optional readings*: Hayfield and Racine, "Nonparametric Econometrics: The `np` Package"; Gelman and Pardoe, "Average Predictive Comparisons for Models with Nonlinearity, Interactions, and Variance Components" [PDF]

Advanced Data Analysis from an Elementary Point of View

Posted by crshalizi at January 26, 2012 10:30 | permanent link