Additive model

In statistics, an additive model (AM) is a nonparametric regression method. It was suggested by Jerome H. Friedman and Werner Stuetzle (1981) and is an essential part of the ACE algorithm. The AM uses a one-dimensional smoother to build a restricted class of nonparametric regression models. Because of this, it is less affected by the curse of dimensionality than a p-dimensional smoother. Furthermore, the AM is more flexible than a standard linear model, while being more interpretable than a general regression surface at the cost of approximation errors. Problems with AM, like many other machine-learning methods, include model selection, overfitting, and multicollinearity.

Description

Given a data set $\{y_i,x_{i1},\dots ,x_{ip}{\}}_{i=1}^n$ of n statistical units, where $\{x_{i1},\dots ,x_{ip}{\}}_{i=1}^n$ represent predictors and $y_i$ is the outcome, the additive model takes the form

$\mathrm{E}[y_i|x_{i1},\dots ,x_{ip}]={\beta }_0+\sum _{j=1}^p{f}_j(x_{ij})$

or

$Y={\beta }_0+\sum _{j=1}^p{f}_j(X_j)+\epsilon$

Where $\mathrm{E}[ϵ]=0$, $\mathrm{V}\mathrm{a}\mathrm{r}(ϵ)={\sigma }^2$ and $\mathrm{E}[f_j(X_j)]=0$. The functions $f_j(x_{ij})$ are unknown smooth functions fit from the data. Fitting the AM (i.e. the functions $f_j(x_{ij})$) can be done using the backfitting algorithm proposed by Andreas Buja, Trevor Hastie and Robert Tibshirani (1989).

See also

• Generalized additive model
• Backfitting algorithm
• Projection pursuit regression
• Generalized additive model for location, scale, and shape (GAMLSS)
• Median polish
• Projection pursuit

Further reading

• Breiman, L. and Friedman, J.H. (1985). "Estimating Optimal Transformations for Multiple Regression and Correlation", Journal of the American Statistical Association 80:580–598. doi:10.1080/01621459.1985.10478157