Basis expansions
Contents
Basis expansions#

Problem: How do we model a non-linear relationship?
Left: Regression of
wage
ontoage
.Right: Logistic regression for classes
wage>250
andwage<250
Strategy:#
Define a model:
\[Y = \beta_0 + \beta_1 f_1(X) + \beta_2 f_2(X) + \dots + \beta_d f_d(X) + \epsilon.\]
Fit this model through least-squares regression: \(f_j\)’s are nonlinear, model is linear!
Some options for \(f_1,\dots,f_d\):
Polynomials, \(f_i(x) = x^i\).
Indicator functions, \(f_i(x) = \mathbf{1}(c_i \leq x < c_{i+1})\).
Piecewise constant functions#

Piecewise polynomial functions#
