l_1 Trend Filtering

SIAM Review, problems and techniques section, 51(2):339–360, May 2009.

• l1_trend_filter.pdf

• l1_tf software package

• R package wrapper by Hadley Wickham

The problem of estimating underlying trends in time series data arises in a variety of disciplines. In this paper we propose a variation on Hodrick-Prescott (H-P) filtering, a widely used method for trend estimation. The proposed l1 trend filtering method} substitutes a sum of absolute values (i.e., an l1-norm) for the sum of squares used in H-P filtering to penalize variations in the estimated trend. The l1 trend filtering method produces trend estimates that are piecewise linear, and therefore is well suited to analyzing time series with an underlying piecewise linear trend. The kinks, knots, or changes in slope of the estimated trend can be interpreted as abrupt changes or events in the underlying dynamics of the time series. Using specialized interior-point methods, l1 trend filtering can be carried out with not much more effort than H-P filtering; in particular, the number of arithmetic operations required grows linearly with the number of data points. We describe the method and some of its basic properties, and give some illustrative examples. We show how the method is related to l1 regularization based methods in sparse signal recovery and feature selection, and list some extensions of the basic method.