PNOPT: Proximal Newton OPTimizer
PNOPT (pronounced pee-en-opt) is a MATLAB package that uses proximal Newton-type methods to minimize composite functions. For details, please refer to the PNOPT paper.
Installation
Unpack the archive and add the yuekai-PNOPT-xxxxxxx
directory to your
MATLAB search path,
e.g.
addpath /home/yuekai/matlab/yuekai-PNOPT-xxxxxxx/
We suggest users also install TFOCS (pronounced tee-fox), a MATLAB package that uses accelerated first-order methods to solve conic programs.
Usage
PNOPT has the calling sequence:
[ x, f, output ] = pnopt( smoothF, nonsmoothF, x0, options );
The required input arguments are:
-
smoothF
: a smooth function, -
nonsmoothF
: a nonsmooth function, -
x0
: a starting point for the solver.
The user can also supply an options
structure created using the
pnopt_optimset
function
to customize the behavior of PNOPT. pnopt_optimset
shares a similar
interface with MATLAB's optimset
function:
options = pnopt_optimset( 'param1', val1, 'param2', val2, ... );
Calling pnopt_optimset
with no inputs and outputs prints available
options.
PNOPT returns:
-
x
: an optimal solution, -
f
: the optimal value, -
output
: a structure containing information collected during the execution of PNOPT.
Creating the smooth and nonsmooth functions
Smooth and nonsmooth functions must satisty these conventions:
-
smoothF(x)
should return the function value and gradient atx
, i.e.[ fx, gradx ] = smoothF(x)
, -
nonsmoothF(x)
should return function value atx
, i.e.f_x = nonsmoothF(x)
, -
nonsmoothF(x,t)
should return the proximal pointy
and the function value aty
, i.e.[ f_y, y ] = nonsmoothF(x,t )
.
PNOPT is compatible with the function generators included with TFOCS that accept vector arguments so users can use these generators to create commonly used smooth and nonsmooth functions. Please refer to section 3 of the TFOCS user guide for details.
Demo
n = 100; p = 200; X = randn(n,p); y = sign( X * ( (rand(p,1) > .5) .* randn(p,1) ) + randn(n,1) ); logistic_obj = @(w) LogisticLoss(w,X,y); lambda = 10; l1_pen = prox_l1(lambda); w0 = zeros(p,1); pnopt_options = pnopt_optimset( 'optim', 1e-8 ); [ w, f ] = pnopt( logistic_obj, l1_pen, w0, pnopt_options ); figure stem( w ) xlim( [ 1; p ] );
pnopt_demo
================================================================
PNOPT v. 0.9.1 (Dec. 15, 2013)
================================================================
Fun. Prox Step len. Obj. val. Optim.
----------------------------------------------------------------
0 | 1 0 6.9315e+01 4.4998e+00
10 | 11 49 1.0000e+00 6.7911e+01 1.8744e-03
13 | 14 66 1.0000e+00 6.7911e+01 2.0037e-04
----------------------------------------------------------------