An Oracle-Structured Bundle Method for Distributed Optimization

T. Parshakova, F. Zhang, and S. Boyd

Manuscript posted August 2022.

We consider the problem of minimizing a function that is a sum of convex agent functions plus a convex common public function that couples them. The agent func- tions can only be accessed via a subgradient oracle; the public function is assumed to be structured and expressable in a domain specific language (DSL) for convex optimization. We focus on the case when the evaluation of the agent oracles can require significant effort, which justifies the use of solution methods that carry out significant computation in each iteration. We propose a cutting-plane or bundle-type method for the distributed optimization problem, which has a number of advantages over other methods that are compatible with the access methods, such as proximal subgradient methods: it has very few parameters that need to be tuned; it often produces a reasonable approximate solution in just a few tens of iterations; and it tolerates agent failures. This paper is accompanied by an open source package that implements the proposed method.