Distributed Machine Learning and Matrix Computations

The emergence of large distributed matrices in many applications has brought with it a slew of new algorithms and tools. Over the past few years, machine learning and numerical linear algebra on distributed matrices has become a thriving field. Manipulating such large matrices makes it necessary to think about distributed systems issues such as communication cost.

This workshop aims to bring closer researchers in distributed systems and large scale numerical linear algebra to foster cross-talk between the two fields. The goal is to encourage distributed systems researchers to work on machine learning and numerical linear algebra problems, to inform machine learning researchers about new developments on large scale matrix analysis, and to identify unique challenges and opportunities.

The workshop will conclude with a session of contributed posters.

Speakers

• Jeff Dean (Google)
• Carlos Guestrin (Washington)
• Reza Zadeh (Stanford)
• Ameet Talwalkar (UCLA)
• Inderjit Dhillon (UT Austin)
• Jeremy Freeman (Janelia Farm)
• Virginia Smith (Berkeley)
• Ankur Dave (Berkeley)
• David Woodruff (IBM)
• Jure Leskovec (Stanford)

Schedule

Accepted Papers

Minerva: A Scalable and Highly Efficient Training Platform for Deep Learning
Minjie Wang, Tianjun Xiao, Jianpeng Li, Jiaxing Zhang, Chuntao Hong, Zheng Zhang

Maxios: Large Scale Nonnegative Matrix Factorization for Collaborative Filtering
Simon Shaolei Du, Boyi Chen, Yilin Liu, Lei Li

Factorbird - a Parameter Server Approach to Distributed Matrix Factorization
Sebastian Schelter, Venu Satuluri, Reza Zadeh

Improved Algorithms for Distributed Boosting
Jeff Cooper, Lev Reyzin

Parallel and Distributed Inference in Coupled Tensor Factorization Models supplementary
Umut Simsekli, Beyza Ermis, Ali Taylan Cemgil, Figen Oztoprak, S. Ilker Birbil

Dogwild! — Distributed Hogwild for CPU and GPU
Cyprien Noel, Simon Osindero

Generalized Low Rank Models
Madeleine Udell, Corinne Horn, Reza Zadeh, Stephen Boyd

Elastic Distributed Bayesian Collaborative Filtering
Alex Beutel, Markus Weimer, Tom Minka, Yordan Zaykov, Vijay Narayanan

LOCO: Distributing Ridge Regression with Random Projections
Brian McWilliams, Christina Heinze, Nicolai Meinshausen, Gabriel Krummenacher, Hastagiri P. Vanchinathan

Logistic Matrix Factorization for Implicit Feedback Data
Christopher C. Johnson

Tighter Low-rank Approximation via Sampling the Leveraged Element
Srinadh Bhojanapalli, Prateek Jain, Sujay Sanghavi

A Comparison of Lasso-type Algorithms on Distributed Parallel Machine Learning Platforms
Jichuan Zeng, Haiqin Yang, Irwin King, Michael R. Lyu

A Randomized Algorithm for CCA
Paul Mineiro, Nikos Karampatziakis

FROGWILD! – Fast PageRank Approximations on Graph Engines
Ioannis Mitliagkas, Alexandros G. Dimakis, Michael Borokhovich, Constantine Caramanis

CometCloudCare (C3): Distributed Machine Learning Platform-as-a-Service with Privacy Preservation
Vamsi K. Potluru, Javier Diaz-Montes, Anand D. Sarwate, Sergey M. Plis, Vince D. Calhoun, Barak A. Pearlmutter, Manish Parashar

Global Convergence of Stochastic Gradient Descent for Some Nonconvex Matrix Problems (to appear)
Christopher De Sa, Kunle Olukotun, Christopher Re

Format

This workshop will consist of invited talks and paper submissions for a poster session. The target audience of this workshop includes industry and academic researchers interested in machine learning, large distributed systems, numerical linear algebra, and related fields.

As this is a workshop, there will be no printed proceedings.

Keynote Abstract

Jeff Dean: Over the past few years, we have built a software infrastructure for training neural networks that applies to a wide variety of deep learning models. This system has been used for training and deploying models for a wide variety of applications at Google. One of the properties we focus on in the system is that we want to be able to train large models on large datasets quickly, so that we can turn around experiments rapidly and quickly figure out the next set of experiments to perform, given the results of the previous round of experiments. As such, we have developed a number of different techniques that aid in rapid training of large models. I will discuss many of these techniques in this talk.