Tolga Ergen

I am a Ph.D. student in the department of Electrical Engineering at Stanford University, where I am advised by Prof. Mert Pilanci. I received my B.S. and M.S. degrees in Electrical and Electronics Engineering from Bilkent University in 2016 and 2018, respectively, where I worked with Prof. Suleyman Serdar Kozat.

Email  /  CV  /  Google Scholar

profile photo

Research

My research interests lie at intersection of convex optimization and deep learning. Currently, I focus on the understanding of neural network through the lens of convex optimization. Previously, I also developed efficient second order randomized algorithms to train neural networks.

Publications

Revealing the Structure of Deep Neural Networks via Convex Duality
Tolga Ergen, Mert Pilanci
ICML 2021  
arXiv

deep neural networks, convex duality, non-convex optimization

Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs
Tolga Ergen, Mert Pilanci
ICML 2021  

deep neural networks, convex optimization

Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization
Tolga Ergen, Arda Sahiner, Batu Ozturkler, John Pauly, Morteza Mardani, Mert Pilanci
PDF

neural networks, convex analysis

Implicit Convex Regularizers of CNN Architectures: Convex Optimization of Two- and Three-Layer Networks in Polynomial Time
Tolga Ergen, Mert Pilanci
ICLR 2021 (Spotlight Presentation)  
PDF

convolutional neural networks, convex optimization, deep learning

Vector-output ReLU Neural Network Problems are Copositive Programs: Convex Analysis of Two Layer Networks and Polynomial-time Algorithms
Arda Sahiner, Tolga Ergen, John Pauly, Mert Pilanci
ICLR 2021  
PDF

neural networks, convex analysis, non-convex optimization

Convex Programs for Global Optimization of Convolutional Neural Networks in Polynomial-Time
Tolga Ergen, Mert Pilanci
NeurIPS 2020 Workshop on Optimization for Machine Learning (Oral Presentation)  
PDF

convolutional neural networks, convex optimization, deep learning

Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks
Mert Pilanci, Tolga Ergen
ICML 2020 
PDF

neural networks, convex analysis, non-convex optimization

Convex Geometry of Two-Layer ReLU Networks: Implicit Autoencoding and Interpretable Models
Tolga Ergen, Mert Pilanci
AISTATS 2020 
PDF

neural networks, convex analysis, non-convex optimization

Convex Neural Autoregressive Models: Towards Tractable, Expressive, and Theoretically-Backed Models for Sequential Forecasting and Generator
Vikul Gupta, Burak Bartan, Tolga Ergen, Mert Pilanci
ICASSP 2021 
PDF

generative models, neural networks, convex optimization

Convex Geometry and Duality of Over-parameterized Neural Networks
Tolga Ergen, Mert Pilanci
arXiv

neural networks, convex analysis, non-convex optimization

A Novel Distributed Anomaly Detection Algorithm Based on Support Vector Machines
Tolga Ergen, Serdar Kozat
Digital Signal Processing 
PDF

support vector machines, distributed optimization

Convex Duality and Cutting Plane Methods for Over-parameterized Neural Networks
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Optimization for Machine Learning  
PDF

neural networks, convex analysis, non-convex optimization

Random Projections for Learning Non-convex Models
Tolga Ergen, Mert Pilanci
NeurIPS 2019 Workshop on Beyond First Order Methods in Machine Learning  
PDF

randomized algorithms, non-convex optimization

Convex Optimization for Shallow Neural Networks
Tolga Ergen, Mert Pilanci
ALLERTON 2019 
PDF

neural networks, convex optimization

Energy-Efficient LSTM Networks for Online Learning
Tolga Ergen, Ali Mirza, Serdar Kozat
IEEE TNNNLS 
PDF

recurrent neural networks, online learning, non-convex optimization

Unsupervised Anomaly Detection with LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS 
PDF

recurrent neural networks, support vector machines, non-convex optimization

Team-Optimal Online Estimation of Dynamic Parameters over Distributed Tree Networks
Fatih Kilic, Tolga Ergen, Muhammed Sayin, Serdar Kozat.
Signal Processing 
PDF

online learning, distributed optimization

Online Training of LSTM Networks in Distributed Systems for Variable Length Data Sequences
Tolga Ergen, Serdar Kozat
IEEE TNNLS 
PDF

recurrent neural networks, distributed optimization

Efficient Online Learning Algorithms Based on LSTM Neural Networks
Tolga Ergen, Serdar Kozat
IEEE TNNLS 
PDF

recurrent neural networks, online learning

A Highly Efficient Recurrent Neural Network Architecture for Data Regression
Tolga Ergen, Emir Ceyani
IEEE SIU 2018 
PDF

recurrent neural networks, online learning

A Novel Anomaly Detection Approach Based on Neural Networks
Tolga Ergen, Mine Kerpicci
IEEE SIU 2018 
PDF

neural networks, non-convex optimization

Computationally Efficient Online Regression via LSTM Neural Networks
Tolga Ergen, Serdar Kozat
EUSIPCO 2017 
PDF

recurrent neural networks, online learning

An Efficient Bandit Algorithm for General Weight Assignments
Kaan Gokcesu, Tolga Ergen, Selami Ciftci, Serdar Kozat
IEEE SIU 2017 
PDF

adversarial multi armed bandit, non-convex optimization

Neural Networks Based Online Learning.
Tolga Ergen, Serdar Kozat
IEEE SIU 2017 
PDF

neural networks, online learning

Novelty Detection Using Soft Partitioning and Hierarchical Models
Tolga Ergen, Kaan Gokcesu, Mustafa Simsek, Serdar Kozat
IEEE SIU 2017 
PDF

online learning, non-convex optimization

Online Distributed Nonlinear Regression via Neural Networks
Tolga Ergen, Serdar Kozat
IEEE SIU 2017 
PDF

neural networks, distributed optimization

Teaching
(Fall 2019, 2020, 2021) EE269, Signal Processing for Machine Learning, TA

(Winter 2020, 2021) EE270, Large Scale Matrix Computation, Optimization and Learning, TA

(Spring 2020, 2021) EE364B, Convex Optimization II, TA
(Fall 2016, 2017, Spring 2018) EEE424, Digital Signal Processing, TA

(Spring 2018) EEE102, Introduction to Digital Circuit Design, TA

Academic Service

Reviewer for NeurIPS, ICML, IEEE TNNLS, IEEE SPL

template