New York City, United States

Aaron Defazio

Aaron's research focuses on improving the practice of machine learning through the development of more reliable and theoretically sound methods such as performance optimization, initialization, and normalization. He also drives current research frontiers in applied areas and is currently involved in MRI imaging reconstruction and automated theorem proving.

Aaron's Publications

April 07, 2020

RESEARCH

On the Curved Geometry of Accelerated Optimization

In this work we propose a differential geometric motivation for Nesterov’s accelerated gradient method (AGM) for strongly-convex problems. By considering the optimization procedure as occurring on a Riemannian manifold with a natural structure,…

Duc Le, Xiaohui Zhang, Weiyi Zhang, Christian Fuegen, Geoffrey Zweig, Michael L. Seltzer

April 07, 2020

April 07, 2020

RESEARCH

On the Ineffectiveness of Variance Reduced Optimization for Deep Learning

The application of stochastic variance reduction to optimization has shown remarkable recent theoretical and practical success. The applicability of these techniques to the hard non-convex optimization problems encountered during training of…

Aaron Defazio, Leon Bottou

April 07, 2020