[2019-12-30] Prof. Jason D. Lee, Princeton University, "Beyond Linearization in Neural Networks"

Poster:Post date:2019-12-09
Title: Beyond Linearization in Neural Networks
Date: 2019-12-30 10:20am-11:30am
Location: R210, CSIE
Speaker: Prof. Jason D. Lee, Princeton University
Hosted by: Prof. Yen-Huan Li


Deep Learning has had phenomenal empirical successes in many domains including computer vision, natural language processing, and speech recognition. To consolidate and boost the empirical success, we need to develop a more systematic and deeper understanding of the elusive principles of deep learning.
In this talk, I will provide analysis of several elements of deep learning including non-convex optimization, overparametrization, and generalization error. Recent theoretical work has established connections between neural networks and linearized models governed by Neural Tangent Kernels (NTK). Such theory leads to concrete convergence and generalization results, yet the empirical performance of neural networks are observed to exceed their linearized models.
Towards closing this gap, we investigate the training of overparametrized neural networks that are more global than the NTK regime. We show that by utilizing more of the parameter space, SGD can learn with lower sample complexity than NTK under mild distributional assumptions.
Jason D. Lee is an assistant professor of Electrical Engineering at Princeton University and associated faculty in Computer Science. Previously, he was an assistant professor at USC for three years, and before that he was a postdoc in the Computer Science Department at UC Berkeley working with Michael Jordan, and also collaborated with Ben Recht. He received his PhD in Computational Mathematical Engineering advised by Trevor Hastie and Jonathan Taylor. He received a BS in Mathematics from Duke University advised by Mauro Maggioni. His research interests are broadly in Statistics (Statistical Learning and High-dimensional Statistics), Machine Learning, and Optimization.
Last modification time:2019-12-09 AM 10:29

cron web_use_log