Optimization Methods for Deep Learning
Deep learning involves a difficult non-convex optimization
problem. The goal of this course is to study the implementation
of optimization methods for deep learning.
We will run this course in the following formats:
For potential students: you want to make sure that you are
interested in optimization for deep learning.
- lectures (by the instructor)
- project presentations (by students): we will have many.
We will heavily use the software
Among the various types of networks, we will pay more attention
You will get hands-on experiences in implementing a deep
Slides and recordings
This section (and slides) will be continuously updated.
- Course information
( video )
Optimization problems for deep learning
Stochastic gradient methods for deep learning
Some explanation: we do not have a final exam, so the class should end on 6/14, which is a holiday. To give you more time on doing the project, the decision is to run the project 6 presentation on 6/21 and we decide not to have a class on 5/31.
Issues related to COVID-19
- According to school's regulation, all students must wear
- If the covid situation becomes serious, we will move the course online.
- If you are sick, please do not come to the class.
Acknowledgements: the following people have greatly helped to prepare
materials for this course (including creating the software used for the course
and trying some projects).
Chien-Chih Wang, Kent Loong Tan, Pin-Yen Lin, Cheng-Hung Liu (former
and current members in my group), Pengrui Quan (UCLA), and Leonardo Galli (University of Florence)