Optimization Methods for Deep Learning
Course Outline
Deep learning involves a difficult non-convex optimization
problem. The goal of this course is to study the implementation
of optimization methods for deep learning.
We will run this course in the following formats:
- lectures (by the instructor)
- project presentations (by students): we will have many.
For potential students: you want to make sure that you are
interested in optimization for deep learning.
We will heavily use the software
simpleNN
Among the various types of networks, we will pay more attention
to CNN.
You will get hands-on experiences in implementing a deep
learning code
Slides and recordings
This section (and slides) will be continuously updated.
- Course information
( video )
-
Optimization problems for deep learning
-
Stochastic gradient methods for deep learning
-
Gradient calculation
-
Implementation
-
Automatic differentiation
-
Newton method
Projects
Some explanation: we do not have a final exam, so the class should end on 6/14, which is a holiday. To give you more time on doing the project, the decision is to run the project 6 presentation on 6/21 and we decide not to have a class on 5/31.
Exams
No exam
Grading
100% Projects.
Issues related to COVID-19
- According to school's regulation, all students must wear
masks
- If the covid situation becomes serious, we will move the course online.
- If you are sick, please do not come to the class.
Acknowledgements: the following people have greatly helped to prepare
materials for this course (including creating the software used for the course
and trying some projects).
Chien-Chih Wang, Kent Loong Tan, Pin-Yen Lin, Cheng-Hung Liu (former
and current members in my group), Pengrui Quan (UCLA), and Leonardo Galli (University of Florence)
Last modified: