Optimization Methods for Deep Learning

Course Outline

Deep learning involves a difficult non-convex optimization problem. The goal of this course is to study the implementation of optimization methods for deep learning. We will run this course in the following formats: For potential students: you want to make sure that you are interested in optimization for deep learning.

We will heavily use the software simpleNN

Among the various types of networks, we will pay more attention to CNN.

You will get hands-on experiences in implementing a deep learning code

Slides and recordings

This section (and slides) will be continuously updated.


Some explanation: we do not have a final exam, so the class should end on 6/14, which is a holiday. To give you more time on doing the project, the decision is to run the project 6 presentation on 6/21 and we decide not to have a class on 5/31.


No exam


100% Projects.

Issues related to COVID-19

Acknowledgements: the following people have greatly helped to prepare materials for this course (including creating the software used for the course and trying some projects). Chien-Chih Wang, Kent Loong Tan, Pin-Yen Lin, Cheng-Hung Liu (former and current members in my group), Pengrui Quan (UCLA), and Leonardo Galli (University of Florence)

Last modified: