Optimization Methods for Deep Learning

Course Outline

Deep learning involves a difficult non-convex optimization problem. The goal of this course is to study the implementation of optimization methods for deep learning. We will run this course in the following formats: For potential students: you want to make sure that you are interested in optimization on deep learning.

We will heavily use the software simpleNN

Among the various types of networks, we will pay more attention to CNN.

You will get hands-on experiences in implementing a deep learning code

This course was first offered at UCLA in winter 2019 (see course page here; 21 enrolled and 19 finished).

Slides, Reading list, and Projects

This section (and slides) will be continuously updated.

Feb 17

Feb 24 March 2 March 9 March 16 March 23 March 30 April 6 April 13 April 20 April 27 May 4 May 11 May 18 May 25 June 1 June 8 Other possible projects


No exam


100% Projects.
Acknowledgements: the following people have greatly helped to prepare materials for this course (including creating the software used for the course and trying some projects). Chien-Chih Wang, Kent Loong Tan, Pin-Yen Lin, Cheng-Hung Liu (former and current members in my group), Pengrui Quan (UCLA), and Leonardo Galli (University of Florence)

Last modified: