Optimization and Machine Learning

Course Outline

Optimization techniques are used in all kinds of machine learning problems because in general we would like to minimize the testing error. This course will contain two parts. The first part focuses on convex optimization techniques. We discuss methods for least-squares, linear and quadratic programs, semidefinite programming, and others. We also touch theory behind these methods (e.g., optimality conditions and duality theory). In the second part of this course we will investigate how optimization techniques are applied to various machine learning problems (e.g., SVM, maximum entropy, conditional random fields, sparse reconstruction for signal processing applications). We further discuss that for different machine learning applications how to choose right optimization methods.

Course Objective


Once every two weeks. Please write your homework/reports in English. For late homework, the score will be exponentially decreased. See FAQ about how to submit your homework.

All HW questions can be found in the textbook.


You can bring notes and the textbook. Other books or electronic devices are not allowed. Sorry that the 2nd midterm is a bit late, but this is due to my conference schedule.

Sample exams in the past: exam1, exam2, exam3.

For midterms, discussions will be in the following week. For the Final exam, it will be at ?? on ?? (room ??, CSIE building).


30% homework, 70% Exam. (tentative)

Some (usually 10%) may fail if they don't work hard.


Last modified: