FAQ

• Half hour before the class starts we will send members the meeting ID and the password through NTU's CEIBA system.
• Please DO NOT share the ID/password to other people.
• Please do not email me to ask for the meeting code (unless I forgot to send out info half hour before the class). I cannot handle such emails.
• Before the class please check your Internet connection. If it does not work, you should quickly move to other places with the access
• If NTU's CEIBA system is down and somehow I cannot email you, I will post meeting ID and password on the course web page
• If I lost Internet during my lecture, the system will assign someone as a host. Please do not leave the meeting. I will quickly switch to use the data plan of my cell phone or move to a nearby place with a different Wifi.
• Please display your full name
• Please turn off microphone unless you need to ask questions or present projects
• We encourage you to ask questions during my lectures or your classmates' presentations
• We encourage you to turn on the video when asking questions. Then I know who you are
• Any other policy will be posted here

The main issue is that this course highly depends on students' projects and presentations. In the class we will ask each other about their detailed experimental settings. Thus it's not good to involve others who didn't do projects. Further as we run the course fully online, keeping a smaller group helps to make the discussion more effective.

Please check the course web page https://www.csie.ntu.edu.tw/~cjlin/courses/optdl2020/

The easiest way to find the course page is to search my name. On my homepage there is a section "Courses." It's listed there.

You should have basic understanding of the two areas optimization and deep learning. However, I don't expect you are an expert on these areas.

We assume that you know Python and Matlab/Octave.

We focus on the algorithms, their implementations, and their practical use. Thus we won't consider theoretical issues such as "do stochastic gradient methods get stuck on saddle points" or "do neural networks have saddle points"

In a way you can think that we position ourselves as those who write optimization implementations in existing deep learning tools.

Clearly, the design of this course is related to my past experiences on applying optimization algorithms on other machine learning methods. For example, we developed the widely used SVM package LIBSVM

It's not clear yet at this moment. Things depend on

• number of students, and
• contents of the project.

You (or your team) must do every project and submit a report.

For presentation, due to time limitation, we may randomly split students so that each gives a presentation once only every several projects. Here is what happened at UCLA in 2019.

• There are five projects. Students did all of them and submitted reports.
• For each of the first three projects, one third of students did presentations
• For the last two projects, students were first splitted to two-person teams. Then for each project, half of the teams did presentations
• The presentation time may vary. It will be announced in the project description.

In the beginning, students all do the same project. But in the final stage we may have several projects for you to choose (as materials become more research oriented).

If the presentation is on Monday, you should submit your report before 11:59pm on the previous Thursday.

This setting ensures that I have time to read your reports.

We will use NTU's CEIBA system for your project submission (see URL here. This place is only used for submitting reports/slides). No late submission will be accepted.

One or two pages (including references). Use the latex template here. See the resulting pdf. The template is based on Yinxue Xiao's first project report at UCLA in 2019

Please submit one pdf file only. No other files. No sub-directories.

Right before the class. For example, if we have a presentation session on Monday, then those who will do presentations must submit slides before 10am on that day.

When using outside resources, proper citation is necessary. This includes papers, text books, software libraries, websites, and helps from others.

Discussion is perfectly fine.

Please state what you have done. No fake results and no exaggeration. For doing research, failure is an option

For every project you will get a score. In the end we do a weighted average. If you enjoy doing research on the topic, then very likely your performance will be excellent and you will get a good grade.

We assume that you will use your own computers (or computers in your lab.)

• We won't consider super large data sets.
• We will mainly use CPU
• We will mainly use open-source packages (e.g., Python, Tensorflow, etc.). We will use MATLAB, but you can get the open-source version Octave.

No, you can use whatever you like. However, from experiences in the past we strongly suggest you to use linux.

Please email me. Any improvements on slides will be useful to students in the future.

Yes, absolutely.