Machine Learning Foundations/Techniques, Fall 2020

Course Description

Machine learning allows computational systems to adaptively improve their performance with experience accumulated from the data observed. This course introduces the basics of learning theories, the design and analysis of learning algorithms, and some applications of machine learning.


Course Information


Class Policy

Course Plan (tentative)

Machine Learning Foundations

datesyllabustodo/donesuggested reading
09/15 course introduction course slides
09/18 topic 1: when can machines learn?
the learning problem;
course slides; LFD 1.0, 1.1.1, 1.2.4
09/22 learning to answer yes/no course slides; LFD 1.1.2, 3.1 ;
course slides on machine learning and artificial intelligence
09/25 types of learning homework 1 announced notes about convergence of PLA ;
course slides; LFD 1.2; LFD 1.3
09/26 no class on this make-up day because it will be ineffective to take four hours within two days
09/29 feasibility of learning notes about proof of Hoeffding ;
course slides; LFD 1.3
10/02 no class because of long-weekend of mid-autumn festival
10/06 topic 2: why can machines learn?
training versus testing
course slides; LFD 2.0, 2.1.1
10/09 no class because of long-weekend of double-ten holiday
10/13 the VC dimension course slides; LFD 2.2
10/16 noise and error homework 1 due; homework 2 announced course slides; LFD 1.4
10/20 topic 3: how can machines learn?
linear regression
course slides; LFD 3.2
10/23 logistic regression course slides; LFD 3.3
10/27 linear models for classification course slides; LFD 3.3 (for SGD part only)
10/30 nonlinear transformation homework 2 due; homework 3 announced course slides; LFD 3.4
11/03 topic 4: how can machines learn better?
hazard of overfitting
course slides; LFD 4.0, 4.1
11/06 regularization course slides; LFD 4.2
11/10 validation course slides; LFD 4.3
11/13 three learning principles;
machine learning for modern artificial intelligence
homework 3 due; homework 4 announced course slides; LFD 5;
course slides

Machine Learning Techniques

datesyllabustodo/donesuggested reading
11/17 topic 4: how can machines learn by embedding numerous features?
linear support vector machine
course slides; LFD e-8.1
11/20 no class because of NTU sports day
11/24 dual support vector machine;
kernel support vector machine
course slides; LFD e-8.2;
course slides; LFD e-8.3
11/27 soft-margin support vector machine homework 4 due; homework 5 announced course slides; LFD e-8.4
12/01 topic 7: how can machines learn by distilling hidden features?
neural network
course slides; LFD e-7.1, e-7.2, e-7.3, e-7.4 (selected parts)
12/04 matrix factorization course slides;
extended reading:
12/08 guest lecture: Machine Learning and Data in Big Tech Companies by Dr. Scott Chen
12/11 no class because of NeurIPS 2020
12/15 neural networks, matrix factorization (unfinished parts)
12/18 topic 2: how can machines learn by combining predictive features?
blending and bagging
course slides;
extended reading:
12/22 adaptive boosting course slides;
extended reading:
12/25 decision tree (selected) and random forest (selected) homework 5 due; homework 6 announced course slides; course slides;
extended reading:
12/29 gradient boosted decision tree; deep learning basics (selected) course slides; course slides; LFD e-7.6
extended reading:
01/01 no class because of long-weekend of new year's day
01/05 modern deep learning: activation course slides;
extended reading:
01/08 modern deep learning: initialization, optimization, regularization course slides;
extended reading: course slides;
extended reading: course slides (not taught);
extended reading:
01/12 finale and award ceremony course slides
01/15 no class and winter vacation started (really?) homework 6 due
01/19 no class and winter vacation started (really?) final project due