Hsuan-Tien Lin

Home | MOOCs | AIsk | Courses | Research Group | Awards | Publications | Presentations | Programs/Data


I am fortunate to be among the very first NTU EECS professors to offer two Mandarin-teaching MOOCs (massive open online courses) on NTU@Coursera. The two MOOCs are Machine Learning Foundations (Mathematical, Algorithmic) and Machine Learning Techniques and are based on the textbook Learning from Data: A Short Course that I co-authored. The book is consistently among the best sellers in Machine Learning on Amazon.

The slides of the MOOCs below are available as is with no explicit or implied warranties. The slides themselves are shared by CC-BY-NC 3.0, but the copyright of all materials (figures in particular) remain with the original copyright holder (in almost all cases the authors of the Learning from Data: A Short Course book).

Here are some quick links for each MOOC. The erratas are here.

Machine Learning Foundations Mathematical Foundations MOOC
Algorithmic Foundations MOOC
all handout slides free youtube videos
Machine Learning Techniques Techniques MOOC all handout slides free youtube videos

Detailed outlines for each MOOC, along with the presentation sldies, are listed below.

(Under preparation) Data Structures and Algorithms

(New) I finally make up my mind to develop this MOOC on 09/28/2022, with the hope to challenge myself after 14 years of teaching, and to promote a healthier online teaching and learning environment. More slides and recordings will be uploaded when they are ready. Stay tuned!
Lecture 1 Algorithm:
  1. definition of algorithm
  2. pseudo code of algorithm
  3. criteria of algorithm
  4. correctness proof of algorithm
handout slides
Lecture 2 Data Structure:
  1. definition of data structure
  2. ordered array as data structure
  3. Get (search) in ordered array
  4. why data structures and algorithms
handout slides

Machine Learning Foundations

When can machines learn?
Lecture 1 the learning problem:
  1. Course Introduction
  2. What is Machine Learning
  3. Applications of Machine Learning
  4. Components of Machine Learning
  5. Machine Learning and Other Fields
handout slides; presentation slides
Lecture 2 learning to answer yes/no:
  1. Perceptron Hypothesis Set
  2. Perceptron Learning Algorithm (PLA)
  3. Guarantee of PLA
  4. Non-Separable Data
handout slides; presentation slides
Lecture 3 types of learning:
  1. Learning with Different Output Space
  2. Learning with Different Data Label
  3. Learning with Different Protocol
  4. Learning with Different Input Space
handout slides; presentation slides
Lecture 4 feasibility of learning:
  1. Learning is Impossible?
  2. Probability to the Rescue
  3. Connection to Learning
  4. Connection to Real Learning
handout slides; presentation slides
Why can machines learn?
Lecture 5 training versus testing:
  1. Recap and Preview
  2. Effective Number of Lines
  3. Effective Number of Hypotheses
  4. Break Point
handout slides; presentation slides
Lecture 6 theory of generalization:
  1. Restriction of Break Point
  2. Bounding Function: Basic Cases
  3. Bounding Function: Inductive Cases
  4. A Pictorial Proof
handout slides; presentation slides
Lecture 7 the VC dimension:
  1. Definition of VC Dimension
  2. VC Dimension of Perceptrons
  3. Physical Intuition of VC Dimension
  4. Interpreting VC Dimension
handout slides; presentation slides
Lecture 8 noise and error:
  1. Noise and Probabilistic Target
  2. Error Measure
  3. Algorithmic Error Measure
  4. Weighted Classification
handout slides; presentation slides
How can machines learn?
Lecture 9 linear regression:
  1. Linear Regression Problem
  2. Linear Regression Algorithm
  3. Generalization Issue
  4. Linear Regression for Binary Classification
handout slides; presentation slides
Lecture 10 logistic regression:
  1. Logistic Regression Problem
  2. Logistic Regression Error
  3. Gradient of Logistic Regression Error
  4. Gradient Descent
handout slides; presentation slides
Lecture 11 linear models for classification:
  1. Linear Models for Binary Classification
  2. Stochastic Gradient Descent
  3. Multiclass via Logistic Regression
  4. Multiclass via Binary Classification
handout slides; presentation slides
Lecture 12 nonlinear transformation:
  1. Quadratic Hypotheses
  2. Nonlinear Transform
  3. Price of Nonlinear Transform
  4. Structured Hypothesis Sets
handout slides; presentation slides
How can machines learn better?
Lecture 13 hazard of overfitting:
  1. What is Overfitting?
  2. The Role of Noise and Data Size
  3. Deterministic Noise
  4. Dealing with Overfitting
handout slides; presentation slides
Lecture 14 regularization:
  1. Regularized Hypothesis Set
  2. Weight Decay Regularization
  3. Regularization and VC Theory
  4. General Regularizers
handout slides; presentation slides
Lecture 15 validation:
  1. Model Selection Problem
  2. Validation
  3. Leave-One-Out Cross Validation
  4. V-Fold Cross Validation
handout slides; presentation slides
Lecture 16 three learning principles:
  1. Occam's Razor
  2. Sampling Bias
  3. Data Snooping
  4. Power of Three
handout slides; presentation slides

Machine Learning Techniques

embedding numerous features
Lecture 1 linear support vector machine:
  1. Course Introduction
  2. Large-Margin Separating Hyperplane
  3. Standard Large-Margin Problem
  4. Support Vector Machine
  5. Reasons behind Large-Margin Hyperplane
handout slides; presentation slides
Lecture 2 dual support vector machine:
  1. Motivation of Dual SVM
  2. Lagrange Dual SVM
  3. Solving Dual SVM
  4. Messages behind Dual SVM
handout slides; presentation slides
Lecture 3 kernel support vector machine:
  1. Kernel Trick
  2. Polynomial Kernel
  3. Gaussian Kernel
  4. Comparison of Kernels
handout slides; presentation slides
Lecture 4 soft-margin support vector machine:
  1. Motivation and Primal Problem
  2. Dual Problem
  3. Messages behind Soft-Margin SVM
  4. Model Selection
handout slides; presentation slides
Lecture 5 kernel logistic regression:
  1. Soft-Margin SVM as Regularized Model
  2. SVM versus Logistic Regression
  3. SVM for Soft Binary Classification
  4. Kernel Logistic Regression
handout slides; presentation slides
Lecture 6 support vector regression:
  1. Kernel Ridge Regression
  2. Support Vector Regression Primal
  3. Support Vector Regression Dual
  4. Summary of Kernel Models
handout slides; presentation slides
combining predictive features
Lecture 7 blending and bagging:
  1. Motivation of Aggregation
  2. Uniform Blending
  3. Linear and Any Blending
  4. Bagging (Bootstrap Aggregation)
handout slides; presentation slides
Lecture 8 adaptive boosting:
  1. Motivation of Boosting
  2. Diversity by Re-weighting
  3. Adaptive Boosting Algorithm
  4. Adaptive Boosting in Action
handout slides; presentation slides
Lecture 9 decision tree:
  1. Decision Tree Hypothesis
  2. Decision Tree Algorithm
  3. Decision Tree Heuristics in C&RT
  4. Decision Tree in Action
handout slides; presentation slides
Lecture 10 random forest:
  1. Random Forest Algorithm
  2. Out-Of-Bag Estimate
  3. Feature Selection
  4. Random Forest in Action
handout slides; presentation slides
Lecture 11 gradient boosted decision tree:
  1. Adaptive Boosted Decision Tree
  2. Optimization View of AdaBoost
  3. Gradient Boosting
  4. Summary of Aggregation Models
handout slides; presentation slides
distilling hidden features
Lecture 12 neural network:
  1. Motivation
  2. Neural Network Hypothesis
  3. Neural Network Learning
  4. Optimization and Regularization
handout slides; presentation slides
Lecture 13 deep learning:
  1. Deep Neural Network
  2. Autoencoder
  3. Denoising Autoencoder
  4. Principal Component Analysis
handout slides; presentation slides
Lecture 14 radial basis function network:
  1. RBF Network Hypothesis
  2. RBF Network Learning
  3. k-Means Algorithm
  4. k-Means and RBF Network in Action
handout slides; presentation slides
Lecture 15 matrix factorization:
  1. Linear Network Hypothesis
  2. Basic Matrix Factorization
  3. Stochastic Gradient Descent
  4. Summary of Extraction Models
handout slides; presentation slides
happy learning!
Lecture 16 finale:
  1. Feature Exploitation Techniques
  2. Error Optimization Techniques
  3. Overfitting Elimination Techniques
  4. Machine Learning in Practice
handout slides; presentation slides

Last updated at CST 13:07, October 04, 2023
Please feel free to contact me: htlin.email.png
Valid HTML 4.0!