Security and Privacy of Machine Learning, Spring 2024

Thursdays 9:10am - 12:10pm, CSIE Building, R111

You can sign up for this course at NTU COOL (instruction here). We will send out the permission numbers after the first class.

Instructor: Shang-Tse Chen
  • Email: stchen at csie.ntu.edu.tw
  • Office hour: after classes, or by appointment
TA: Bo-Han Kung
  • Email: d10922019 at ntu.edu.tw
  • Office hour: TBD
Modern machine learning models has reached and even surpassed human performance in many areas. However, many of the successful cases only hold in clean and controlled settings, which can be far from real scenarios. This course will introduce you to potential vulnerabilities of ML models. We will design and implement various attacks during model training and testing phases, as well as methods to make ML models more robust. We will also cover other important aspects of ML, including privacy and fairness.

Course Schedule Evolving

We will use NTU COOL for slides, homework assignments, announcement, and discussion.

Date Topics Reading Note
2/22 * Course introduction
* Adversarial attacks
2/29 Empirical defenses to evasion attacks
* Towards Deep Learning Models Resistant to Adversarial Attacks
* Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
3/7 Theoretical analysis of adversarial examples
* Adversarially Robust Generalization Requires More Data
* Robustness May Be at Odds with Accuracy
* Adversarial Examples Are Not Bugs, They Are Features
* Adversarial examples from computational constraints
3/14 Certified Defenses
* Evaluating robustness of neural networks with mixed integer programming
* Certified Defenses against Adversarial Examples
* Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability

Student presentation 1: Transferability
3/21 Certified Defenses
* MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radius
* Denoised Smoothing: A Provable Defense for Pretrained Classifiers
* LOT: Layer-wise Orthogonal Training on Improving l2 Certified Robustness
* Globally-Robust Neural Networks

Student presentation 2: Adversarial attack beyond L_p constraints
3/28 Poisoning attacks & defenses
* Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks
* Certified Defenses for Data Poisoning Attacks
* Spectral Signatures in Backdoor Attacks

Student presentation 3: Detection of adversarial examples
4/4 Holiday; no class
4/11 Confidentiality of ML models
* Towards Data-Free Model Stealing in a Hard Label Setting
* Increasing the Cost of Model Extraction with Calibrated Proof of Work
* Stealing Part of a Production Language Model

Student presentation 4: Attack / defense in 3D-based models
4/18 Differential Privacy I
* Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network
* Bilateral Dependency Optimization: Defending Against Model-inversion Attacks
* NetGuard: Protecting Commercial Web APIs from Model Inversion Attacks using GAN-generated Fake Samples

Student presentation 5: Robust overfitting
4/25 Differential Privacy II
* Deep Learning with Differential Privacy
* Scalable Private Learning with PATE
* Exploring the Benefits of Visual Prompting in Differential Privacy

Student presentation 6: Attack / defense in Graph Neural Networks
Final project proposal due
5/2 Fairness
* Differentially Private Database Release via Kernel Mean Embeddings
* Differentially Private Diffusion Models
* Differentially Private Fine-tuning of Language Models

Student presentation 7: Adversarial ML in LLM
5/9 ICLR 2024 Student presentation 9: Robustness and privacy in distributed learning
Student presentation 10: Privacy-robustness tradeoffs
Student presentation 11: Privacy in foundation models
5/16 Fairness
Student presentation 12: Connection between adversarial robustness and fairness
5/23 Fairness
Student presentation 13: Adversarial ML in VLM
5/30 Final project presentation
6/6 Final project presentation
6/13 Summer vacation starts!
Final project report due

Homework

  • You will need to do some programming with standard deep learning libraries (e.g., PyTorch, Tensorflow).
  • We will most likely use a mid-sized dataset, such as CIFAR-10. It will be easier for you if you have access to GPUs.
  • Try using Google Colab or applying for AWS education grant if you need computing resources.

Reading Critique

  • Choose a paper from the suggested reading list and write a paper critique of at most two pages.
  • The critique should address:
    • Summary of the paper
    • The strength of the paper
    • The weakness of the paper
    • Potential improvements
  • The summary is due at noon before each class, starting from the 2nd week.
  • Each critique is worth 3 points.
  • You can submit more than 5 critiques, and we will use the highest 5 scores.

Paper Presentation

  • A group of students (size TBD based on class size) will present and lead the discussion on an extended topic related to this course.
  • The presentation including QA should be within 50 minutes.
  • The presenter should answer live questions and comments under the recorded video on NTU COOL.

Class Participation

  • You get 1 point for each question asked to the student presenter in class.

Final Project

  • You will work in groups on a topic related to this course.
  • Example project format:
    • Tackle an open problem (not necessarily need to be successful)
    • Improve algorithms in a paper with techniques that you come up with
    • Apply the techniques you learned to novel applications
    • Benchmark algorithms from multiple papers and get some insights
    • Literature survey of some related areas that we did not cover
  • You need to turn in a 2-page proposal by 4/25.
  • Presentation should be similar to a conference talk (25 minutes presentation + 5 minites QA).
  • Final report should be typed with Latex (with NeurIPS format) and no more than 6 pages.

Grading Policy

  • Homework: 20%
  • Reading critique: 15%
  • Paper presentation: 20%
  • Class Participation: 5%
  • Project: 40%
    • Proposal (5%)
    • Presentation (15%)
    • Final report (20%)
  • All due times are at 11:59 pm the day before class.
    • No late submission is accepted.
    • Exception: you email Shang-Tse and get the approval before the deadline.