Security and Privacy of Machine Learning, Fall 2021

Mondays 2:20 pm - 5:20 pm
The course will be held virtually for the first three weeks of the semester.
We will use this Google Meet link for the course.
We will switch back to hybrid teaching from 10/18 onwards (Localtion: CSIE room 101).

Instructor: Shang-Tse Chen
  • Email: stchen at csie.ntu.edu.tw
  • Office hour: after classes, or by appointment
 TA: Hung-Jui Wang
  • Email: r10922061 at ntu.edu.tw
  • Office hour: Fridays 1:30 - 2:30 pm (Google Meet)
Modern machine learning models has reached and even surpassed human performance in many areas. However, many of the successful cases only hold in clean and controlled settings, which can be far from real scenarios. This course will introduce you to potential vulnerabilities of ML models. We will design and implement various attacks during model training and testing phases, as well as methods to make ML models more robust. We will also cover other important aspects of ML, including privacy and fairness.

Course Schedule Evolving

We will use NTU COOL for slides, homework assignments, announcement, and discussion.

Date Topics Reading Note
9/27 * Course introduction
* Evasion attacks (i.e., adversarial examples)
 * Intriguing properties of neural networks
* Explaining and harnessing adversarial examples
* Towards Evaluating the Robustness of Neural Networks
10/4 Empirical defenses to evasion attacks
 * Towards Deep Learning Models Resistant to Adversarial Attacks
* Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
 HW1 out
10/11 Holiday; no class
10/18 Theoretical analysis of adversarial examples
 * Adversarially Robust Generalization Requires More Data
* Robustness May Be at Odds with Accuracy
* Adversarial Examples Are Not Bugs, They Are Features
* Adversarial examples from computational constraints

Student presentation: Transferability
  * Backpropagating Linearly Improves Transferability of Adversarial Examples
  * Admix: Enhancing the Transferability of Adversarial Attacks
  * Cross-Domain Transferability of Adversarial Perturbations
10/25 Certified Defenses
 * Evaluating robustness of neural networks with mixed integer programming
* Certified Defenses against Adversarial Examples
* Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability

Student presentation: Detection of adversarial examples
  * Detecting Adversarial Examples Is (Nearly) As Hard As Classifying Them
  * ML-LOO: Detecting Adversarial Examples with Feature Attribution
  * Provably robust classification of adversarial examples with detection
 HW1 Phase 1 due
11/1 Certified Defenses
 * Certified Adversarial Robustness via Randomized Smoothing
* Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

Student presentation: Adversarial attack beyond L_p constraints
  * Functional Adversarial Attacks
  * Perceptual Adversarial Robustness: Defense Against Unseen Threat Models
  * Towards Verifying Robustness of Neural Networks Against A Family of Semantic Perturbations
11/8 Poisoning attacks & defenses
* Spectral Signatures in Backdoor Attacks
* Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

Student presentation: Adversarial ML in NLP
  * TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP
  * Defense against Synonym Substitution-based Adversarial Attacks via Dirichlet Neighborhood Ensemble
  * Bad Characters: Imperceptible NLP Attacks
11/15 NTU Anniversary; no class
HW1 Phase 2 due
11/22 Confidentiality of ML models
 * Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
* Stealing Machine Learning Models via Prediction APIs

Student Presentation: Adversarial ML in audio
  * Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
  * Devil’s Whisper: A General Approach for Physical Adversarial Attacks against Commercial Black-box Speech Recognition Devices
  * WaveGuard: Understanding and Mitigating Audio Adversarial Examples
 Final project proposal due
11/29 Differential privacy
 * section 1 of this tutorial
* Deep Learning with Differential Privacy
* Scalable Private Learning with PATE

Student presentation: Adversarial ML in multi-modal learning
  * Cross-Modal Learning with Adversarial Samples
  * Towards Robust Sensor Fusion in Visual Perception
  * Can audio-visual integration strengthen robustness under multimodal attacks?
12/6 Differential privacy
 * Auditing Differentially Private Machine Learning: How Private is Private SGD?
* Differentially-Private Clustering of Easy Instances

Student presentation: Model interpretation and visualization
  * Interpreting Adversarially Trained Convolutional Neural Networks
  * Bluff: Interactively Deciphering Adversarial Attacks on Deep Neural Networks
  * Proper Network Interpretability Helps Adversarial Robustness in Classification
12/13 Differential privacy
 * Learning from Mixtures of Private and Public Populations
* GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators

Student presentation: Robustness and privacy in distributed learning
  * Provably Secure Federated Learning against Malicious Clients
  * Towards Federated Learning With Byzantine-Robust Client Weighting
  * InstaHide: Instance-hiding Schemes for Private Distributed Learning
12/20 Fairness
 * On Formalizing Fairness in Prediction with Machine Learning
* Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification

Student presentation: Connection between adversarial robustness and fairness
  * To be Robust or to be Fair: Towards Fairness in Adversarial Training
  * Individual Fairness Revisited: Transferring Techniques from Adversarial Robustness
  * Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning
12/27 Fairness
 * Training individually fair ML models with Sensitive Subspace Robustness
* Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting

Student presentation: Connection between privacy and fairness
  * On the Compatibility of Privacy and Fairness
  * Decision Making with Differential Privacy under a Fairness Lens
  * Differentially Private and Fair Deep Learning:A Lagrangian Dual Approach
1/3 Final project presentation
1/10 Final project presentation
1/17 Winter vacation starts!
 Final project report due

Homework

  • You will need to do some programming with standard deep learning libraries (e.g., PyTorch, Tensorflow).
  • We will most likely use a mid-sized dataset, such as CIFAR-10. It will be easier for you if you have access to GPUs.
  • Try using Google Colab or applying for AWS education grant if you need computing resources.

Reading Critique

  • Choose a paper from the suggested reading list and write a paper critique of at most two pages.
  • The critique should address:
    • Summary of the paper
    • The strength of the paper
    • The weakness of the paper
    • Potential improvements
  • The summary is due at noon before each class, starting from the 2nd week.
  • Each critique is worth 3 points.
  • You can submit more than 5 critiques, and we will use the highest 5 scores.
  • Student speakers of that week can only write a crituqe for the main reading list.

Paper Presentation

  • A group of students (size TBD based on class size) will present and lead the discussion on an extended topic related to this course.
  • The presentation including QA should be within 50 minutes.
  • The presenter should answer live questions and comments under the recorded video on NTU COOL.

Class Participation

  • You get 1 point for each question asked to the student presenter in class or a comment left under the recorded video on NTU COOL (within one week of the presentation).

Final Project

  • You will work in groups on a topic related to this course.
  • Example project format:
    • Tackle an open problem (not necessarily need to be successful)
    • Improve algorithms in a paper with techniques that you come up with
    • Apply the techniques you learned to novel applications
    • Benchmark algorithms from multiple papers and get some insights
    • Literature survey of some related areas that we did not cover
  • You need to turn in a 1-page proposal by 11/15.
  • Presentation should be similar to a conference talk (25 minutes presentation + 5 minites QA).
  • Final report should be typed with Latex (with NeurIPS format) and no more than 6 pages.

Grading Policy

  • Homework: 20%
  • Reading critique: 15%
  • Paper presentation: 20%
  • Class Participation: 5%
  • Project: 40%
    • Proposal (5%)
    • Presentation (15%)
    • Final report (20%)
  • All due times are at noon.
    • No late submission is accepted.
    • Exception: you email Shang-Tse and get the approval before the deadline.