Course Information

   Yun-Nung (Vivian) Chen
   Virtual (YT Video) / Physical (R104)
   Wednesday 09:10 - 12:10

Syllabus

Date Lecture Recitation Note
2024/09/04 Course Logistics [ 0]
Introduction [ 1.1] [ 1.2] [ 1.3] 		Dev Infra & Tooling
[ PyTorch] [ Debugging] 		Physical
Neural Network Basics [ 2.1] [ 2.2] [ 2.3] [ 2.4]
Backpropagation [ 2.5]
HW 0
2024/09/11 Sequence Modeling [ 3.1] [ 3.2] [ 3.3] [ 3.4]
Attention Mechanism [ 4.1] [ 4.2] 		NLP Lifecycle
[ Step by Step] [Colab] 		Physical
HW 1
[ hw 1]
彈性補充 Word Embeddings [ Intro] [ Word2Vec] [ Training]
[ Variants] [ GloVe] [ Eval] 		Physical
2024/09/18 Transformer [ 4.3] [ 4.4] [ 4.5]
Tokenization [ 5.1]
BERT [ 5.2]
BERT Variants [ 5.3] [ 5.4] [ 5.5] [ 5.6] 		Underlying Logics of Projects
[ Step by Step] 		Virtual
2024/09/25 NLG Decoding [ 6.1] [ 6.2] [ 6.3]
NLG Evaluation [ 6.4] [ 6.5]
Virtual
HW 2
[ hw 2]
2024/10/02 Typhoon Break
2024/10/09 Prompt-Based Learning [ 7.1] [ 7.2] [ 7.3]
[ 7.4] [ 7.5] [ 7.6] [ 7.7] 		LLM Basics & MoE [ Basics]
[ Architecture] [ MoE] 		Virtual
2024/10/16 LLM Adaptation [ 8.1] [ 8.2] [ 8.3] [ 8.4]
QA Session & TA Workshop 		LLM LoRA Training
[ LoRA] 		Physical
HW 3
[ hw 3]
2024/10/23 Midterm Break
2024/10/30 Conversational Modeling [ 9.1] [ 9.2] [ 9.3]
[ 9.4] [ 9.5] [ 9.6] [ 9.7] [ 9.8] 		Physical
2024/11/06 Final Project Announcement [ project]
Guest Lecture by Dr. Shao-Hua Sun 		Physical
2024/11/13 Break
2024/11/20 Retrieval-Augmented Generation [ 10.1] [ 10.2] [ 10.3] Physical
2024/11/27 Beyond Supervised Learning [ 11.1] [ 11.2] [ 11.3]
[ 11.4] [ 11.5] 		LLM Inference & Evaluation
[ Infer & Eval] 		Physical
2024/12/04 Language Agents [ 12.1] [ 12.2] [ 12.3]
[ 12.4] [ 12.5] 		LLM Deployment
[ Deployment] 		Physical
2024/12/11 Issues and Development in Pre-Trained Models
[ 13.1] [ 13.2] [ 13.3] [ 13.4] [ 13.5] 		Physical
2024/12/18 Final Project Presentation

Homework

The assignments should be submitted via NTU COOL.
NO LATE SUBMISSION IS ALLOWED!

  • Please make sure that you have computing resources for training deep models.
  • Please make sure that your GitHub repo for homework is private.

Teaching Team

Please send your questions to adl-ta@csie.ntu.edu.tw.
Please make sure the title of your mail starts with [ADL2024].

Role Name Work Office Hour Location
Head TA 林彥廷 Recitation By request
TA 黃兆緯 HW 1 R524
TA 郭芃廷 HW 2 R524
TA 張文毓 HW 3 R524
TA 周敦翔 Project R524
TA 陳剛詰 Project R524