2021 ML Study Group Learning Path

Extremely grateful to Chun-Yi Lee, Chen-Hao Chao, Stanley, Chien Feng, Johnson Sun, Bob Cheng for organizing all these resources, preparing the assignments, and hosting this ML study group.

Lecture

• ML Lecture: Machine Learning by Andrew Ng
• CV Lecture: CS231n by Fei-Fei Li
• RL Lecture: Reinforcement Learning by David Silver

Progress

Machine Learning

Week 1

• ML Lecture 1
  • Introduction
  • Linear Regression with One Variable
  • Linear Algebra Review

Week 2

• ML Lecture 2
  • Linear Regression with Multiple Variables

Week 3

• ML Lecture 3
  • Logistic Regression
  • Regularization

Week 4

• ML Lecture 4
  • Neural Networks: Representation

Week 5

• ML Lecture 5
  • Neural Networks: Learning

Week 6

• ML Lecture 6
  • Advice for Applying Machine Learning
  • Machine Learning System Design

Week 7

• ML Lecture 7
  • Support Vector Machines

Week 8

• ML Lecture 8
  • Unsupervised Learning
  • Dimensionality Reduction
• Assignment 1: Python 101

Week 9

• ML Lecture 9
  • Anomaly Detection
  • Recommender Systems

Week 10

• ML Lecture 10
  • Large Scale Machine Learning

Computer Vision

Week 11

• CV Lecture 1, 2
  • Introduction to CNN for Visual Recognition
  • Image Classification
• Assignment 2: Dimension Reduction (PCA, t-SNE)

Week 12

• CV Lecture 3, 4
  • Loss Functions and Optimization
  • Introduction to Neural Networks
• Self Study & Representation
  • ImageNet, Cifar-10 Benchmarks
  • Distance Metrics (L1, L2)
  • Support Vector Machine (SVM)
  • KNN Algorithm
  • Softmax function & Cross-entropy loss
  • Computational Graphs (BP algorithm)
  • ReLu and other activation functions

Week 13

• CV Lecture 5, 6
  • Convolutional Neural Networks
  • Training Neural Networks 1
• Paper
  • Dropout Layer
  • Batch Normalization
  • Wasserstein Distance

Week 14

• CV Lecture 7, 8
  • Training Neural Networks 2
  • Deep Learning Software
• Assignment 3: SVM & K-means

Week 14

• CV Lecture 9, 10
  • CNN Architectures
  • Recurrent Neural Networks
• Paper
  • ResNet
  • GRU (Gated Recurrent Unit)
  • Object Detection (R-CNN v.s. YOLO)
  • Semantic Segmentation (FCN)

Week 15

• CV Lecture 11, 12
  • Detection and Segmentation
  • Visualizing and Understanding

Week 16

• CV Lecture 13
  • Generative Models
• Assignment 4: TensorFlow 2

Week 17

• Paper
  • Deeplab
  • FlowNet
  • Depth Estimation

Week 18

• Paper
  • Squeeze and Excitation
  • MobileNets
  • WGAN
  • Adversarial Attack

Week 19

• Assignment 5: AlexNet, Data augmentation, and Imbalanced Dataset

Reinforcement Learning

Week 20

• RL Lecture 1
  • Introduction to Reinforcement Learning

Week 21

• RL Lecture 2
  • Markov Decision Process

Week 22

• RL Lecture 3
  • Planning by Dynamic Programming

Week 23

• Assignment 6: Autoencoder & Manifold Learning

Week 24

• RL Lecture 4
  • Model-Free Prediction

Week 25

• RL Lecture 5
  • Model-Free Control

Week 26

• RL Lecture 6
  • Value Function Approximation

Week 27

• Assignment 7: DCGAN & WGAN

Week 28

• RL Lecture 7
  • Policy Gradient Methods

Week 29

• RL Lecture 8
  • Integrating Learning and Planning

Week 30

• Assignment 8: SARSA & Q-learning

Week 31

• RL Lecture 9
  • Exploration and Exploitation

Week 32

• Paper
  • DQN
  • Curiosity
  • Prioritized Experience Replay

Week 33

• Paper
  • Double DQN
  • Dueling Network
  • Asynchronous Methods for Deep Reinforcement Learning
  • Noisy Networks for Exploration

Week 34

• Paper
  • Rainbow DQN
  • DRQN

Week 35

• Paper
  • DDPG
  • PPO
  • RND
  • APE-X
  • HER

Week 36

• Assignment 9: DQN
• Self Study: PyTorch

Week 37

• Paper
  • TD3 (Continuous Control)
  • IMPALA (Distributed RL)
  • Never Give Up (Exploration)

Week 38

• Paper
  • D4PG
  • DQfD
  • SAC
  • Bootstrapped DQN

Week 39

• Assignment 10: DDPG, PPO

Week 40

• Paper
  • SVG(model based)
  • BCQ(offline RL)
  • Agent57

Week 41

• Paper
  • Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning (AWR)
  • Maximum a Posteriori Policy Optimisation (MPO)
  • Phasic Policy Gradient (PPG)
  • Diversity is All You Need: Learning Skills without a Reward Function (DIAYN)

Week 42

• Paper
  • Generative Adversarial Imitation Learning (Gail)
  • Distributional Reinforcement Learning with Quantile Regression (QRDQN)
  • Dream to Control: Learning Behaviors by Latent Imagination (Dreamer)

Week 43

• Intro. of Lab Environment
• Future Projects Introduction

Week 44

• Intro. of Unity Environments
• Mobius: RL Self-driving Car Training Platform

Week 45

• Assignment 11: SAC
• SGM

Week 46

• S2F2: Forecasting and Tracking

Week 47

• ROS

Week 48

• Rainbow UDA
• DAFormer
• RAFT