Administration

Intro – Long

Notebook – Intro to Python

Notebook – Intro to PyTorch

Linear Regression vs. Neural Network

Logistic Regression vs. Neural Network

General Linear Model (GLM) vs. Neural Network

Hidden Layers: 0 vs. 1 vs. 2

Backpropagation – Part 1

Backpropagation – Part 2

Implement a NN in NumPy

Notebook – Implementation Redo: Classes instead of Functions (NumPy)

Classification – Softmax and Cross Entropy – Theory

Classification – Softmax and Cross Entropy – Derivatives

Notebook – Implementing Classification (NumPy)

Automatic Differentiation

Backpropagation vs. Forward Propagation (Forward vs. Reverse mode autodiff)

Tanh & Permutation Symmetries

Notebook – Symmetries: tanh, permutations, ReLU

Generalization and the Bias-Variance Trade-off

Generalization Code

L2 Regularization / Weight Decay

Dropout Regularization

Notebook – Dropout implementation (NumPy)

Notebook – Early Stopping

Classical Activations

ReLU Variants

A Brief History of ReLU

SGD Variants: Momentum, NAG, AdaGrad, RMSprop, AdaDelta, Adam, AdaMax, Nadam- Part 1: Theory

SGD Variants: Momentum, NAG, AdaGrad, RMSprop, AdaDelta, Adam, AdaMax, Nadam – Part 2: Code

Variational Auto Encoders (VAE)