Neural Networks (NN) have long been seen as mysterious and complex entities that are difficult to understand – like “black boxes” that power AI magic. But in reality, they’re non-linear parametric function approximators that work surprisingly well on a variety of problems with “compound structure”.
In this course, we aim to demystify NN and make them less intimidating.

We’ll cover the theory and practical aspects of neural networks, including how to create, train, and evaluate them using Python. We’ll be using the PyTorch framework, a popular tool for NN that comes with pre-built components. However, to fully understand the inner workings of NN, we’ll also code most of the course materials using only Python and NumPy. Most video classes include code examples that demonstrate the concepts in practice. By the end of the course, you’ll have a deep understanding of what neural networks are and how to use them effectively.
Curriculum
- 11 Sections
- 42 Lessons
- Lifetime
- Intro4
- Comparison to other methods3
- Expressivity (Capacity)1
- Training7
- 4.1Backpropagation – Part 1
- 4.2Backpropagation – Part 2
- 4.3Implement a NN in NumPy
- 4.4Notebook – Implementation Redo: Classes instead of Functions (NumPy)
- 4.5Classification – Softmax and Cross Entropy – Theory
- 4.6Classification – Softmax and Cross Entropy – Derivatives
- 4.7Notebook – Implementing Classification (NumPy)
- Autodiff2
- Symmetries in weight space2
- Generalization6
- Improved Training11
- 8.1Weight initialization – Part 1 – What not to do
- 8.2Notebook – Weight initialization Part 1
- 8.3Weight initialization – Part 2 – What to do
- 8.4Notebook – Weight initialization Part 2
- 8.5Notebook – TensorBoard
- 8.6Learning Rate Decay
- 8.7Notebook – Input Normalization
- 8.8Batch Normalization – Part 1: Theory
- 8.9Batch Normalization – Part 2: Derivatives
- 8.10Notebook – BatchNorm (PyTorch)
- 8.11Notebook – BatchNorm (NumPy)
- Activation Functions3
- Optimizers2
- Auto Encoders1
Requirements
- Linear Algebra
- Calculus
- Intro to Probability
- Intro to Statistics
- Python