Lecture 10: Optimizers

Key Word(s): Optimizers, Adam, SGD, Momentum, RMSProp Adagrad, Gradient Clipping, Learning Rate

Slides

• Lecture 10 : Optimizers (PDF)
• Lecture 10 : Bias Correction (PDF)

Exercises

• Lecture 10: Exercise: Gradient Clipping [Notebook]
• Lecture 10: Exercise: RMSProp vs Learning Rate Decay [Notebook]