The Abstract Machine

BREAKING APART – ADAM : A METHOD FOR STOCHASTIC OPTIMIZATION

ADAM (Adaptive Moment Estimation) is a, as the name suggests, adaptive optimization algorithm targeted mainly for training deep neural networks. It is special in the sense that – 1. It combines all the benefits of previous state-of-the-art techniques and eliminates most of their bottlenecks. 2. It is robust towards vanishing and exploding gradients because of…