Adam In detail
Optimization (218 pages)
Optimization Overhead (264 pages)
Adapting • Advantages • Applies • Average • Backpropagation • Benefits • Clustering • Compared • Component • Consists • Constant • Correction • Decaying • Does • Dropout • During • Each • Effectively • Ensure • Estimates • Faster • First • Fixed • Fraction • From • Global • Gradient • Handle • Large • Magnitude • Memory • Moments • Neural • Optimal • Parameters • Particularly • Process • Rate • Regularizing • Scales • Speeds • Stability • Stochastic • Technique • Throughout • Update • Updates • Used • Using • Which