Central limit theorem for the averaged Adam optimizer
Steffen Dereich, Arnulf Jentzen
math.PR cs.LG math.ST
Abstract
In this article, we analyse convergence of the averaged Adam optimizer to an attracting zero of the Adam vector field. We provide a central limit theorem that, in particular, quantifies exactly the speed of convergence. The order of convergence is $n^{-1/2}$ in the number of steps of the algorithm which coincides with the order observed for classical stochastic approximation algorithms. The covariance in the central limit theorem is given in terms of properties of the Adam algorithm in the state of the attractor.
Topics
Classified with taxonomy v2 on Sat, 5 Sept 2026.