Automated Numerical Stability Analysis of Deep Learning Operators
Xinye Chen
Abstract
Finite-precision arithmetic unavoidably introduces numerical approximation errors. Numerical computations may use insufficient precision or an improper formulation, which leads to numerical instability. In this paper, we introduce the first unified software tool that integrates CESTAC for detecting the numerical stability of deep learning operators. Our developed software not only enables numerical validation with a single computation pass but also detects the sources of numerical instability and provides numerical stability monitoring during deep learning training and inference. We verified its effectiveness on the detection of polluted operators with injected numerical instabilities across various tasks. We believe that our developed method and tool provide valuable insights into developing numerically stable computing kernels, which are particularly critical for numerically stable and efficient deep learning training and inference.
Topics
Classified with taxonomy v2 on Wed, 2 Sept 2026.