Spectral Perturbation of the Empirical Fisher Information Matrix under Weight Quantization

We study the spectral perturbation of the empirical Fisher Information Matrix (FIM) of a parametric statistical model under two structured perturbations: departure of the input from a reference (in-distribution) ensemble, and finite-precision (quantized) perturbation of the model's parameters. For the first, under an explicit local curvature-monotonicity hypothesis on the dominant eigenvalue lambda_max of the FIM, we show departure from a reference manifold provably elevates lambda_max relative to a calibration baseline (Proposition 3.2), and discuss why this hypothesis is required, since curvature need not increase monotonically under every perturbation. Our principal result is a directional eigenvalue perturbation bound, via Weyl's inequality, showing lambda_max under a quantization noise perturbation is lower bounded by its unperturbed value up to a third-order remainder, and, under a mild genericity condition, strictly exceeds it at leading order (Theorem 4.3). We give two tractable approximations to lambda_max -- one heuristic, one with a rigorous two-sided bound -- and a completeness result for a threshold-based partition of an augmented state space. These results motivate using sigma_t = lambda_max(F_t)/lambda_base as a runtime monitoring statistic for deployed language models: the quantization result offers a mechanism for an empirical observation of our own, where a calibration threshold for this statistic was approximately 244 times larger than a preliminary full-precision estimate on a 4-bit quantized model, a single measurement rather than a value derived in closed form. We report supporting measurements (twelve models, n=1,080 trajectories) broadly consistent with our predictions, discuss the scope and limitations of every result, and state as an open problem the closed-form prediction of the quantization inflation magnitude our bound does not supply.

Geometric bias in eigenspace perturbation under random heterogeneous noise

Spectral methods rely on the stability of principal eigenspaces under random perturbations. Classically, this is quantified by the Davis-Kahan and Wedin theorems, which bound the eigenspace error via the operator norm of the noise and the relevant spectral gaps. While sharp for arbitrary deterministic perturbations, these worst-case bounds can be wasteful in the low-rank signal-plus-noise setting, as they fail to capture the interaction between the signal geometry and the noise distribution. We study the spectral perturbation of signal-plus-noise matrices corrupted by sparse random noise with an arbitrary, inhomogeneous variance profile. Under heterogeneous variances, the empirical eigenvectors suffer a systematic, deterministic geometric bias invisible to classical bounds. Leveraging the Quadratic Vector Equation (QVE) and fine-grained isotropic local laws, we derive near-optimal, non-asymptotic bounds for the leading eigenspaces in the operator and 2-to-infinity norms. These separate the usual signal-to-noise contribution, stochastic fluctuations, and structured geometric bias terms determined by the alignment between the signal eigenspaces and the row-wise variance profile. We further develop refined rowwise bounds that adapt to the variance-weighted leverage of the signal space, yielding sharper guarantees in delocalized regimes. As applications, we establish strong consistency of adjacency spectral clustering for degree-corrected stochastic block models with heterogeneous degrees and unbalanced communities, recovering the logarithmic expected-degree scale in the regular balanced case. We also study spectral embedding for generalized random dot product graphs, showing that the full signal embedding admits sharp rowwise control, whereas spectral truncation can retain a systematic geometric bias determined by the omitted signal directions and the variance profile.