Generalized nonparametric regression in reproducing kernel Hilbert spaces: Consistency and rates of convergence

We develop a comprehensive theory for regularized M-estimation in reproducing kernel Hilbert spaces. Under mild conditions on the loss we establish existence and measurability of the estimator, covering a wide range of convex and non-convex losses, including bounded robust losses. We further prove sharp rates of convergence with an explicit bias-variance decomposition governed by a novel complexity measure. We show that the variance is independent of misspecification, while the bias depends on a source condition parameter known in the learning literature. For tensor product Sobolev spaces we obtain new rates that connect to spaces of functions with dominating mixed smoothness, substantially extending existing results and explaining why these estimators circumvent the curse of dimensionality. Our methodology, combining elements from both functional analysis and empirical process theory, allows for an asymptotic linearisation of the objective function that avoids both closed-form solutions and global Lipschitz assumptions, and may be of independent interest. The estimators are implemented in C++ and theory is supported by numerical experiments.

Brownian Kernel Ladders

We introduce Brownian kernel ladders (BKLs), a recursive hierarchy of integral reproducing kernel Hilbert spaces built from linear functionals by repeatedly integrating Brownian pullback kernels indexed by functions from the preceding layer. The nonnegative 1-homogeneity of the Brownian kernel yields a kernel-preserving canonical spherical normalization and propagates square-root regularity through the hierarchy. Allowing all canonical ladder measures to vary produces a full adaptive BKL envelope with an infimal complexity. For this envelope, we prove depth-dependent Hölder and pointwise estimates, quasi-Banach structure, nestedness, and, under a geometric trace condition, strict growth with ballwise separation. We also establish existence of regularized empirical-risk minimizers for continuous losses uniformly bounded below, with almost-everywhere uniqueness of population predictions under strict convexity and pointwise uniqueness under full support. For statistical estimation, we study one realized ladder and finite dictionaries fixed independently of the estimation sample. For a dictionary of $M$ ladders, the Gaussian complexity of the union of radius-$r$ top-layer RKHS balls has $n^{-1/2}$ dependence, no explicit ambient-dimension factor, and model-selection factor $1+\sqrt{2\ln M}$. Corresponding high-probability oracle and excess-risk bounds follow; polynomial-size dictionaries retain a near-parametric rate. The theory separates adaptive representational richness from the statistical cost of ladder selection.