Feature selection is a highly relevant task in a data-driven knowledge discovery project. Several techniques have been developed aiming at finding the features that influence most an outcome to predict, including mutual information and, in recent years, the data-based sensitivity analysis. The present research focus on analyzing the advantages and disadvantages of each of these two techniques, by applying both to a bank telemarketing case. Thereafter, a logistic regression model is built on the tuned set of features identified by each of the two techniques as the most influencing set of features on the success of a telemarketing contact, in a total of 13 features for mutual information and 9 features for the data-based sensitivity analysis. The latter performs better for lower values of false positives while the former is slightly better for a higher false positive ratio. Thus, mutual information becomes a better choice if bank managers intend to reduce slightly the cost of contacts without risking losing a high number of successes. Such results show that mutual information, although not recent, is still a valid method for feature selection. On the other side, the data-based sensitivity analysis selection achieved good prediction results with less features.
Athanasios Vlontzos, Giorgos Papanastasiou, Bernhard Kainz +1cs.AI
Given a model that is already trained, which features does it rely on causally versus spuriously? Existing methods require access to the training procedure and cannot answer this post-hoc. We introduce the \textbf{Normalised Sensitivity Ratio~(NSR)}, a post-hoc, model-agnostic diagnostic for this question under a structured-shift regime: environments differ primarily in the mean of spurious features while the causal mechanism and causal marginals remain stable, as in multi-site clinical data or multi-batch genomics. Within this regime, causal features induce constant model sensitivity across environments while spurious features track shift. NSR formalises this as the squared coefficient of variation of per-environment sensitivity. Under a linear structural causal model (SCM) with $K\ge3$ non-degenerate environments, NSR achieves exact identification (Theorem~1). We fully characterise failure: weak shifts ($O(\varepsilon^4)$ collapse), degenerate geometry, and proxy attenuation ($O((1-α)^4)$), giving practitioners quantitative criteria for assessing whether the regime holds. Finite-sample rates are $O_p(n^{-1})$ under the null and $O_p(n^{-1/2})$ under the alternative. Experiments confirm all theoretical predictions on synthetic data (area under the ROC curve [AUROC] $= 1.000$ under conditions satisfying the regime), show consistent rankings across five model families (Kendall $τ\ge0.529$), and recover six of eight causal features on bike-sharing data (Precision@7 $= 0.75$) without modifying any trained model.
Active subspaces provide an explainable, eigenvalue-ordered principle for studying how scalar-valued quantities of interest change the most, on average, over a reduced basis of Euclidean domains. Composition with parallel transport generalizes this principle from Euclidean space to quantities of interest defined over Riemannian manifolds, and the resulting intrinsic formulation is contrasted with the extrinsic, embedding-based gradient average of manifold learning. Either strategy is studied in an intrinsically local sense, restricted to mean-centered geodesic-balls, and within that scope the two are not identical: on the central tangent space, eigenvalues agree to second order in the geodesic radius of the sampled domain, while dominant eigenspaces agree at the same order relative to the spectral gap. Extending activity beyond that central space then calls for either recomputed decompositions over changing tangent spaces or, intrinsically, parallel transport of a single central frame. Hyperspheres are emphasized throughout as a particular manifold of interest, motivated by applications over preshape spaces for statistical shape analysis. Numerical examples over the 2-sphere illustrate the formalism, including the derived ridge recovery at a curvature-limited quadratic rate.