Auditable memory requires a precise contract: which output is preserved, relative to which reference solve, and across which updates. We introduce Support Vector Attention (SV-Attention), a one-class support vector data description (SVDD) gate whose coefficients enter the readout. Zero-coefficient keys are reserve; positive-coefficient keys are active. Removing a reserve token without re-solving preserves the current readout. Maintained deletion, which updates the existing solver state, targets a fresh retained-key fit under the same coefficient cap C. Across 1,200 fp64 deletion/refit trials on Gaussian, redundant, MIMIC-IV, and learned keys, 1,199 complete. Median maximum gate-score discrepancy over declared probes ranges from 4.5e-13 to 5.7e-7, and fresh refitting is 24-223 times slower than maintained deletion on the reference CPU. A deterministic example shows that current reserve status does not guarantee equivalence after future admissions. At matched token counts, rare-group recall is 0.861 versus 0.319 for an oracle attention-mass proxy (H2O-style). In a held-out-channel ICU control, SpO2 below 90 percent defines events but SpO2 is excluded from every selector; event-hour retention is 0.464 versus 0.225 for an RBF-density baseline. A separate batched approximation supports end-to-end training; at 3.22M parameters, seven paired seeds yield mean best-validation bits per character of 2.178 versus 2.383 (p=0.001). The contracts are point-in-time and fixed-C; future-safe streaming and general-purpose performance remain open, and larger fixed-step runs suggest slower optimization.
Modern spoken language understanding (SLU) systems are increasingly deployed in real-world settings, where specific functionalities may need to be removed due to policy or safety constraints. In SLU, a functionality corresponds to an intent and its associated slot-generation behavior. However, in autoregressive models, suppressing a target intent does not eliminate the conditional mapping that generates slots conditioned on that intent. When the intent prefix is externally supplied, the model can reconstruct the original intent-slot structure. We identify this structural failure as \textbf{\emph{capability persistence}}. We propose \textit{\underline{B}inding \underline{S}ubspace (BSU)}, a representation-level framework that isolates and attenuates intent-conditioned directions underlying this mapping. Across SLU benchmarks, BSU substantially reduces forced-prefix recoverability while preserving retained performance.
We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social-reasoning versus STEM-reasoning in OLMo3-7B. Training-data attribution measures how strongly each training document influences a model's predictions on a benchmark, but document-level scores are too noisy to identify which corpus regions support which capabilities. We compute gradient-based attribution (TrackStar via Bergson) over a working set drawn from the de-duplicated Dolma3 mix, aggregate influence across WebOrganizer's 24-format x 24-topic taxonomy (576 bins), and contrast benchmark pairs in a 2x2 design that varies domain (social vs. STEM) and capability type (reasoning vs. knowledge): SocialIQA and MMLU Social Sciences against ARC-Challenge and MMLU STEM. Social and STEM reasoning draw on qualitatively distinct corpus regions, and the contrast is sharper at the reasoning level than at the knowledge level. Targeted machine unlearning provides partial causal validation: forgetting high-attribution topics (e.g., Literature for SocialIQA) degrades the aligned benchmark more than within-topic random baselines. We validate on other open-data model, Comma v0.1 7B-2T (Common Pile) and DCLM-Baseline-7B (DataComp-LM): causal selectivity holds on both models, while the provenance map is ecosystem-specific. We open-source all code, data artifacts, influence scores, and checkpoints at https://github.com/eilab-gt/capabilibara and https://huggingface.co/HCAI-Lab.