Controlling Refusal Behavior of LLMs via Stiefel-Constrained Rotation Steering
Kirill Bunin, Dmitry Bylinkin, Vladimir Aletov, Daniil Medyakov, Vladimir Solodkin, Aleksandr Beznosikov
Abstract
Activation steering has emerged as a lightweight approach for controlling model refusal at inference time. A growing line of research explores trainable rotations of activations to develop geometrically principled intervention mechanisms. However, existing techniques rely on auxiliary constructs, such as refusal vectors, to define these rotations. In our work, we develop a self-contained methodology for learning parameter-efficient rotational transformations based on Riemannian optimization. We empirically validate the proposed scheme, demonstrating its superiority in intervention efficiency. An extensive ablation study highlights the importance of key design choices in our method. Our results identify the proposed rotation-based steering scheme as a promising direction for more reliable control over the behavior of LLMs.
Topics
Classified with taxonomy v2 on Wed, 2 Sept 2026.