Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models
Okan S. Coskun, Florian Rottach, Carsten Eickhoff, William Rudman
Abstract
We investigate the geometry of decision-making in Multiple Choice Question Answering (MCQA) through the lens of isotropy. Analyzing five open-weight models across diverse datasets, we identify decision-critical transition layers characterized by a shift in isotropy, coinciding with a major representational change and the emergence of task-relevant clusters. We demonstrate that this synchronized geometric behavior is strongly correlated with downstream accuracy ($r\approx0.84$), displaying its relevance for successful decision-making. Furthermore, we show that this transition is robust to prompt variations, suggesting that it reflects a general mechanism of model behavior.
Topics
Classified with taxonomy v2 on Sat, 5 Sept 2026.