On the Smallness of the Large Language Models Scaling Exponents
Sauro Succi, Peter V. Coveney, Alex Hansen
cs.AI
Abstract
We discuss reasons why the scaling exponents of current Large Language Models (LLMs) applications are indicating an unsustainable regime in terms of energy resources. We further show that attributing the smallness of such exponents to a numerical bias due to the neglect of a non-zero value of the loss function in the limit of infinite data (``pedestal effect") does not remove the unsustainability issue. Finally, the effects of the smoothness (roughness) of the data on the scaling exponents is commented upon based on an analogy with phenomenological models of fluid turbulence.
Topics
Classified with taxonomy v2 on Wed, 2 Sept 2026.