Reproducing Recurrent Transformers: The CoTFormer
Aras Kavuncu, Bryan Vullo, Alberto Berni
cs.LG
Abstract
The CoTFormer architecture formalizes Chain-of-Thought as a form of recurrent latent computation, preserving intermediate states as attendable representations to mimic explicit reasoning traces. In this work, we evaluate CoTFormer and its structural variants across perplexity and compute efficiency metrics. Furthermore, we extend evaluation to controlled algorithmic settings to determine whether this recurrent framework improves out-of-distribution generalisation on inductive reasoning tasks.
Topics
Classified with taxonomy v2 on Sat, 5 Sept 2026.