Spiking neural networks (SNNs) are promoted as an energy-efficient substrate because sparse, event-driven activity replaces dense multiply-accumulates with cheap accumulates. We argue the energy dividend of sparsity is not a property of SNNs but of the task. Holding architecture fixed and swapping only the hidden unit (continuous vs. leaky-integrate-and-fire), plus a two-sided target-firing-rate probe, we measure how far activity can be pushed down before quality breaks. Low-load feed-forward perception sparsifies to 5% firing at no accuracy cost; a recurrent language model cannot go below ~50% -- the recurrent state must stay active to carry information. A spiking Transformer, by contrast, sparsifies freely to 2% (3 seeds) -- so the ceiling is a property of recurrent compression, not sequence modeling. Attention escapes the floor only by storing the full key-value cache, trading a firing floor for a memory wall: on neuromorphic hardware, recurrence and attention pay on different axes, neither escapes. We formalize the ceiling with an information-theoretic bound rho >= H_b^{-1}(log2 M / H) and confirm its predictions: the floor rises with memory load, falls with state width, and (refuting a naive memory-only reading) rises with task difficulty. A layer-wise input floor further caps op reduction under dense input, isolating event-driven perception as where neuromorphic hardware wins.
Imane Jriri, Tarik Houichime, Younes El Amranics.SE cs.CL
As Green Software Engineering matures, energy efficiency has transitioned into a mission-critical non-functional requirement. While software design patterns ensure structural integrity, their inherent abstraction layers impose an implicit "metabolic cost" that often remains obscured during the design phase. This paper empirically investigates the energy dynamics of the Memento design pattern, contrasting a direct, unabstracted baseline against Classic full-snapshot and Differential delta-encoding strategies. Leveraging the RAPL interface for high-fidelity hardware telemetry, we quantify energy dissipation across state volumes scaling from 10 MB to 200 MB. Our empirical results expose a critical architectural trade-off: the Differential strategy minimizes memory traffic, yielding a maximum energy reduction of 65.8% for mid-scale states, but collides with a catastrophic "memory wall" at 200 MB. At this saturation point, algorithmic optimizations are completely neutralized by severe GC thrashing and non-linear power spikes. We synthesize these findings into evidence-based heuristics, providing architects with a robust framework to reconcile structural design quality with sustainable Green IT imperatives.
A central goal of current Spiking Neural Network (SNN) research is to improve their accuracy toward becoming low-power alternatives to Artificial Neural Networks (ANNs). This work further argues that realizing this ambition requires improving not only accuracy but also robustness, defined as the ability to maintain correct predictions under input perturbations. We identify two key issues in existing SNN methods that undermine robustness. First, binary spiking activations can produce large activation-state changes under small perturbations. Second, the lack of effective weight constraints makes network outputs more sensitive to input variations. To this end, we propose Burst Spiking Neural Networks (BuSNNs), built upon Burst-enhanced Spiking Neurons (BSNs) and a Dynamic Weight Constraint (DWC) mechanism. BSNs incorporate burst firing to provide a graded spiking pattern. This spiking mechanism mitigates perturbation-induced transitions in activation states and thereby enhances robustness. DWC penalizes connection weights based on activation states, effectively reducing weight magnitudes and improving robustness while preserving accuracy. We provide theoretical analyses to support these robustness effects. Experimental results further show that, on smaller-scale benchmarks such as CIFAR-10, BuSNNs outperform both SNN and ANN counterparts in accuracy and robustness. On large-scale ImageNet, BuSNN with the MS ResNet-34 backbone further improves top-1 accuracy and corruption robustness over the corresponding SNN baseline by 3.18% and 2.66%, respectively. Despite using spike-based activations, BuSNNs surpass 4-bit activation-quantized ANN baselines and approach 8-bit ANN baselines on ImageNet. They also preserve SNNs' low-power advantage. This work studies the accuracy-robustness problem in SNNs, advancing their practical viability in robust and energy-efficient applications.