Burst Spiking Neural Networks

A central goal of current Spiking Neural Network (SNN) research is to improve their accuracy toward becoming low-power alternatives to Artificial Neural Networks (ANNs). This work further argues that realizing this ambition requires improving not only accuracy but also robustness, defined as the ability to maintain correct predictions under input perturbations. We identify two key issues in existing SNN methods that undermine robustness. First, binary spiking activations can produce large activation-state changes under small perturbations. Second, the lack of effective weight constraints makes network outputs more sensitive to input variations. To this end, we propose Burst Spiking Neural Networks (BuSNNs), built upon Burst-enhanced Spiking Neurons (BSNs) and a Dynamic Weight Constraint (DWC) mechanism. BSNs incorporate burst firing to provide a graded spiking pattern. This spiking mechanism mitigates perturbation-induced transitions in activation states and thereby enhances robustness. DWC penalizes connection weights based on activation states, effectively reducing weight magnitudes and improving robustness while preserving accuracy. We provide theoretical analyses to support these robustness effects. Experimental results further show that, on smaller-scale benchmarks such as CIFAR-10, BuSNNs outperform both SNN and ANN counterparts in accuracy and robustness. On large-scale ImageNet, BuSNN with the MS ResNet-34 backbone further improves top-1 accuracy and corruption robustness over the corresponding SNN baseline by 3.18% and 2.66%, respectively. Despite using spike-based activations, BuSNNs surpass 4-bit activation-quantized ANN baselines and approach 8-bit ANN baselines on ImageNet. They also preserve SNNs' low-power advantage. This work studies the accuracy-robustness problem in SNNs, advancing their practical viability in robust and energy-efficient applications.

Quadratic integrate-and-fire neurons exhibit less fragmented loss landscapes and outperform leaky integrate-and-fire neurons in spike-based gradient descent

The ability to train spiking neural networks is essential for modeling biological neural networks as well as for neuromorphic computing. However, for the extensively used leaky integrate-and-fire (LIF) neurons, arbitrarily small parameter changes can induce spike (dis)appearances that disrupt subsequent activity, leading to unstable neural representations and permanently silent neurons during exact spike-based gradient descent. Recent work shows that a class of neuron models, which includes the quadratic integrate-and-fire (QIF) neuron, avoids these discontinuities and enables continuous and even smooth spike-based gradient descent. However, it remains unclear whether these advantages translate into practice. Here, we demonstrate that they do so via a controlled comparison between networks of LIF and QIF neurons on the popular Spiking Heidelberg Digits dataset. Specifically, in a first step, we perform a thorough hyperparameter search to optimize both models, revealing a clear performance advantage of QIF neurons. In a second step, we visualize the loss and gradient landscapes. Consistent with their inferior performance, we find that the loss landscapes of LIF neurons, which are discontinuous, appear more fragmented and the related gradients more erratic. An analysis of the landscapes of single samples indicates that these features arise from changes in the temporal order of spikes, which often cause disruptive spike (dis)appearances. Overall, our results advocate replacing LIF neurons with neuron models exhibiting continuous spiking dynamics, such as QIF neurons, for gradient descent training.