自校准首脉冲时间编码神经元模型

doi:10.11896/jsjkx.221200003

Abstract

Abstract: Because of the complex spatio-temporal dynamic process of spike neurons and the non-differentiable spike information,the training of spike neural network(SNN) has always been very difficult.The ANN-to-SNN method for indirect training of deep SNN avoids the difficulties of direct training of deep SNN.However,the performance of the SNN obtained in this approach is greatly affected by the spike information encoding mechanism.Among many coding mechanisms,TTFS has a good biological basis and is energy efficient,but existing TTFS codes use a single-spike formalism,which has weak information representation capability and large time windows for encoding.Therefore,based on the single spike coding of TTFS,a calibration spike is added to form a self-calibrating first spike time to first spike coding mechanism,and the corresponding SC-TTFS neuron model is constructed.In SC-TTFS,the first spike is the spike that must be emitted,while the calibration spike determines whether it is emitted according to the residual membrane potential after the first spike is emitted,which is used to compensate the quantification error and truncation error caused by the coding spike and to reduce the time window required for coding.The advantages of this approach are verified by comparing and analyzing the corresponding conversion errors of various codes and ANN-SNN conversion experiments on various network architectures.On CIFAR10 and CIFAR100 datasets,the proposed algorithm is verified by experiments based on VGG and ResNet network structures,and it achieves ANN-SNN transformation with non-destructive accuracy on both network structures and two data sets.Compared to state-of-the-art similar methods,the SNN constructed by the proposed method has the smallest network inference latency.In addition,on the VGG structure,the proposed method improves the energy efficiency by about 80% compared with TTFS coding.

Key words: Spiking neural network, Spike encoding mechanism, ANN-SNN conversion

CLC Number:

TP183

FENG Ren, CHEN Yunhua, XIONG Zhimin, CHEN Pinghua. Self-calibrating First Spike Temporal Encoding Neuron Model[J].Computer Science, 2024, 51(3): 244-250.

References

[1]ZHANG T L,XU B.Research Status and Prospect of PulseNeural Network [J].Journal of Computer Science,2021,44(9):1767-1785.
[2]BOHTE S M,KOK J N,LA POUTRÉ J A.Spikeprop: backpropagation for networks of spiking neurons[C]//ESANN.Bruges,2000:419-424.
[3]NEFTCI E O,MOSTAFA H,ZENKE F.Surrogate Gradient Learning in Spiking Neural Networks:Bringing the Power of Gradient-based optimization to spiking neural networks[J].IEEE Signal Processing Magazine,2019,36(6):51-63.
[4]LEE J H,DELBRUCK T,PFEIFFER M.Training deep spiking neural networks using backpropagation[J].Frontiers in Neuroscience,2016,10:508.
[5]MIRSADEGHI M,SHALCHIAN M,KHERADPISHEH S R,et al.Spike time displacement based error backpropagation in convolutional spiking neural networks[J].arXiv:2108.13621,2021.
[6]TAVANAEI A,MAIDA A S.Multi-layer unsupervised learningin a spiking convolutional neural network[C]//2017 International Joint Conference on Neural Networks(IJCNN).IEEE,2017:2023-2030.
[7]TAVANAEI A,MAIDA A.BP-STDP:Approximating back-propagation using spike timing dependent plasticity[J].Neurocomputing,2019,330:39-47.
[8]CAO Y,CHEN Y,KHOSLA D.Spiking deep convolutional neural networks for energy-efficient object recognition[J].International Journal of Computer Vision,2015,113(1):54-66.
[9]RUECKAUER B,LUNGU I A,HU Y,et al.Conversion of continuous-valued deep networks to efficient event-driven networks for image classification[J].Frontiers in Neuroscience,2017,11:682.
[10]HAN B,SRINIVASAN G,ROY K.Rmp-snn:Residual mem-brane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:13558-13567.
[11]MUELLER E,HANSJAKOB J,AUGE D,et al.Minimizing Inference Time:Optimization Methods for Converted Deep Spiking Neural Networks[C]//2021 International Joint Conference on Neural Networks(IJCNN).IEEE,2021.
[12]LI Y,DENG S,DONG X,et al.A free lunch from ANN:Towards efficient,accurate spiking neural networks calibration[C]//International Conference on Machine Learning.PMLR,2021:6316-6325.
[13]KIM J,KIM H,HUH S,et al.Deep neural networks withweighted spikes[J].Neurocomputing,2018,311:373-386.
[14]LI Y,ZENG Y,ZHAO D.Bsnn:Towards faster and better conversion of artificial neural networks to spiking neural networks with bistable neurons[J].arXiv:2105.12917,2021.
[15]RUECKAUER B,LIU S C.Temporal pattern coding in deep spiking neural networks[C]//2021 International Joint Con-ference on Neural Networks(IJCNN).IEEE,2021.
[16]PARK S,KIM S,CHOE H,et al.Fast and efficient information transmission with burst spikes in deep spiking neural networks[C]//2019 56th ACM/IEEE Design Automation Conference(DAC).IEEE,2019.
[17]GUO W,FOUDA M E,ELTAWIL A M,et al.Neural coding in spiking neural networks:A comparative study for robust neuromorphic systems[J].Frontiers in Neuroscience,2021,15:638474.
[18]RUECKAUER B,LIU S C.Conversion of analog to spiking neural networks using sparse temporal coding[C]//2018 IEEE International Symposium on Circuits and Systems(ISCAS).IEEE,2018.
[19]HAN B,ROY K.Deep spiking neural network:Energy efficien-cy through time based coding[C]//European Conference on Computer Vision.Cham:Springer,2020:388-404.
[20]ZHANG L,ZHOU S,ZHI T,et al.Tdsnn:From deep neural networks to deep spike neural networks with temporal-coding[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:1319-1326.
[21]PARK S,KIM S,NA B,et al.T2FSNN:Deep spiking neural networks with time-to-first-spike coding[C]//2020 57th ACM/IEEE Design Automation Conference(DAC).IEEE,2020.
[22]TOVEE M J,ROLLS E T.Information encoding in short firing rate epochs by single neurons in the primate temporal visual cortex[J].Visual cognition,1995,2(1):35-58.
[23]KUMAR S,KAPOSVARI P,VOGELS R.Encoding of predictable and unpredictable stimuli by inferior temporal cortical neurons[J].Journal of Cognitive Neuroscience,2017,29(8):1445-1454.
[24]STÖCKL C,MAASS W.Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes[J].Nature Machine Intelligence,2021,3(3):230-238.
[25]DAVIDSON S,FURBER S B.Comparison of artificial and spiking neural networks on digital hardware[J].Frontiers in Neuroscience,2021,15:651141.
[26]SIMONYAN K,ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv:1409.1556,2014.
[27]HE K,ZHANG X,REN S,et al.Deep residual learning forimage recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778.
[28]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[J/OL].https://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=0D60E5DD558A91470E0EA1725FF36E0A?doi=10.1.1.222.9220&rep=rep1&type=pdf.
[29]SENGUPTA A,YE Y,WANG R,et al.Going deeper in spiking neural networks:VGG and residual architectures[J].Frontiers in Neuroscience,2019,13:95.
[30]RATHI N,ROY K.DIET-SNN:A low-latency spiking neural network with direct input encoding and leakage and threshold optimization[J].IEEE Transactions on Neural Networks and Learning Systems,2021,34(6):3174-3182.
[31]MEROLLA P A,ARTHUR J V,ALVAREZ-ICAZA R,et al.A million spiking-neuron integrated circuit with a scalable communication network and interface[J].Science,2014,345(6197):668-673.
[32]FURBER S B,GALLUPPI F,TEMPLE S,et al.The spinnaker project[J].Proceedings of the IEEE,2014,102(5):652-665.
[33]MORADI S,MANOHAR R.The impact of on-chip communication on memory technologies for neuromorphic systems[J].Journal of Physics D:Applied Physics,2018,52(1):014003.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Self-calibrating First Spike Temporal Encoding Neuron Model

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 3

Metrics

Comments

Recommended 0

[1]	MA Weiwei, ZHENG Qinhong, LIU Shanshan. Study and Evaluation of Spiking Neural Network Model Based on Bee Colony Optimization [J]. Computer Science, 2023, 50(8): 221-225.
[2]	MA Weiqi, YUAN Jiabin, ZHA Keke, FAN Lili. Onboard Rock Detection Algorithm Based on Spiking Neural Network [J]. Computer Science, 2023, 50(1): 98-104.
[3]	HUANG Zenan, LIU Xiaojie, ZHAO Chenhui, DENG Yabin, GUO Donghui. Spiking Neural Network Model for Brain-like Computing and Progress of Its Learning Algorithm [J]. Computer Science, 2023, 50(1): 229-242.