International Journal of Systems Engineering

Archive

Submit a Manuscript

Publishing with us to make your research visible to the widest possible audience.

Propose a Special Issue

Building a community of authors and readers to discuss the latest research and develop new ideas.

Time-Reduced Model for Multilayer Spiking Neural Networks

Spiking neural networks (SNNs) is a type of biological neural network model, which is more biologically plausible and computationally powerful than traditional artificial neural networks (ANNs). SNNs can achieve the same goals as ANNs, and can build a large-scale network structure (i.e. deep spiking neural network) to accomplish complex tasks. However, training deep spiking neural network is difficult due to the non-differentiable nature of spike events, and it requires much computation time during the training period. In this paper, a time-reduced model adopting two methods is presented for reducing the computation time of a deep spiking neural network (i.e. approximating the spike response function by the piecewise linear method, and choosing the suitable number of sub-synapses). The experimental results show that the methods of piecewise linear approximation and choosing the suitable number of sub-synapses is effective. This method can not only reduce the training time but also simplify the network structure. With the piecewise linear approximation method, the half of computation time of the original model can be reduced by at least. With the rule of choosing the number of sub-synapses, the computation time of less than one-tenth of the original model can be reduced for XOR and Iris tasks.

Spiking Neural Network, Computation Time, Linear Approximation, Sub-Synapses

APA Style

Yanjing Li. (2023). Time-Reduced Model for Multilayer Spiking Neural Networks. International Journal of Systems Engineering, 7(1), 1-8. https://doi.org/10.11648/j.ijse.20230701.11

ACS Style

Yanjing Li. Time-Reduced Model for Multilayer Spiking Neural Networks. Int. J. Syst. Eng. 2023, 7(1), 1-8. doi: 10.11648/j.ijse.20230701.11

AMA Style

Yanjing Li. Time-Reduced Model for Multilayer Spiking Neural Networks. Int J Syst Eng. 2023;7(1):1-8. doi: 10.11648/j.ijse.20230701.11

Copyright © 2023 Authors retain the copyright of this article.
This article is an open access article distributed under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. J. Wu, Y. Chua, and H. Li, “A biologically plausible speech recognition framework based on spiking neural networks,” in 2018 International Joint Conference on Neural Networks (IJCNN), 2018, pp. 1–8.
2. J. Liu and G. Zhao, “A bio-inspired SOSNN model for object recognition,” in 2018 International Joint Conference on Neural Networks (IJCNN), 2018, pp. 1–8.
3. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, “Deep neural networks for acoustic modeling in speech recognition,” IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82–97, 2012.
4. F. Karim, S. Majumdar, H. Darabi, and S. Chen, “LSTM fully convolutional networks for time series classification.,” IEEE Access, vol. 6, pp. 1662–1669, 2018.
5. S. Min, B. Lee, and S. Yoon, “Deep learning in bioinformatics,” Brief. Bioinform., vol. 18, no. 5, pp. 851–869, 2017.
6. W. Maass, “Networks of spiking neurons: The third generation of neural network models,” Neural Networks, vol. 10, no. 9, pp. 1659–1671, 1997.
7. J. P. Dominguez-morales, Q. Liu, R. James, D. Gutierrez-Galan, A. Jimenez-Fernandez, S. Davidson, and S. Furber, “Deep spiking neural network model for time-variant signals classification : a real-time speech recognition approach,” in 2018 International Joint Conference on Neural Networks (IJCNN), 2018, pp. 1–8.
8. Y. Lecun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
9. M. Courbariaux, Y. Bengio, and J.-P. David, “Binaryconnect: Training deep neural networks with binary weights during propagations,” arXiv Prepr. arXiv1511.00363, pp. 1–9, 2016.
10. P. Wang and J. Cheng, “Fixed-point factorized networks,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 3966–3974.
11. W. Chen, J. T. Wilson, S. Tyree, K. Q. Weinberger, and Y. Chen, “Compressing neural networks with the hashing trick,” in International Conference on International Conference on Machine Learning, 2015, vol. 37, pp. 2285–2294.
12. O. Booij and H. Tat Nguyen, “A gradient descent rule for spiking neurons emitting multiple spikes,” Inf. Process. Lett., vol. 95, no. 6, pp. 552–558, 2005.
13. Y. C. Yoon, “LIF and simplified SRM neurons encode signals into spikes via a form of asynchronous pulse sigma-delta modulation,” IEEE Trans. Neural Networks Learn. Syst., vol. 28, no. 5, pp. 1192–1205, 2017.
14. A. Tavanaei, M. Ghodrati, S. R. Kheradpisheh, T. Masquelier, and A. Maida, “Deep learning in spiking neural networks,” arXiv Prepr. arXiv1804.08150, pp. 1–18, 2018.
15. M. Zhang, J. Li, Y. Wang, and G. Gao, “R-tempotron: A robust tempotron learning rule for spike timing-based decisions,” in International Computer Conference on Wavelet Active Media Technology and Information Processing, 2016, pp. 139–142.
16. I. Sporea and A. Gruning, “Supervised learning in multilayer spiking neural networks,” Neural Comput., vol. 25, no. 2, pp. 473–509, 2013.
17. N. Soltani and A. J. Goldsmith, “Directed information between connected leaky integrate-and-fire neurons,” IEEE Trans. Inf. Theory, vol. 63, no. 9, pp. 5954–5967, 2017.
18. Y. Xu, X. Zeng, L. Han, and J. Yang, “A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks,” Neural Networks, vol. 43, no. 4, pp. 99–113, 2013.
19. Q. Yu, H. Tang, K. C. Tan, and H. Yu, “A brain-inspired spiking neural network model with temporal encoding and learning,” Neurocomputing, vol. 138, no. 11, pp. 3–13, 2014.
20. Y. Luo, Q. Fu, J. Liu, J. Harkin, L. McDaid, and Y. Cao, “An extended algorithm using an adaptation of momentum and learning rate for spiking neurons emitting multiple spikes,” Int. Work. Artif. Neural Networks, pp. 569–579, 2017.
21. Q. Kang, B. Huang, and M. Zhou, “Dynamic behavior of artificial Hodgkin-Huxley neuron model subject to additive noise,” IEEE Trans. Cybern., vol. 46, no. 9, pp. 2083–2093, 2016.