{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,6]],"date-time":"2025-10-06T09:21:23Z","timestamp":1759742483965},"reference-count":24,"publisher":"MIT Press - Journals","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2019,6]]},"abstract":"<jats:p>Although deep neural networks (DNNs) have led to many remarkable results in cognitive tasks, they are still far from catching up with human-level cognition in antinoise capability. New research indicates how brittle and susceptible current models are to small variations in data distribution. In this letter, we study the stochasticity-resistance character of biological neurons by simulating the input-output response process of a leaky integrate-and-fire (LIF) neuron model and proposed a novel activation function, rand softplus (RSP), to model the response process. In RSP, a scale factor [Formula: see text] is employed to mimic the stochasticity-adaptability of biological neurons, thereby enabling the antinoise capability of a DNN to be improved by the novel activation function. We validated the performance of RSP with a 19-layer residual network (ResNet) and a 19-layer visual geometry group (VGG) on facial expression recognition data sets and compared it with other popular activation functions, such as rectified linear units (ReLU), softplus, leaky ReLU (LReLU), exponential linear unit (ELU), and noisy softplus (NSP). The experimental results show that RSP is applied to VGG-19 or ResNet-19, and the average recognition accuracy under five different noise levels exceeds the other functions on both of the two facial expression data sets; in other words, RSP outperforms the other activation functions in noise resistance. Compared with the application in ResNet-19, the application of RSP in VGG-19 can improve a network's antinoise performance to a greater extent. In addition, RSP is easier to train compared to NSP because it has only one parameter to be calculated automatically according to the input data. Therefore, this work provides the deep learning community with a novel activation function that can better deal with overfitting problems.<\/jats:p>","DOI":"10.1162\/neco_a_01192","type":"journal-article","created":{"date-parts":[[2019,4,13]],"date-time":"2019-04-13T00:05:57Z","timestamp":1555113957000},"page":"1215-1233","source":"Crossref","is-referenced-by-count":20,"title":["Improving the Antinoise Ability of DNNs via a Bio-Inspired Noise Adaptive Activation Function Rand Softplus"],"prefix":"10.1162","volume":"31","author":[{"given":"Yunhua","family":"Chen","sequence":"first","affiliation":[{"name":"School of Computers, Guangdong University of Technology, Guangzhou 51006, China"}]},{"given":"Yingchao","family":"Mai","sequence":"additional","affiliation":[{"name":"School of Computers, Guangdong University of Technology, Guangzhou 51006, China"}]},{"given":"Jinsheng","family":"Xiao","sequence":"additional","affiliation":[{"name":"School of Electronic Information, Wuhan University, Wuhan 430072, China"}]},{"given":"Ling","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computers, Guangdong University of Technology, Guangzhou 51006, China"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1097\/00004647-200110000-00001"},{"key":"B2","author":"Clevert D. A.","year":"2015","journal-title":"Fast and accurate deep network learning by exponential linear units (ELUS)"},{"key":"B3","first-page":"248","author":"Deng J.","year":"2009","journal-title":"Proceedings of the IEEE conference on Computer Vision and Pattern Recognition"},{"key":"B4","first-page":"472","volume-title":"Advances in neural information processing systems, 13","author":"Dugas C.","year":"2001"},{"key":"B5","first-page":"315","author":"Glorot X.","year":"2011","journal-title":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics"},{"key":"B6","first-page":"770","author":"He K.","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.7.1527"},{"key":"B8","author":"Hunsberger E.","year":"2015","journal-title":"Spiking deep networks with LIF neurons"},{"key":"B9","first-page":"543","author":"Kahou S. E.","year":"2013","journal-title":"Proceedings of the International Conference on Multimodal Interaction"},{"key":"B10","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1145\/2818346.2830590","author":"Kim B. K.","year":"2015","journal-title":"Proceedings of the International Conference on Multimodal Interaction"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1007\/s00422-008-0272-7"},{"key":"B12","author":"LeCun Y.","year":"2010","journal-title":"MNIST handwritten digit database"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1016\/S0960-9822(03)00135-0"},{"key":"B14","author":"Liu M.","year":"2014","journal-title":"Deeply learning deformable facial action parts model for dynamic expression analysis"},{"key":"B15","author":"Liu Q.","year":"2017","journal-title":"Noisy softplus: An activation function that enables SNNs to be trained as ANNs"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46681-1_49"},{"key":"B18","author":"Maas A. L.","year":"2013","journal-title":"Proceedings of the ICML Workshop on Deep Learning for Audio, Speech and Language Processing"},{"key":"B19","first-page":"807","author":"Nair V.","year":"2010","journal-title":"Proceedings of the International Conference on International Conference on Machine Learning"},{"key":"B20","author":"Recht B.","year":"2018","journal-title":"Do cifar-10 classifiers generalize to cifar-10?"},{"key":"B21","author":"Simonyan K.","year":"2014","journal-title":"Very deep convolutional networks for large-scale image recognition"},{"key":"B22","first-page":"2066","author":"Sun S.","year":"2016","journal-title":"Proceedings of the 30th International Conference of the Association for the Advancement of Artificial Intelligence"},{"key":"B23","first-page":"1","author":"Szegedy C.","year":"2015","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B24","first-page":"1","author":"Whitehill J.","year":"2008","journal-title":"Proceedings of the Eighth IEEE International Conference on Automatic Face and Gesture Recognition"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2017.11.012"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco_a_01192","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:43:09Z","timestamp":1615585389000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/31\/6\/1215-1233\/8476"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6]]},"references-count":24,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,6]]}},"alternative-id":["10.1162\/neco_a_01192"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_01192","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6]]}}}