{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T12:58:52Z","timestamp":1778763532046,"version":"3.51.4"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"13","license":[{"start":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T00:00:00Z","timestamp":1709078400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T00:00:00Z","timestamp":1709078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Bandirma Onyedi Eylul University"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2024,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Activation functions are used to extract meaningful relationships from real-world problems with the help of deep learning models. Thus, the development of activation functions which affect deep learning models\u2019 performances is of great interest to researchers. In the literature, mostly, nonlinear activation functions are preferred since linear activation functions limit the learning performances of the deep learning models. Non-linear activation functions can be classified as fixed-parameter and trainable activation functions based on whether the activation function parameter is fixed (i.e., user-given) or modified during the training process of deep learning models. The parameters of the fixed-parameter activation functions should be specified before the deep learning model training process. However, it takes too much time to determine appropriate function parameter values and can cause the slow convergence of the deep learning model. In contrast, trainable activation functions whose parameters are updated in each iteration of deep learning models training process achieve faster and better convergence by obtaining the most suitable parameter values for the datasets and deep learning architectures. This study proposes parametric RSigELU (P+RSigELU) trainable activation functions, such as P+RSigELU Single (P+RSigELUS) and P+RSigELU Double (P+RSigELUD), to improve the performance of fixed-parameter activation function of RSigELU. The performances of the proposed trainable activation functions were evaluated on the benchmark datasets of MNIST, CIFAR-10, and CIFAR-100 datasets. Results show that the proposed activation functions outperforms PReLU, PELU, ALISA, P+FELU, PSigmoid, and GELU activation functions found in the literature. To access the codes of the activation function; <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/serhatklc\/P-RsigELU-Activation-Function\">https:\/\/github.com\/serhatklc\/P-RsigELU-Activation-Function<\/jats:ext-link>.<\/jats:p>","DOI":"10.1007\/s00521-024-09538-9","type":"journal-article","created":{"date-parts":[[2024,2,28]],"date-time":"2024-02-28T13:02:41Z","timestamp":1709125361000},"page":"7595-7607","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Parametric RSigELU: a new trainable activation function for deep learning"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9483-4425","authenticated-orcid":false,"given":"Serhat","family":"Kili\u00e7arslan","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mete","family":"Celik","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,2,28]]},"reference":[{"key":"9538_CR1","doi-asserted-by":"crossref","unstructured":"Adem K, Kili\u00e7arslan S (2019) Performance analysis of optimization algorithms on stacked autoencoder. In: 2019 3rd international symposium on multidisciplinary studies and innovative technologies (ISMSIT) (pp. 1\u20134). IEEE","DOI":"10.1109\/ISMSIT.2019.8932880"},{"issue":"24","key":"9538_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00521-022-07625-3","volume":"34","author":"K Adem","year":"2022","unstructured":"Adem K (2022) P+FELU: flexible and trainable fast exponential linear unit for deep learning architectures. Neural Comput Appl 34(24):1\u201312","journal-title":"Neural Comput Appl"},{"issue":"6","key":"9538_CR3","doi-asserted-by":"publisher","first-page":"4220","DOI":"10.3906\/elk-1903-112","volume":"27","author":"K Adem","year":"2019","unstructured":"Adem K, K\u00f6zkurt C (2019) Defect detection of seals in multilayer aseptic packages using deep learning. Turk J Electr Eng Comput Sci 27(6):4220\u20134230","journal-title":"Turk J Electr Eng Comput Sci"},{"key":"9538_CR4","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1016\/j.eswa.2018.08.050","volume":"115","author":"K Adem","year":"2019","unstructured":"Adem K, Kili\u00e7arslan S, C\u00f6mert O (2019) Classification and diagnosis of cervical cancer with stacked autoencoder and softmax classification. Expert Syst Appl 115:557\u2013564","journal-title":"Expert Syst Appl"},{"key":"9538_CR44","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1016\/j.eswa.2018.11.042","volume":"120","author":"VS Bawa","year":"2019","unstructured":"Bawa VS,\u00a0 Kumar V (2019)\u00a0Linearized sigmoidal activation: A novel activation function with tractable non-linear characteristics to boost representation capability Expert Systems with Applications 120346-356 https:\/\/doi.org\/10.1016\/j.eswa.2018.11.042","journal-title":"Expert Systems with Applications"},{"key":"9538_CR5","doi-asserted-by":"publisher","first-page":"120613","DOI":"10.1109\/ACCESS.2021.3105355","volume":"9","author":"K Biswas","year":"2021","unstructured":"Biswas K, Kumar S, Banerjee S, Pandey AK (2021) TanhSoft\u2014dynamic trainable activation functions for faster learning and better performance. IEEE Access 9:120613\u2013120623","journal-title":"IEEE Access"},{"issue":"1","key":"9538_CR6","first-page":"33","volume":"13","author":"MA B\u00fclb\u00fcl","year":"2023","unstructured":"B\u00fclb\u00fcl MA (2023) Kuru Fasulye Tohumlar\u0131n\u0131n \u00c7ok S\u0131n\u0131fl\u0131 S\u0131n\u0131fland\u0131r\u0131lmas\u0131 \u0130\u00e7in Hibrit Bir Yakla\u015f\u0131m. J Inst Sci Technol 13(1):33\u201343","journal-title":"J Inst Sci Technol"},{"key":"9538_CR7","doi-asserted-by":"crossref","unstructured":"Chieng HH, Wahid N, Ong P (2020) Parametric flatten-T swish: an adaptive non-linear activation function for deep learning. arXiv preprint arXiv:2011.03155","DOI":"10.32890\/jict.20.1.2021.9267"},{"key":"9538_CR8","unstructured":"Clevert DA, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (elus). arXiv preprint arXiv:1511.07289"},{"issue":"3","key":"9538_CR9","doi-asserted-by":"publisher","first-page":"219","DOI":"10.17671\/gazibtd.457917","volume":"12","author":"ACI \u00c7i\u011fdem","year":"2019","unstructured":"\u00c7i\u011fdem ACI, \u00c7IRAK, A. (2019) T\u00fcrk\u00e7e Haber Metinlerinin Konvol\u00fcsyonel Sinir A\u011flar\u0131 ve Word2Vec Kullan\u0131larak S\u0131n\u0131fland\u0131r\u0131lmas\u0131. Bili\u015fim Teknolojileri Dergisi 12(3):219\u2013228","journal-title":"Bili\u015fim Teknolojileri Dergisi"},{"issue":"3\u20134","key":"9538_CR10","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1561\/2000000039","volume":"7","author":"L Deng","year":"2014","unstructured":"Deng L, Yu D (2014) Deep learning: methods and applications. Found trends\u00ae in signal process 7(3\u20134):197\u2013387","journal-title":"Found trends\u00ae in signal process"},{"key":"9538_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2022.105635","volume":"148","author":"A Diker","year":"2022","unstructured":"Diker A (2022) An efficient model of residual based convolutional neural network with Bayesian optimization for the classification of malarial cell images. Comput Biol Med 148:105635","journal-title":"Comput Biol Med"},{"key":"9538_CR12","doi-asserted-by":"publisher","DOI":"10.1016\/j.datak.2022.102075","volume":"141","author":"E D\u00f6nmez","year":"2022","unstructured":"D\u00f6nmez E (2022) Enhancing classification capacity of CNN models with deep feature selection and fusion: a case study on maize seed classification. Data Knowl Eng 141:102075","journal-title":"Data Knowl Eng"},{"issue":"2","key":"9538_CR13","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1007\/s11760-020-01746-9","volume":"15","author":"I El Jaafari","year":"2021","unstructured":"El Jaafari I, Ellahyani A, Charfi S (2021) Parametric rectified nonlinear unit (PRenu) for convolution neural networks. SIViP 15(2):241\u2013246","journal-title":"SIViP"},{"issue":"8","key":"9538_CR14","doi-asserted-by":"publisher","first-page":"10579","DOI":"10.1007\/s13369-022-06654-3","volume":"47","author":"A Elen","year":"2022","unstructured":"Elen A, Ba\u015f S, K\u00f6zkurt C (2022) An adaptive Gaussian kernel for support vector machine. Arab J Sci Eng 47(8):10579\u201310588","journal-title":"Arab J Sci Eng"},{"key":"9538_CR15","unstructured":"Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. In proceedings of the thirteenth international conference on artificial intelligence and statistics (pp. 249\u2013256). JMLR workshop and conference proceedings"},{"key":"9538_CR16","doi-asserted-by":"crossref","unstructured":"Godfrey LB (2019) An evaluation of parametric activation functions for deep learning. In: IEEE international conference on systems, man and cybernetics (SMC). 3006\u20133011","DOI":"10.1109\/SMC.2019.8913972"},{"key":"9538_CR17","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1016\/j.patrec.2018.09.006","volume":"116","author":"F Godin","year":"2018","unstructured":"Godin F, Degrave J, Dambre J, De Neve W (2018) Dual rectified linear units (DReLUs): a replacement for tanh activation functions in quasi-recurrent neural networks. Pattern Recogn Lett 116:8\u201314","journal-title":"Pattern Recogn Lett"},{"key":"9538_CR18","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In proceedings of the IEEE international conference on computer vision (pp. 1026\u20131034)","DOI":"10.1109\/ICCV.2015.123"},{"key":"9538_CR19","unstructured":"Hendrycks D, Gimpel K (2016). Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415"},{"issue":"2","key":"9538_CR20","doi-asserted-by":"publisher","first-page":"2076","DOI":"10.3390\/app13020809","volume":"13","author":"E I\u015f\u0131k","year":"2023","unstructured":"I\u015f\u0131k E, Ademovi\u0107 N, Harirchian E, Avcil F, B\u00fcy\u00fcksara\u00e7 A, Hadzima-Nyarko M, Antep B (2023) Determination of natural fundamental period of minarets by using artificial neural network and assess the impact of different materials on their seismic vulnerability. Appl Sci 13(2):2076\u20133417","journal-title":"Appl Sci"},{"issue":"4","key":"9538_CR21","doi-asserted-by":"publisher","first-page":"6345","DOI":"10.1007\/s11042-022-14313-w","volume":"82","author":"S Kili\u00e7arslan","year":"2023","unstructured":"Kili\u00e7arslan S (2023) A novel nonlinear hybrid HardSReLUE activation function in transfer learning architectures for hemorrhage classification. Multimed Tools Appl 82(4):6345\u20136365","journal-title":"Multimed Tools Appl"},{"issue":"1","key":"9538_CR22","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/s12652-022-04433-4","volume":"14","author":"S Kili\u00e7arslan","year":"2023","unstructured":"Kili\u00e7arslan S (2023) PSO\u2009+\u2009GWO: a hybrid particle swarm optimization and grey wolf optimization based algorithm for fine-tuning hyper-parameters of convolutional neural networks for cardiovascular disease detection. J Ambient Intell Humaniz Comput 14(1):87\u201397","journal-title":"J Ambient Intell Humaniz Comput"},{"key":"9538_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2020.102231","volume":"63","author":"S Kilicarslan","year":"2021","unstructured":"Kilicarslan S, Celik M, Sahin \u015e (2021) Hybrid models based on genetic algorithm and deep learning algorithms for nutritional anemia disease classification. Biomed Signal Proc Control 63:102231","journal-title":"Biomed Signal Proc Control"},{"key":"9538_CR24","doi-asserted-by":"publisher","first-page":"114805","DOI":"10.1016\/j.eswa.2021.114805","volume":"174","author":"S Kili\u00e7arslan","year":"2021","unstructured":"Kili\u00e7arslan S, Celik M (2021) RSigELU: a nonlinear activation function for deep neural networks. Expert Syst Appl 174:114805","journal-title":"Expert Syst Appl"},{"key":"9538_CR25","doi-asserted-by":"publisher","first-page":"109577","DOI":"10.1016\/j.mehy.2020.109577","volume":"137","author":"S Kili\u00e7arslan","year":"2020","unstructured":"Kili\u00e7arslan S, Adem K, Celik M (2020) Diagnosis and classification of cancer using hybrid model based on relieff and convolutional neural network. Med Hypothese 137:109577","journal-title":"Med Hypothese"},{"issue":"3","key":"9538_CR26","doi-asserted-by":"publisher","first-page":"75","DOI":"10.54187\/jnrs.1011739","volume":"10","author":"S Kili\u00e7arslan","year":"2021","unstructured":"Kili\u00e7arslan S, Adem K, \u00c7elik M (2021) An overview of the activation functions used in deep learning algorithms. J New Result Sci 10(3):75\u201388","journal-title":"J New Result Sci"},{"key":"9538_CR27","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2023.119503","volume":"217","author":"S Kili\u00e7arslan","year":"2023","unstructured":"Kili\u00e7arslan S, K\u00f6zkurt C, Ba\u015f S, Elen A (2023) Detection and classification of pneumonia using novel superior exponential (supex) activation function in convolutional neural networks. Expert Syst Appl 217:119503","journal-title":"Expert Syst Appl"},{"key":"9538_CR28","first-page":"971","volume":"30","author":"G Klambauer","year":"2017","unstructured":"Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) Self-normalizing neural networks. In Adv neural inf process syst 30:971\u2013980","journal-title":"In Adv neural inf process syst"},{"issue":"24","key":"9538_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00500-023-09279-2","volume":"27","author":"C K\u00f6zkurt","year":"2023","unstructured":"K\u00f6zkurt C, Kili\u00e7arslan S, Ba\u015f S, Elen A (2023) \u03b1 SechSig and \u03b1TanhSig: two novel non-monotonic activation functions. Soft Comput 27(24):1\u201317","journal-title":"Soft Comput"},{"key":"9538_CR30","first-page":"1106","volume":"25","author":"A Krizhevsky","year":"2012","unstructured":"Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. In: adv neural inf process syst 25:1106\u20131114","journal-title":"In: adv neural inf process syst"},{"key":"9538_CR31","unstructured":"Krizhevsky A, Hinton G. (2009) Learning multiple layers of features from tiny images. Master\u2019s thesis, University of Tront."},{"issue":"11","key":"9538_CR32","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","volume":"86","author":"Y LeCun","year":"1998","unstructured":"LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278\u20132324","journal-title":"Proc IEEE"},{"issue":"1","key":"9538_CR33","first-page":"3","volume":"30","author":"AL Maas","year":"2013","unstructured":"Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In Proc Icml 30(1):3","journal-title":"In Proc Icml"},{"key":"9538_CR34","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2020.114048","volume":"166","author":"G Maguolo","year":"2021","unstructured":"Maguolo G, Nanni L, Ghidoni S (2021) Ensemble of convolutional neural networks trained with different activation functions. Expert Syst Appl 166:114048","journal-title":"Expert Syst Appl"},{"key":"9538_CR35","unstructured":"Nair V, Hinton GE (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) (pp. 807\u2013814)"},{"issue":"4","key":"9538_CR36","first-page":"1917","volume":"12","author":"I Pacal","year":"2022","unstructured":"Pacal I (2022) Deep learning approaches for classification of breast cancer in ultrasound (US) images. J Inst Sci Technol 12(4):1917\u20131927","journal-title":"J Inst Sci Technol"},{"key":"9538_CR37","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2021.104519","volume":"134","author":"I Pacal","year":"2021","unstructured":"Pacal I, Karaboga D (2021) A robust real-time deep learning based automatic polyp detection system. Comput Biol Med 134:104519","journal-title":"Comput Biol Med"},{"issue":"25","key":"9538_CR38","doi-asserted-by":"publisher","first-page":"18813","DOI":"10.1007\/s00521-023-08757-w","volume":"35","author":"I Pacal","year":"2023","unstructured":"Pacal I, K\u0131l\u0131carslan S (2023) Deep learning-based approaches for robust classification of cervical cancer. Neural Comput Appl 35(25):18813\u201318828","journal-title":"Neural Comput Appl"},{"key":"9538_CR39","doi-asserted-by":"publisher","first-page":"151359","DOI":"10.1109\/ACCESS.2019.2948112","volume":"7","author":"Z Qiumei","year":"2019","unstructured":"Qiumei Z, Dan T, Fenghua W (2019) Improved convolutional neural network based on fast exponentially linear unit activation function. IEEE Access 7:151359\u2013151367","journal-title":"IEEE Access"},{"key":"9538_CR40","unstructured":"Ramachandran P, Zoph B, Le QV (2017). Searching for activation functions. arXiv preprint arXiv:1710.05941"},{"key":"9538_CR41","doi-asserted-by":"crossref","unstructured":"Trottier L, Gigu P, Chaib-draa B (2017). Parametric exponential linear unit for deep convolutional neural networks. In: 16th IEEE international conference on machine learning and applications (ICMLA) (pp. 207\u2013214). IEEE","DOI":"10.1109\/ICMLA.2017.00038"},{"key":"9538_CR42","doi-asserted-by":"publisher","first-page":"1655","DOI":"10.1007\/s00217-023-04245-6","volume":"249","author":"EK Y\u0131lmaz","year":"2023","unstructured":"Y\u0131lmaz EK, Adem K, K\u0131l\u0131\u00e7arslan S, Ayd\u0131n HA (2023) Classification of lemon quality using hybrid model based on stacked autoencoder and convolutional neural network. Eur Food Res Technol 249:1655\u20131667","journal-title":"Eur Food Res Technol"},{"issue":"10","key":"9538_CR43","doi-asserted-by":"publisher","first-page":"7427","DOI":"10.1007\/s10489-021-02247-z","volume":"51","author":"Y Ying","year":"2021","unstructured":"Ying Y, Zhang N, Shan P, Miao L, Sun P, Peng S (2021) PSigmoid: improving squeeze-and-excitation block with parametric sigmoid. Appl Intell 51(10):7427\u20137439","journal-title":"Appl Intell"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09538-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-024-09538-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-09538-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T13:21:43Z","timestamp":1711027303000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-024-09538-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,28]]},"references-count":44,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2024,5]]}},"alternative-id":["9538"],"URL":"https:\/\/doi.org\/10.1007\/s00521-024-09538-9","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,28]]},"assertion":[{"value":"1 November 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 February 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}