{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T18:59:32Z","timestamp":1782154772949,"version":"3.54.5"},"reference-count":48,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000},"content-version":"vor","delay-in-days":53,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Electronics and Microelectronics Laboratory"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Computational Intelligence and Neuroscience"],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:p>Deep network in network (DNIN) model is an efficient instance and an important extension of the convolutional neural network (CNN) consisting of alternating convolutional layers and pooling layers. In this model, a multilayer perceptron (MLP), a nonlinear function, is exploited to replace the linear filter for convolution. Increasing the depth of DNIN can also help improve classification accuracy while its formation becomes more difficult, learning time gets slower, and accuracy becomes saturated and then degrades. This paper presents a new deep residual network in network (DrNIN) model that represents a deeper model of DNIN. This model represents an interesting architecture for on\u2010chip implementations on FPGAs. In fact, it can be applied to a variety of image recognition applications. This model has a homogeneous and multilength architecture with the hyperparameter \u201cL\u201d (\u201cL\u201d defines the model length). In this paper, we will apply the residual learning framework to DNIN and we will explicitly reformulate convolutional layers as residual learning functions to solve the vanishing gradient problem and facilitate and speed up the learning process. We will provide a comprehensive study showing that DrNIN models can gain accuracy from a significantly increased depth. On the CIFAR\u201010 dataset, we evaluate the proposed models with a depth of up to <jats:italic>L<\/jats:italic>\u2009=\u20095 DrMLPconv layers, 1.66x deeper than DNIN. The experimental results demonstrate the efficiency of the proposed method and its role in providing the model with a greater capacity to represent features and thus leading to better recognition performance.<\/jats:p>","DOI":"10.1155\/2021\/6659083","type":"journal-article","created":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T23:50:08Z","timestamp":1614124208000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":58,"title":["Deep Residual Network in Network"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2417-3972","authenticated-orcid":false,"given":"Hmidi","family":"Alaeddine","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2588-3642","authenticated-orcid":false,"given":"Malek","family":"Jihene","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2021,2,23]]},"reference":[{"key":"e_1_2_11_1_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-020-05008-0"},{"key":"e_1_2_11_2_2","unstructured":"IoffeS.andSzegedyC. Batch normalization: accelerating deep network training by reducing internal covariate shift 2015 http:\/\/arxiv.org\/abs\/1502.03167."},{"key":"e_1_2_11_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_39"},{"key":"e_1_2_11_4_2","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Bengio Y.","year":"2010","journal-title":"Proceedings of AISTATS"},{"key":"e_1_2_11_5_2","doi-asserted-by":"crossref","unstructured":"HeK. ZhangX. RenS. andSunJ. Delving deep into rectifiers: surpassing human-level performance on imagenet classification 2015 http:\/\/arxiv.org\/abs\/1502.01852.","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_2_11_6_2","unstructured":"SutskeverI. MartensJ. DahlG. E. andHintonG. E. DasguptaS.andMcallesterD. On the importance of initialization and momentum in deep learning 28 Proceedings of the 30th International Conference on Machine Learning (ICML-13) May 2013 New Brunswick NJ USA JMLR Workshop and Conference Proceedings 1139\u20131147."},{"key":"e_1_2_11_7_2","unstructured":"LeeC.-Y. XieS. GallagherP. ZhangZ. andTuZ. Deeply-supervised nets 2014 http:\/\/arxiv.org\/abs\/1409.5185."},{"key":"e_1_2_11_8_2","unstructured":"RaikoT. ValpolaH. andLecunY. LawrenceN. D.andGirolamiM. A. Deep learning made easier by linear transformations in perceptrons 22 Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics (AISTATS-12) April 2012 La Palma Canary Islands Spain 924\u2013932."},{"key":"e_1_2_11_9_2","unstructured":"ChenT. GoodfellowI. andShlensJ. Net2net: accelerating learning via knowledge transfer 2016 http:\/\/arxiv.org\/abs\/1511.05641."},{"key":"e_1_2_11_10_2","unstructured":"RomeroA. BallasN. KahouS. E. AntoineC. GattaC. andBengioY. FitNets: hints for thin deep nets 2014 http:\/\/arxiv.org\/abs\/1412.6550."},{"key":"e_1_2_11_11_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.2.234"},{"key":"e_1_2_11_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-49430-8_2"},{"key":"e_1_2_11_13_2","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot X.","year":"2010","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_11_14_2","unstructured":"HeK. ZhangX. RenS. andSunJ. Deep residual learning for image recognition 2015 http:\/\/arxiv.org\/abs\/1512.03385."},{"key":"e_1_2_11_15_2","doi-asserted-by":"crossref","unstructured":"AlomMd. Z. HasanM. YakopcicC. TahaT. andAsariV. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation 2018 https:\/\/arxiv.org\/abs\/1802.06955.","DOI":"10.1109\/NAECON.2018.8556686"},{"key":"e_1_2_11_16_2","unstructured":"ZeilerM. D.andFergusR. Visualizing and understanding convolutional networks 2013 http:\/\/arxiv.org\/abs\/1311.2901."},{"key":"e_1_2_11_17_2","unstructured":"SimonyanK.andZissermanA. Very deep convolutional networks for large-scale image recognition 2014 http:\/\/arxiv.org\/abs\/1409.1556."},{"key":"e_1_2_11_18_2","doi-asserted-by":"crossref","unstructured":"HuangG. LiuZ. WeinbergerK. Q. andVan Der MaatenL. Densely connected convolutional networks 2017 https:\/\/arxiv.org\/abs\/1608.06993.","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_2_11_19_2","unstructured":"IandolaF. HanS. MoskewiczM. AshrafK. DallyW. andKeutzerK. SqueezeNet: alexnet-level accuracy with 50x fewer parameters and <0.5MB model size 2016 https:\/\/arxiv.org\/abs\/1602.07360v4."},{"key":"e_1_2_11_20_2","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/1229\/1\/012034"},{"key":"e_1_2_11_21_2","unstructured":"LinM. ChenQ. andYanS. Network in network 2013 http:\/\/arxiv.org\/abs\/1312.4400."},{"key":"e_1_2_11_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2699184"},{"key":"e_1_2_11_23_2","doi-asserted-by":"crossref","unstructured":"ZagoruykoS.andKomodakisN. Wide residual networks 2016 http:\/\/arxiv.org\/abs\/1605.07146.","DOI":"10.5244\/C.30.87"},{"key":"e_1_2_11_24_2","doi-asserted-by":"crossref","unstructured":"SzegedyC. Going deeper with convolutions Proceedingsof the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2015 Boston MA USA 1\u20139 https:\/\/doi.org\/10.1109\/cvpr.2015.7298594 2-s2.0-84937522268.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_2_11_25_2","doi-asserted-by":"crossref","unstructured":"CiresanD. MeierU. andSchmidhuberJ. Multi-column deep neural networks for image classification 2012 http:\/\/arxiv.org\/abs\/1202.2745.","DOI":"10.1109\/CVPR.2012.6248110"},{"key":"e_1_2_11_26_2","unstructured":"GregorK.andLeCunY. Emergence of complex-like cells in a termporal product network with local receptive fields 2010 http:\/\/arxiv.org\/abs\/1006.0448."},{"key":"e_1_2_11_27_2","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky A.","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_11_28_2","doi-asserted-by":"crossref","unstructured":"JarrettK. KavukcuogluK. RanzatoM. A. andLeCunY. What is the best multi-stage architecture for object recognition? Proceedings of the IEEE International Conference on Computer Vision September 2009 Corfu Greece 2146\u20132153.","DOI":"10.1109\/ICCV.2009.5459469"},{"key":"e_1_2_11_29_2","doi-asserted-by":"crossref","unstructured":"LeCunY. KavukcuogluK. andFarabetC. Convolutional networks and applications in vision Proceedings of the International Symposium on Circuits and Systems (ISCAS 2010) June 2010 Paris France 253\u2013256.","DOI":"10.1109\/ISCAS.2010.5537907"},{"key":"e_1_2_11_30_2","unstructured":"ZeilerM. D.andFergusR. Stochastic pooling for regularization of deep convolutional neural networks 2013 http:\/\/arxiv.org\/abs\/1301.3557."},{"key":"e_1_2_11_31_2","doi-asserted-by":"crossref","unstructured":"HeK. ZhangX. RenS. andSunJ. Spatial pyramid pooling in deep convolutional networks for visual recognition 2014 http:\/\/arxiv.org\/abs\/1406.4729.","DOI":"10.1007\/978-3-319-10578-9_23"},{"key":"e_1_2_11_32_2","unstructured":"ChanT. JiaK. GaoS. LuJ. ZengZ. andMaY. PCANet: a simple deep learning baseline for image classification? 2014 http:\/\/arxiv.org\/abs\/1404.3606."},{"key":"e_1_2_11_33_2","unstructured":"LeeC. GallagherP. andTuZ. Generalizing pooling functions in convolutional neural networks: mixed gated and tree 2015 https:\/\/arxiv.org\/abs\/1509.08985."},{"key":"e_1_2_11_34_2","unstructured":"SpringenbergJ. DosovitskiyA. BroxT. T. andRiedmillerM. Striving for simplicity: the all convolutional net 2014 http:\/\/arxiv.org\/abs\/1412.6806."},{"key":"e_1_2_11_35_2","doi-asserted-by":"crossref","unstructured":"GongY. WangL. GuoR. andLazebnikS. Multi-scale orderless pooling of deep convolutional activation features 2014 http:\/\/arxiv.org\/abs\/1403.1840.","DOI":"10.1007\/978-3-319-10584-0_26"},{"key":"e_1_2_11_36_2","doi-asserted-by":"crossref","unstructured":"YooD. ParkS. LeeJ. andKweonI. Multi-scale pyramid pooling for deep convolutional representation Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) September 2015 Boston MA USA 1\u20135.","DOI":"10.1109\/CVPRW.2015.7301274"},{"key":"e_1_2_11_37_2","unstructured":"GrahamB. Fractional max-pooling 2014 https:\/\/arxiv.org\/abs\/1412.6071."},{"key":"e_1_2_11_38_2","unstructured":"MurrayN.andPerronninF. Generalized max pooling Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) September 2015 Boston MA USA 2473\u20132480."},{"key":"e_1_2_11_39_2","unstructured":"GoodfellowI. J. Warde-FarleyD. MirzaM. CourvilleA. C. andBengioY. Maxout networks 2013 https:\/\/arxiv.org\/abs\/1302.4389."},{"key":"e_1_2_11_40_2","doi-asserted-by":"crossref","unstructured":"LiaoZ.andCarneiroG. On the importance of normalisation layers in deep learning with piecewise linear activation units 2016 https:\/\/arxiv.org\/abs\/1508.00330.","DOI":"10.1109\/WACV.2016.7477624"},{"key":"e_1_2_11_41_2","unstructured":"ClevertD. Fast and accurate deep network learning by exponential linear units (ELUs) 2015 https:\/\/arxiv.org\/abs\/1511.07289."},{"key":"e_1_2_11_42_2","unstructured":"ChangJ.-R.andChenY.-S. Batch-normalized maxout network in network 2015 http:\/\/arxiv.org\/abs\/1511.02583."},{"key":"e_1_2_11_43_2","unstructured":"GoodfellowI. J. ShlensJ. andSzegedyC. Explaining and harnessing adversarial examples 2014 https:\/\/arxiv.org\/abs\/1412.6572."},{"key":"e_1_2_11_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.09.001"},{"key":"e_1_2_11_45_2","doi-asserted-by":"crossref","unstructured":"LiuY. ZhangJ. GaoC. QuJ. andJiL. NaturalLogarithm-rectified activation function in convolutional neural networks 2019 https:\/\/arxiv.org\/abs\/1908.03682.","DOI":"10.1109\/ICCC47050.2019.9064398"},{"key":"e_1_2_11_46_2","first-page":"1929","article-title":"A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava N.","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_11_47_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-019-0197-0"},{"key":"e_1_2_11_48_2","unstructured":"XieS. GirshickR. Doll\u00e1rP. TuZ. andHeK. Aggregated residual transformations for deep neural networks 2016 https:\/\/arxiv.org\/abs\/1611.05431."}],"container-title":["Computational Intelligence and Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2021\/6659083.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2021\/6659083.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2021\/6659083","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T11:11:35Z","timestamp":1722942695000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2021\/6659083"}},"subtitle":[],"editor":[{"given":"Mario","family":"Versaci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1]]}},"alternative-id":["10.1155\/2021\/6659083"],"URL":"https:\/\/doi.org\/10.1155\/2021\/6659083","archive":["Portico"],"relation":{},"ISSN":["1687-5265","1687-5273"],"issn-type":[{"value":"1687-5265","type":"print"},{"value":"1687-5273","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1]]},"assertion":[{"value":"2020-12-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-01-26","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-02-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"6659083"}}