{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T03:59:22Z","timestamp":1781323162889,"version":"3.54.1"},"reference-count":57,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,27]],"date-time":"2021-12-27T00:00:00Z","timestamp":1640563200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Convolutional neural networks (CNNs) have powerful representation learning capabilities by automatically learning and extracting features directly from inputs. In classification applications, CNN models are typically composed of: convolutional layers, pooling layers, and fully connected (FC) layer(s). In a chain-based deep neural network, the FC layers contain most of the parameters of the network, which affects memory occupancy and computational complexity. For many real-world problems, speeding up inference time is an important matter because of the hardware design implications. To deal with this problem, we propose the replacement of the FC layers with a Hopfield neural network (HNN). The proposed architecture combines both a CNN and an HNN: A pretrained CNN model is used for feature extraction, followed by an HNN, which is considered as an associative memory that saves all features created by the CNN. Then, to deal with the limitation of the storage capacity of the HNN, the proposed work uses multiple HNNs. To optimize this step, the knapsack problem formulation is proposed, and a genetic algorithm (GA) is used solve it. According to the results obtained on the Noisy MNIST Dataset, our work outperformed the state-of-the-art algorithms.<\/jats:p>","DOI":"10.3390\/a15010011","type":"journal-article","created":{"date-parts":[[2021,12,27]],"date-time":"2021-12-27T01:00:54Z","timestamp":1640566854000},"page":"11","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Optimal CNN\u2013Hopfield Network for Pattern Recognition Based on a Genetic Algorithm"],"prefix":"10.3390","volume":"15","author":[{"given":"Fekhr Eddine","family":"Keddous","sequence":"first","affiliation":[{"name":"Laboratoire LISSI, University Paris Est Creteil, 94000 Creteil, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9620-9324","authenticated-orcid":false,"given":"Amir","family":"Nakib","sequence":"additional","affiliation":[{"name":"Laboratoire LISSI, University Paris Est Creteil, 94000 Creteil, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,27]]},"reference":[{"key":"ref_1","unstructured":"Simonyan, K., and Andrew, Z. (2015). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sultana, F., Abu, S., and Paramartha, D. (2018, January 22\u201323). Advancements in image classification using convolutional neural network. Proceedings of the 2018 IEEE Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN, Kolkata, India.","DOI":"10.1109\/ICRCICN.2018.8718718"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3840","DOI":"10.1109\/TCYB.2020.2983860","article-title":"Automatically designing CNN architectures using the genetic algorithm for image classification","volume":"50","author":"Sun","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"6232","DOI":"10.1109\/TGRS.2016.2584107","article-title":"Deep feature extraction and classification of hyperspectral images based on convolutional neural networks","volume":"54","author":"Chen","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.tifs.2021.04.042","article-title":"Efficient extraction of deep image features using convolutional neural network (CNN) for applications in detecting and analysing complex food matrices","volume":"113","author":"Liu","year":"2021","journal-title":"Trends Food Sci. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1007\/s11280-018-0556-3","article-title":"Multi-scale deep context convolutional neural networks for semantic segmentation","volume":"22","author":"Zhou","year":"2019","journal-title":"World Wide Web"},{"key":"ref_7","unstructured":"Bakas, S. (2018). Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv."},{"key":"ref_8","unstructured":"Heller, N. (2019). The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes. arXiv."},{"key":"ref_9","unstructured":"Simpson, A.L. (2019). A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.neucom.2017.05.025","article-title":"Medical image retrieval using deep convolutional neural network","volume":"266","author":"Qayyum","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_11","first-page":"1655","article-title":"Fine-tuning CNN image retrieval with no human annotation","volume":"41","author":"Giorgos","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.neucom.2016.12.002","article-title":"Exploiting the complementary strengths of multi-layer CNN features for image retrieval","volume":"237","author":"Yu","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/s13748-019-00203-0","article-title":"Convolutional neural network: A review of models, methodologies and applications to object detection","volume":"9","author":"Dhillon","year":"2020","journal-title":"Prog. Artif. Intell."},{"key":"ref_14","first-page":"1","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Du, J. (2018). Understanding of object detection based on CNN family and YOLO. Journal of Physics: Conference Series, Volume 1004, 2nd International Conference on Machine Vision and Information Technology (CMVIT 2018), Hong Kong, China, 23\u201325 February 2018, IOP Publishing.","DOI":"10.1088\/1742-6596\/1004\/1\/012029"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1016\/j.neucom.2019.10.008","article-title":"Impact of fully connected layers on performance of convolutional neural networks for image classification","volume":"378","author":"Basha","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.neucom.2018.03.080","article-title":"Overfitting remedy by sparsifying regularization on fully-connected layers of CNNs","volume":"328","author":"Xu","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_18","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Liu, Q., and Supratik, M. (2018, January 8\u201313). Unsupervised learning using pretrained CNN and associative memory bank. Proceedings of the 2018 IEEE International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489408"},{"key":"ref_21","unstructured":"Krotov, D., and John, J.H. (2016). Dense associative memory for pattern recognition. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1007\/s10955-017-1806-y","article-title":"On a model of associative memory with huge storage capacity","volume":"168","author":"Demircigil","year":"2017","journal-title":"J. Stat. Phys."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Widrich, M. (2020). Modern hopfield networks and attention for immune repertoire classification. arXiv.","DOI":"10.1101\/2020.04.12.038158"},{"key":"ref_24","unstructured":"Ramsauer, H. (2020). Hopfield networks is all you need. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/BF00339943","article-title":"\u201cNeural\u201d computation of decisions in optimization problems","volume":"52","author":"Hopfield","year":"1985","journal-title":"Biol. Cybern."},{"key":"ref_26","first-page":"1216","article-title":"On the storage capacity of Hopfield models with correlated patterns","volume":"8","year":"1998","journal-title":"Ann. Appl. Probab."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1109\/18.746829","article-title":"On the storage capacity of the Hopfield model with biased patterns","volume":"45","author":"Lowe","year":"1999","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1109\/72.728382","article-title":"Optimal Hopfield network for combinatorial optimization with linear cost function","volume":"9","author":"Matsuda","year":"1998","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1016\/j.ejor.2008.11.002","article-title":"A review of Hopfield neural networks for solving mathematical programming problems","volume":"198","author":"Wen","year":"2009","journal-title":"Eur. J. Oper. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1088\/1757-899X\/862\/5\/052048","article-title":"Classification of handwritten digits using the Hopfield network","volume":"Volume 862","author":"Belyaev","year":"2020","journal-title":"IOP Conference Series: Materials Science and Engineering"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1109\/TNNLS.2015.2496658","article-title":"A generalized Hopfield network for nonsmooth constrained convex optimization: Lie derivative approach","volume":"27","author":"Li","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1209\/0295-5075\/2\/4\/012","article-title":"Saturation level of the Hopfield model for neural network","volume":"2","author":"Crisanti","year":"1986","journal-title":"EPL (Europhys. Lett.)"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1063\/1.2810360","article-title":"Introduction to the theory of neural computation","volume":"44","author":"Hertz","year":"1991","journal-title":"Phys. Today"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1002","DOI":"10.1109\/LGRS.2016.2560222","article-title":"Hopfield neural network approach for supervised nonlinear spectral unmixing","volume":"13","author":"Li","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Song, Y. (2017, January 26\u201328). System parameter identification experiment based on Hopfield neural network for self balancing vehicle. Proceedings of the 36th IEEE Chinese Control Conference (CCC), Dalian, China.","DOI":"10.23919\/ChiCC.2017.8028442"},{"key":"ref_36","first-page":"670","article-title":"A Novel Blind Detection Algorithm Based on Adjustable Parameters Activation Function Hopfield Neural Network","volume":"8","author":"Chen","year":"2017","journal-title":"J. Inf. Hiding Multim. Signal Process."},{"key":"ref_37","first-page":"523","article-title":"Blind Signal Detection Using Complex Transiently Chaotic Hopfield Neural Network","volume":"9","author":"Zhang","year":"2018","journal-title":"J. Inf. Hiding Multim. Signal Process."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1109\/TIT.1987.1057328","article-title":"The capacity of the Hopfield associative memory","volume":"33","author":"McEliece","year":"1987","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"061910","DOI":"10.1103\/PhysRevE.66.061910","article-title":"Storage capacity of attractor neural networks with depressing synapses","volume":"66","author":"Torres","year":"2002","journal-title":"Phys. Rev. E"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1109\/TIT.1985.1057069","article-title":"Information capacity of the Hopfield model","volume":"31","year":"1985","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3151","DOI":"10.1162\/neco_a_01143","article-title":"Dense associative memory is robust to adversarial inputs","volume":"30","author":"Krotov","year":"2018","journal-title":"Neural Comput."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/0377-2217(87)90165-2","article-title":"Exact methods for the knapsack problem and its generalizations","volume":"28","year":"1987","journal-title":"Eur. J. Oper. Res."},{"key":"ref_43","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Geoffrey","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1037\/h0071325","article-title":"Analysis of a complex of statistical variables into principal components","volume":"24","author":"Hotelling","year":"1933","journal-title":"J. Educ. Psychol."},{"key":"ref_45","first-page":"9","article-title":"Clustering techniques and the similarity measures used in clustering: A survey","volume":"134","author":"Irani","year":"2016","journal-title":"Int. J. Comput. Appl."},{"key":"ref_46","unstructured":"(2009). E-G Talbi, Metaheuristics: From Design to Implementation, John Wiley & Sons."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Singh, R.P. (2011, January 27\u201329). Solving 0\u20131 knapsack problem using genetic algorithms. Proceedings of the 2011 IEEE 3rd International Conference on Communication Software and Networks, Xi\u2019an, China.","DOI":"10.1109\/ICCSN.2011.6013975"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1023\/A:1021251113462","article-title":"Simple explanation of the no-free-lunch theorem and its implications","volume":"115","author":"Ho","year":"2002","journal-title":"J. Optim. Theory Appl."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1023\/A:1009642405419","article-title":"A genetic algorithm for the multidimensional knapsack problem","volume":"4","author":"Chu","year":"1998","journal-title":"J. Heuristics"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Sara\u00e7, T., Aydin, S., and Aydin, S. (2007). A genetic algorithm for the quadratic multiple knapsack problem. International Symposium on Brain, Vision, and Artificial Intelligence, Springer.","DOI":"10.1007\/978-3-540-75555-5_47"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Khuri, S., Thomas, B., and J\u00f6rg, H. (1994, January 6\u20138). The zero\/one multiple knapsack problem and genetic algorithms. Proceedings of the 1994 ACM Symposium on Applied Computing, Phoenix, AZ, USA.","DOI":"10.1145\/326619.326694"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Keddous, F., Nguyen, H.-N., and Nakib, A. (2021, January 17\u201321). Characters Recognition based on CNN-RNN architecture and Metaheuristic. Proceedings of the 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), Portland, OR, USA.","DOI":"10.1109\/IPDPSW52791.2021.00082"},{"key":"ref_53","unstructured":"Amit, D.J., and Daniel, J.A. (1992). Modeling Brain Function: The World of Attractor Neural Networks, Cambridge University Press."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1023\/A:1020499411651","article-title":"Generalized hamming distance","volume":"5","author":"Bookstein","year":"2002","journal-title":"Inf. Retr."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Karki, M. (2018, January 5\u20138). Pixel-level reconstruction and classification for noisy handwritten bangla characters. Proceedings of the 16th IEEE International Conference on Frontiers in Handwriting Recognition (ICFHR), Niagara Falls, NY, USA.","DOI":"10.1109\/ICFHR-2018.2018.00095"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"89497","DOI":"10.1109\/ACCESS.2020.2990567","article-title":"Pymoo: Multi-objective optimization in python","volume":"8","author":"Blank","year":"2020","journal-title":"IEEE Access"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Liu, Q., Edward, C., and Supratik, M. (2019). Pcgan-char: Progressively trained classifier generative adversarial networks for classification of noisy handwritten bangla characters. Digital Libraries at the Crossroads of Digital Information for the Future\u201421st International Conference on Asia-Pacific Digital Libraries, ICADL 2019, KualaLumpur, Malaysia, 4\u20137 November 2019, Springer.","DOI":"10.1007\/978-3-030-34058-2_1"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/1\/11\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:53:50Z","timestamp":1760169230000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/15\/1\/11"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,27]]},"references-count":57,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["a15010011"],"URL":"https:\/\/doi.org\/10.3390\/a15010011","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,27]]}}}