{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:55:02Z","timestamp":1773932102785,"version":"3.50.1"},"reference-count":71,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T00:00:00Z","timestamp":1703030400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010665","name":"H2020 Marie Sk\u0142odowska-Curie Actions","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Developing efficient methods to infer relations among different faces consisting of numerous expressions or on the same face at different times (e.g., disease progression) is an open issue in imaging related research. In this study, we present a novel method for facial feature extraction, characterization, and identification based on classical computer vision coupled with deep learning and, more specifically, convolutional neural networks.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We describe the hybrid face characterization system named FRetrAIval (FRAI), which is a hybrid of the GoogleNet and the AlexNet Neural Network (NN) models. Images analyzed by the FRAI network are preprocessed by computer vision techniques such as the oriented gradient-based algorithm that can extract only the face region from any kind of picture. The Aligned Face dataset (AFD) was used to train and test the FRAI solution for extracting image features. The Labeled Faces in the Wild (LFW) holdout dataset has been used for external validation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results and discussion<\/jats:title><jats:p>Overall, in comparison to previous techniques, our methodology has shown much better results on k-Nearest Neighbors (KNN) by yielding the maximum precision, recall, F1, and F2 score values (92.00, 92.66, 92.33, and 92.52%, respectively) for AFD and (95.00% for each variable) for LFW dataset, which were used as training and testing datasets. The FRAI model may be potentially used in healthcare and criminology as well as many other applications where it is important to quickly identify face features such as fingerprint for a specific identification target.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2023.1230383","type":"journal-article","created":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T08:13:58Z","timestamp":1703060038000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Automated facial characterization and image retrieval by convolutional neural networks"],"prefix":"10.3389","volume":"6","author":[{"given":"Syed Taimoor Hussain","family":"Shah","sequence":"first","affiliation":[]},{"given":"Syed Adil Hussain","family":"Shah","sequence":"additional","affiliation":[]},{"given":"Shahzad Ahmad","family":"Qureshi","sequence":"additional","affiliation":[]},{"given":"Angelo","family":"Di Terlizzi","sequence":"additional","affiliation":[]},{"given":"Marco Agostino","family":"Deriu","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,12,20]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1007\/978-981-19-2358-6_61","article-title":"Drowsiness detection using iot and facial expression","author":"Ashlin Deepa","year":"2023","journal-title":"Cogn. Sci. Technol."},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3992988","article-title":"Segmentation of Text Documents Using Hyperspectral Imaging: A Blend of Deep Cnn and Generative Adversarial Network","author":"Asmat","year":"2021","journal-title":"SSRN Electronic Journal"},{"key":"B3","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1109\/34.598228","article-title":"Eigenfaces vs. fisherfaces: recognition using class specific linear projection","volume":"19","author":"Belhumeur","year":"1997","journal-title":"IEEE Transact. Pattern Anal. Mach. Intell"},{"key":"B4","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1016\/j.aci.2019.05.003","article-title":"Neonatal pain detection in videos using the ICOPEvid dataset and an ensemble of descriptors extracted from gaussian of local descriptors","volume":"19","author":"Brahnam","year":"2023","journal-title":"Appl. Comp. Informat."},{"key":"B5","doi-asserted-by":"publisher","first-page":"1042","DOI":"10.1109\/34.254061","article-title":"Face recognition: features versus templates","volume":"15","author":"Brunelli","year":"1993","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B6","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1109\/TSE.1980.230801","article-title":"Query-by-pictorial-example","volume":"6","author":"Chang","year":"","journal-title":"IEEE Transact. Softw. Eng."},{"key":"B7","doi-asserted-by":"publisher","first-page":"884","DOI":"10.1109\/5.664278","article-title":"Next-generation content representation, creation, and searching for new-media applications in education","volume":"86","author":"Chang","year":"1998","journal-title":"Proc. IEEE"},{"key":"B8","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1142\/9789814343138_0035","article-title":"Image information systems: where do we go from here?","volume":"4","author":"Chang","year":"1992","journal-title":"IEEE Transact. Knowl. Data Eng"},{"key":"B9","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1109\/C-M.1981.220245","article-title":"Pictorial data-base systems","volume":"14","author":"Chang","year":"1981","journal-title":"Computer"},{"key":"B10","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1109\/32.6147","article-title":"An intelligent image database system","volume":"14","author":"Chang","year":"1988","journal-title":"IEEE Transact. Softw. Eng"},{"key":"B11","doi-asserted-by":"crossref","DOI":"10.1007\/3-540-09757-0_11","volume-title":"A Relational Database System for Images","author":"Chang","year":""},{"key":"B12","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1109\/5.381842","article-title":"Human and machine recognition of faces: a survey","volume":"83","author":"Chellappa","year":"1995","journal-title":"Proc. IEEE"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1007","DOI":"10.1016\/0031-3203(95)00139-5","article-title":"Face-texture model based on SGLD and its application in face detection in a color scene","volume":"29","author":"Dai","year":"1996","journal-title":"Pattern Recognit"},{"key":"B14","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1109\/CVPR.2005.177","article-title":"\u201cHistograms of oriented gradients for human detection,\u201d","author":"Dalal","year":"2005","journal-title":"Proceedings - 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654960","article-title":"Learning deep face representation","author":"Fan","year":"2014","journal-title":"arXiv [Preprint]."},{"key":"B16","doi-asserted-by":"crossref","first-page":"7748","DOI":"10.3390\/app10217748","article-title":"Recommendation systems: Algorithms, challenges, metrics, and business opportunities","volume":"10","author":"Fayyaz","year":"2020","journal-title":"Appl. Sci."},{"key":"B17","doi-asserted-by":"publisher","first-page":"1033","DOI":"10.1016\/S0031-3203(00)00042-X","article-title":"Multi-cues eye detection on gray intensity image","volume":"34","author":"Feng","year":"2001","journal-title":"Pattern Recognit"},{"key":"B18","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1109\/2.410146","article-title":"Query by image and video content: the QBIC system","volume":"28","author":"Flickner","year":"1995","journal-title":"Computer"},{"key":"B19","article-title":"Labeled faces in the wild: a database for studying face recognition in unconstrained environments","author":"Gary","year":"2008","journal-title":"Tech. Rep."},{"key":"B20","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1109\/34.541411","article-title":"Face recognition system using local autocorrelations and multiscale integration","volume":"18","author":"Goudail","year":"1996","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B21","first-page":"986","volume-title":"KNN Model-Based Approach in Classification","author":"Guo","year":"2003"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1701","DOI":"10.1016\/S0031-3203(99)00141-7","article-title":"Fast face detection via morphology-based pre-processing","volume":"33","author":"Han","year":"2000","journal-title":"Pattern Recognit."},{"key":"B23","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1007\/978-981-13-8581-0_26","article-title":"Face image retrieval using discriminative ternary census transform and spatial pyramid matching","volume":"1031","author":"Hasnat","year":"2019","journal-title":"Commun. Comp. Inf. Sci."},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.11141\/ia.52.5","article-title":"What the machine saw: some questions on the ethics of computer vision and machine learning to investigate human remains trafficking","volume":"52","author":"Huffer","year":"2019","journal-title":"Int. Archaeol"},{"key":"B25","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1016\/S0031-3203(97)00048-4","article-title":"Facial feature detection using geometrical face model: an efficient approach","volume":"31","author":"Jeng","year":"1998","journal-title":"Pattern Recognit"},{"key":"B26","doi-asserted-by":"crossref","DOI":"10.1007\/3-540-45344-X_14","article-title":"\u201cRobust face detection using the hausdorff distance BT,\u201d","volume-title":"Proc. of Conf. on Audio- and Video-Based Biometric Person Authentication","author":"Jesorsky","year":"2001"},{"key":"B27","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1504\/IJCVR.2023.127298","article-title":"Facial expression recognition based on convolutional block attention module and multi-feature fusion","volume":"13","author":"Jiang","year":"2022","journal-title":"Int. J. Comp. Vis. Robot."},{"key":"B28","doi-asserted-by":"publisher","first-page":"2351","DOI":"10.1007\/s10462-021-10066-4","article-title":"A comprehensive survey of recent trends in deep learning for digital images augmentation","volume":"55","author":"Khalifa","year":"2021","journal-title":"Artif. Intell. Rev."},{"key":"B29","article-title":"\u201cAdam: a method for stochastic optimization,\u201d","author":"Kingma","year":"2015","journal-title":"3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings"},{"key":"B30","unstructured":"KrizhevskyA.\n            SutskeverI.\n            HintonG. E\n          ImageNet Classification with Deep Convolutional Neural Networks"},{"key":"B31","doi-asserted-by":"crossref","DOI":"10.1109\/CITSM.2016.7577578","article-title":"\u201cCosine similarity to determine similarity measure: study case in online essay assessment,\u201d","volume-title":"Proceedings of 2016 4th International Conference on Cyber and IT Service Management, CITSM 2016","author":"Lahitani","year":"2016"},{"key":"B32","author":"Li","year":"2015","journal-title":"A Convolutional Neural Network Cascade for Face Detection"},{"key":"B33","doi-asserted-by":"publisher","first-page":"903","DOI":"10.1016\/0031-3203(93)90056-3","article-title":"Algebraic feature extraction for image recognition based on an optimal discriminant criterion","volume":"26","author":"Liu","year":"1993","journal-title":"Pattern Recognit"},{"key":"B34","doi-asserted-by":"publisher","first-page":"4766","DOI":"10.48550\/arXiv.1705.07874","article-title":"A unified approach to interpreting model predictions","author":"Lundberg","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.3390\/electronics11020202","article-title":"Citation: Qasim PCA-based advanced local octa-directional pattern (ALODP-PCA): A texture feature PCA-based advanced local octa-directional pattern (ALODP-PCA): A texture feature descriptor for image retrieval","author":"Mahmood","year":"2022","journal-title":"Electronics"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1525","DOI":"10.1016\/S0031-3203(99)00130-2","article-title":"Real-time face location on gray-scale static images","volume":"33","author":"Maio","year":"2000","journal-title":"Pattern Recognit"},{"key":"B37","first-page":"503","article-title":"\u201cAnalysis of euclidean distance and manhattan distance measure in face recognition,\u201d","volume-title":"IET Conference Publications","author":"Malkauthekar","year":"2013"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1117\/12.2655893","article-title":"\u201cFacial expression recognition based on deep convolutional neural network,\u201d","author":"Mao","year":"2023","journal-title":"Proc. SPIE 12509, Third International Conference on Intelligent Computing and Human-Computer Interaction (ICHCI 2022)"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42452-020-2234-1","article-title":"Facial emotion recognition using convolutional neural networks (FERC)","volume":"2","author":"Mehendale","year":"2020","journal-title":"SN Appl. Sci."},{"key":"B40","volume-title":"An Ontology Approach to Object-Based Image Retrieval.","author":"Mezaris","year":"2004"},{"key":"B41","doi-asserted-by":"publisher","first-page":"696","DOI":"10.1109\/34.598227","article-title":"Probabilistic visual learning for object representation","volume":"19","author":"Moghaddam","year":"1997","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B42","first-page":"665","article-title":"\u201cUsing KNN algorithm for classification of textual documents,\u201d","volume-title":"ICIT 2017 - 8th International Conference on Information Technology, Proceedings","author":"Moldagulova","year":"2017"},{"key":"B43","article-title":"\u201cHidden markov models for face recognition,\u201d","author":"Nefian","year":"1998","journal-title":"ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings"},{"key":"B44","doi-asserted-by":"publisher","first-page":"103743","DOI":"10.1016\/j.jvcir.2022.103743","article-title":"Classification networks for continuous automatic pain intensity monitoring in video using facial expression on the X-ITE pain database","volume":"91","author":"Othman","year":"2023","journal-title":"J. Vis. Commun. Image Represent."},{"key":"B45","doi-asserted-by":"publisher","first-page":"100016","DOI":"10.1016\/j.neuri.2021.100016","article-title":"Digitization of handwritten devanagari text using CNN transfer learning \u2013 a better customer service support","volume":"2","author":"Pande","year":"2022","journal-title":"Neurosci. Informat."},{"key":"B46","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1504\/IJCAT.2013.056023","article-title":"Content-based image retrieval using colour and shape features","volume":"48","author":"Park","year":"2013","journal-title":"Int. J. Comp. Appl. Technol."},{"key":"B47","first-page":"96","article-title":"Histogram refinement","author":"Pass","year":"1996","journal-title":"Science"},{"key":"B48","first-page":"338","volume-title":"Impact of Feature Extraction Techniques on a CBIR System","author":"Raghuwanshi","year":"2019"},{"key":"B49","first-page":"97","article-title":"\u201c\u2018Why should i trust you?': explaining the predictions of any classifier,\u201d","author":"Ribeiro","year":"2016","journal-title":"NAACL-HLT 2016 - 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Demonstrations Session"},{"key":"B50","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/0031-3203(92)90007-6","article-title":"Automatic recognition and analysis of human faces and facial expressions: a survey","volume":"25","author":"Samal","year":"1992","journal-title":"Pattern Recognit"},{"key":"B51","doi-asserted-by":"publisher","first-page":"68","DOI":"10.3169\/mta.7.68","article-title":"Interactive face retrieval framework for clarifying user's visual memory","volume":"7","author":"Sato","year":"2019","journal-title":"ITE Transact. Media Technol. Appl."},{"key":"B52","first-page":"519","volume-title":"Enhanced Bag-of-Features Method Using Grey Wolf Optimization for Automated Face Retrieval","author":"Shukla","year":"2019"},{"key":"B53","article-title":"\u201cVery deep convolutional networks for large-scale image recognition,\u201d","author":"Simonyan","year":"2015","journal-title":"3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations"},{"key":"B54","doi-asserted-by":"crossref","DOI":"10.1109\/CASP.2016.7746183","article-title":"\u201cFace detection and eyes extraction using sobel edge detection and morphological operations,\u201d","volume-title":"Conference on Advances in Signal Processing, CASP 2016","author":"Singh","year":"2016"},{"key":"B55","article-title":"\u201cEye-tracking based relevance feedback for iterative face image retrieval,\u201d","author":"Sun","year":"2019","journal-title":"120. SPIE-Intl Soc Optical Eng"},{"key":"B56","doi-asserted-by":"publisher","first-page":"831","DOI":"10.1109\/34.531802","article-title":"Using discriminant eigenfeatures for image retrieval","volume":"18","author":"Swets","year":"1996","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.4842","article-title":"Going deeper with convolutions","author":"Szegedy","year":"2014","journal-title":"arXiv"},{"key":"B58","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/0031-3203(84)90033-5","article-title":"Image database systems: a survey","volume":"17","author":"Tamura","year":"1984","journal-title":"Pattern Recognit"},{"key":"B59","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1109\/IACS.2019.8809127","article-title":"\u201cDeep face image retrieval: a comparative study with dictionary learning,\u201d","volume-title":"2019 10th International Conference on Information and Communication Systems (ICICS)","author":"Tarawneh","year":"2019"},{"key":"B60","first-page":"1","article-title":"\u201cContent-based image retrieval: an introduction,\u201d","volume-title":"Content-Based Image Retrieval","author":"Tyagi","year":"2017"},{"key":"B61","doi-asserted-by":"publisher","first-page":"1209","DOI":"10.1016\/0031-3203(94)90006-X","article-title":"Connectionist models of face processing: a survey","volume":"27","author":"Valentin","year":"1994","journal-title":"Pattern Recognit"},{"key":"B62","doi-asserted-by":"publisher","first-page":"3067","DOI":"10.1109\/TPAMI.2017.2787130","article-title":"Facial landmark detection with tweaked convolutional neural networks","volume":"40","author":"Wu","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B63","doi-asserted-by":"crossref","first-page":"3469","DOI":"10.1109\/CVPR.2010.5539976","article-title":"\u201cScalable face image retrieval with identity-based quantization and multi-reference re-ranking,\u201d","volume-title":"2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition","author":"Wu","year":"2010"},{"key":"B64","article-title":"\u201cImage content annotation using bayesian framework and complement components analysis,\u201d","volume-title":"Proceedings - International Conference on Image Processing, ICIP","author":"Yang","year":"2005"},{"key":"B65","unstructured":"YangM.\n            KpalmaK.\n            RonsinJ.\n            RonsinJ. A.\n            MingqiangY.\n            KidiyoK.\n          Survey of Shape Feature Extraction Techniques. Peng-Yeng Yin2008"},{"key":"B66","doi-asserted-by":"publisher","first-page":"1169","DOI":"10.3390\/ijerph20021169","article-title":"Differentiating use of facial expression between individuals with and without traumatic brain injury using affectiva software: a pilot study","volume":"20","author":"Yiew","year":"2023","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"B67","doi-asserted-by":"crossref","DOI":"10.1109\/DICTA.2009.48","article-title":"\u201cSemantic image retrieval using region based inverted file,\u201d","volume-title":"DICTA 2009 - Digital Image Computing: Techniques and Applications","author":"Zhang","year":"2009"},{"key":"B68","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.patcog.2003.07.008","article-title":"Review of shape representation and description techniques","volume":"37","author":"Zhang","year":"2004","journal-title":"Pattern Recognit"},{"key":"B69","doi-asserted-by":"crossref","first-page":"1423","DOI":"10.1109\/5.628712","article-title":"Face recognition: eigenface, elastic matching, and neural nets","volume":"85","author":"Zhang","year":"1997","journal-title":"Proc. IEEE"},{"key":"B70","doi-asserted-by":"publisher","first-page":"107976","DOI":"10.1016\/j.patcog.2021.107976","article-title":"Deep center-based dual-constrained hashing for discriminative face image retrieval","volume":"117","author":"Zhang","year":"2021","journal-title":"Pattern Recognit."},{"key":"B71","doi-asserted-by":"publisher","first-page":"105","DOI":"10.17705\/1CAIS.03105","article-title":"The evolution of social commerce: an examination from the people, business, technology, and information perspective","volume":"31","author":"Zhang","year":"2012","journal-title":"CAIS"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1230383\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T08:14:13Z","timestamp":1703060053000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1230383\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,20]]},"references-count":71,"alternative-id":["10.3389\/frai.2023.1230383"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1230383","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,20]]},"article-number":"1230383"}}