{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T09:51:44Z","timestamp":1781949104051,"version":"3.54.5"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T00:00:00Z","timestamp":1718582400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Computerized sentiment detection, based on artificial intelligence and computer vision, has become essential in recent years. Thanks to developments in deep neural networks, this technology can now account for environmental, social, and cultural factors, as well as facial expressions. We aim to create more empathetic systems for various purposes, from medicine to interpreting emotional interactions on social media.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>To develop this technology, we combined authentic images from various databases, including EMOTIC (ADE20K, MSCOCO), EMODB_SMALL, and FRAMESDB, to train our models. We developed two sophisticated algorithms based on deep learning techniques, DCNN and VGG19. By optimizing the hyperparameters of our models, we analyze context and body language to improve our understanding of human emotions in images. We merge the 26 discrete emotional categories with the three continuous emotional dimensions to identify emotions in context. The proposed pipeline is completed by fusing our models.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We adjusted the parameters to outperform previous methods in capturing various emotions in different contexts. Our study showed that the Sentiment_recognition_model and VGG19_contexte increased mAP by 42.81% and 44.12%, respectively, surpassing the results of previous studies.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>This groundbreaking research could significantly improve contextual emotion recognition in images. The implications of these promising results are far-reaching, extending to diverse fields such as social robotics, affective computing, human-machine interaction, and human-robot communication.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2024.1386753","type":"journal-article","created":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T13:54:31Z","timestamp":1718632471000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Contextual emotion detection in images using deep learning"],"prefix":"10.3389","volume":"7","author":[{"given":"Fatiha","family":"Limami","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Boutaina","family":"Hdioud","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachid","family":"Oulad Haj Thami","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,6,17]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"11761","DOI":"10.1109\/ACCESS.2019.2963113","article-title":"Emotion recognition from body movement","volume":"8","author":"Ahmed","year":"2020","journal-title":"IEEE Access"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"1163","DOI":"10.3390\/app13021163","article-title":"Prediction of emotional empathy in intelligent agents to facilitate precise social interaction","volume":"13","author":"Alanazi","year":"2023","journal-title":"Appl. Sci."},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1712.00726","article-title":"Cascade R-CNN: delving into high quality object detection","author":"Cai","year":"2017","journal-title":"arXiv"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2105.08935","article-title":"DeepFaceEditing: deep face generation and editing with disentangled geometry and appearance control","author":"Chen","year":"2021","journal-title":"arXiv"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"3366","DOI":"10.3390\/s22093366","article-title":"LEMON: a lightweight facial emotion recognition system for assistive robotics based on dilated residual convolutional neural networks","volume":"22","author":"Devaram","year":"2022","journal-title":"Sensors"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1109\/TPAMI.2017.2700390","article-title":"Trunk-branch ensemble convolutional neural networks for video-based face recognition","volume":"40","author":"Ding","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"E1454","DOI":"10.1073\/pnas.1322355111","article-title":"Compound facial expressions of emotion","volume":"111","author":"Du","year":"2014","journal-title":"Proc. Natl. Acad. Sci."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.neunet.2017.12.012","article-title":"Sigmoid-weighted linear units for neural network function approximation in reinforcement learning","volume":"107","author":"Elfwing","year":"2018","journal-title":"Neural Networks"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1177\/1059601120959288","article-title":"It\u2019s a matter of time: the role of temporal perceptions in emotional experiences of work interruptions","volume":"46","author":"Feldman","year":"2021","journal-title":"Group Org. Manag."},{"key":"ref10","first-page":"346","article-title":"Spatial pyramid pooling in deep convolutional networks for visual recognition","author":"He","year":"2014"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"90465","DOI":"10.1109\/ACCESS.2021.3091169","article-title":"Context-aware emotion recognition based on visual relationship detection","volume":"9","author":"Hoang","year":"2021","journal-title":"IEEE Access"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"8425","DOI":"10.1038\/s41598-023-35446-4","article-title":"A study on computer vision for facial emotion recognition","volume":"13","author":"Huang","year":"2023","journal-title":"Sci. Rep."},{"key":"ref13","author":"Jianhua","year":"2020"},{"key":"ref14","first-page":"2309","article-title":"EMOTIC: emotions in context dataset","author":"Kosti","year":"2017"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"2755","DOI":"10.1109\/TPAMI.2019.2916866","article-title":"Context based emotion recognition using EMOTIC dataset","volume":"42","author":"Kosti","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref16","article-title":"Global-local attention for emotion recognition","author":"Le","year":"2021","journal-title":"arXiv"},{"key":"ref17","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2019.01024","article-title":"Context-Aware Emotion Recognition Networks","volume-title":"2019 IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Lee","year":""},{"key":"ref18","doi-asserted-by":"publisher","first-page":"2583","DOI":"10.1109\/TPAMI.2018.2791608","article-title":"EAC-net: deep nets with enhancing and cropping for facial action unit detection","volume":"40","author":"Li","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"1195","DOI":"10.1109\/TAFFC.2020.2981446","article-title":"Deep facial expression recognition: a survey","volume":"13","author":"Li","year":"2022","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1007\/s00138-022-01288-9","article-title":"FERGCN: facial expression recognition based on graph convolution network","volume":"33","author":"Liao","year":"2022","journal-title":"Mach. Vis. Appl."},{"key":"ref21","first-page":"936","article-title":"Feature pyramid networks for object detection","author":"Lin","year":"2017"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1405.0312","article-title":"Microsoft COCO: common objects in context","author":"Lin","year":"2015","journal-title":"arXiv"},{"key":"ref23","doi-asserted-by":"crossref","DOI":"10.1109\/CVPRW.2010.5543262","article-title":"The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression","author":"Lucey","year":"2010"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1236","DOI":"10.1109\/TAFFC.2021.3122146","article-title":"Facial expression recognition with visual transformers and attentional selective fusion","volume":"14","author":"Ma","year":"2023","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref25","doi-asserted-by":"publisher","first-page":"5475","DOI":"10.3390\/s23125475","article-title":"Multimodal emotion detection via attention-based fusion of extracted facial and speech features","volume":"23","author":"Mamieva","year":"2023","journal-title":"Sensors"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2003.06692","article-title":"EmotiCon: context-aware multimodal emotion recognition using Frege\u2019s principle","author":"Mittal","year":"2020","journal-title":"arXiv"},{"key":"ref27","first-page":"3487","article-title":"Graph-based person signature for person re-identifications","author":"Nguyen","year":"2021"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1911.02499","article-title":"Dimensional emotion detection from categorical emotion","author":"Park","year":"2021","journal-title":"arXiv"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1506.01497","article-title":"Faster R-CNN: towards real-time object detection with region proposal networks","author":"Ren","year":"2016","journal-title":"arXiv"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1016\/j.procs.2019.05.038","article-title":"Deep learning approach for emotion recognition from human body movements with feedforward deep convolution neural networks","volume":"152","author":"Santhoshkumar","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref31","first-page":"10778","article-title":"EfficientDet: scalable and efficient object detection","author":"Tan","year":"2020"},{"key":"ref32","first-page":"5525","article-title":"WIDER FACE: a face detection benchmark","author":"Yang","year":"2016"},{"key":"ref33","first-page":"1","article-title":"A graph convolutional network for emotion recognition in context","author":"Zeng","year":"2020"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1609.06426","article-title":"From facial expression recognition to interpersonal relation prediction","author":"Zhang","year":"2017","journal-title":"arXiv"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.inffus.2020.01.011","article-title":"Emotion recognition using multi-modal data and machine learning techniques: a tutorial and review","volume":"59","author":"Zhang","year":"2020","journal-title":"Inform. Fusion"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1109\/MSP.2021.3106895","article-title":"Emotion recognition from multiple modalities: fundamentals and methodologies","volume":"38","author":"Zhao","year":"2021","journal-title":"IEEE Signal Process. Mag."},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-018-1140-0","article-title":"Semantic understanding of scenes through the ADE20K dataset","author":"Zhou","year":"2018","journal-title":"arXiv"}],"updated-by":[{"DOI":"10.3389\/frai.2024.1476791","type":"corrigendum","label":"Corrigendum","source":"publisher","updated":{"date-parts":[[2024,9,3]],"date-time":"2024-09-03T00:00:00Z","timestamp":1725321600000}}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1386753\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,3]],"date-time":"2024-09-03T07:37:00Z","timestamp":1725349020000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1386753\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,17]]},"references-count":37,"alternative-id":["10.3389\/frai.2024.1386753"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1386753","relation":{"corrigendum":[{"id-type":"doi","id":"10.3389\/frai.2024.1476791","asserted-by":"object"}]},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,17]]},"article-number":"1386753"}}