{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T12:44:23Z","timestamp":1780058663133,"version":"3.54.0"},"reference-count":44,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T00:00:00Z","timestamp":1685664000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Korean Government","award":["2022R1A5A7026673"],"award-info":[{"award-number":["2022R1A5A7026673"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the advancement of computer hardware and communication technologies, deep learning technology has made significant progress, enabling the development of systems that can accurately estimate human emotions. Factors such as facial expressions, gender, age, and the environment influence human emotions, making it crucial to understand and capture these intricate factors. Our system aims to recommend personalized images by accurately estimating human emotions, age, and gender in real time. The primary objective of our system is to enhance user experiences by recommending images that align with their current emotional state and characteristics. To achieve this, our system collects environmental information, including weather conditions and user-specific environment data through APIs and smartphone sensors. Additionally, we employ deep learning algorithms for real-time classification of eight types of facial expressions, age, and gender. By combining this facial information with the environmental data, we categorize the user\u2019s current situation into positive, neutral, and negative stages. Based on this categorization, our system recommends natural landscape images that are colorized using Generative Adversarial Networks (GANs). These recommendations are personalized to match the user\u2019s current emotional state and preferences, providing a more engaging and tailored experience. Through rigorous testing and user evaluations, we assessed the effectiveness and user-friendliness of our system. Users expressed satisfaction with the system\u2019s ability to generate appropriate images based on the surrounding environment, emotional state, and demographic factors such as age and gender. The visual output of our system significantly impacted users\u2019 emotional responses, resulting in a positive mood change for most users. Moreover, the system\u2019s scalability was positively received, with users acknowledging its potential benefits when installed outdoors and expressing a willingness to continue using it. Compared to other recommender systems, our integration of age, gender, and weather information provides personalized recommendations, contextual relevance, increased engagement, and a deeper understanding of user preferences, thereby enhancing the overall user experience. The system\u2019s ability to comprehend and capture intricate factors that influence human emotions holds promise in various domains, including human\u2013computer interaction, psychology, and social sciences.<\/jats:p>","DOI":"10.3390\/s23115304","type":"journal-article","created":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T10:08:41Z","timestamp":1685700521000},"page":"5304","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Image Recommendation System Based on Environmental and Human Face Information"],"prefix":"10.3390","volume":"23","author":[{"given":"Hye-min","family":"Won","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Ajou University, Suwon-si 16499, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7576-1347","authenticated-orcid":false,"given":"Yong Seok","family":"Heo","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Ajou University, Suwon-si 16499, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1792-0327","authenticated-orcid":false,"given":"Nojun","family":"Kwak","sequence":"additional","affiliation":[{"name":"Graduate School of Convergence Science and Technology, RICS, Seoul National University, Seoul 08826, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,6,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1007\/s40747-020-00212-w","article-title":"Artificial intelligence in recommender systems","volume":"7","author":"Zhang","year":"2021","journal-title":"Complex Intell. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3543846","article-title":"Reinforcement learning based recommender systems: A survey","volume":"55","author":"Afsar","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_3","unstructured":"Bridger, R. (2017). Introduction to Human Factors and Ergonomics, CRC Press."},{"key":"ref_4","unstructured":"Kadir, B. (2020). Designing New Ways of Working in Industry 4.0: Aligning Humans, Technology, and Organization in the Transition to Industry 4.0. [Ph.D. Thesis, Technical University of Denmark]."},{"key":"ref_5","unstructured":"Zhang, S., Yao, L., and Sun, A. (2017). Deep learning based recommender system: A survey and new perspectives. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1109\/TKDE.2005.99","article-title":"Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions","volume":"17","author":"Adomavicius","year":"2005","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1007\/s42979-022-01619-7","article-title":"Emotion Detection-Based Video Recommendation System Using Machine Learning and Deep Learning Framework","volume":"4","author":"Bokhare","year":"2023","journal-title":"SN Comput. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1016\/j.ijinfomgt.2016.01.005","article-title":"Collaborative filtering with facial expressions for online video recommendation","volume":"36","author":"Choi","year":"2016","journal-title":"Int. J. Inf. Manag."},{"key":"ref_9","first-page":"701","article-title":"Emotion based personalized recommendation system","volume":"7","author":"Babanne","year":"2020","journal-title":"Int. Res. J. Eng. Technol. (IRJET)"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1000132","DOI":"10.4172\/2332-2594.1000132","article-title":"Does Temperature Affect Homicide Rate?","volume":"3","author":"Mishra","year":"2015","journal-title":"J. Climatol. Weather. Forecast."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e571","DOI":"10.1016\/S2542-5196(21)00210-2","article-title":"Interpersonal violence associated with hot weather","volume":"5","author":"Mahendran","year":"2021","journal-title":"Lancet Planet. Health"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"102240","DOI":"10.1016\/j.jhealeco.2019.102240","article-title":"Temperature and mental health: Evidence from the spectrum of mental health outcomes","volume":"68","author":"Mullins","year":"2019","journal-title":"J. Health Econ."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1080\/00431672.1959.9926960","article-title":"The discomfort index","volume":"12","author":"Thom","year":"1959","journal-title":"Weatherwise"},{"key":"ref_14","first-page":"131","article-title":"Thermal remote sensing of Thom\u2019s discomfort index (DI): Comparison with in-situ measurements","volume":"5983","author":"Stathopoulou","year":"2005","journal-title":"Remote. Sens. Environ. Monit. Gis Appl. Geol. V"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1080\/23754931.2021.1977977","article-title":"Assessment of Thermal Discomfort Variation in Fiji\u2019s Major Urban Centers","volume":"8","author":"Fong","year":"2022","journal-title":"Pap. Appl. Geogr."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mistry, M. (2020). A high spatiotemporal resolution global gridded dataset of historical human discomfort indices. Atmosphere, 11.","DOI":"10.3390\/atmos11080835"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1038\/s41597-021-01010-w","article-title":"A high-spatial-resolution dataset of human thermal stress indices over South and East Asia","volume":"8","author":"Yan","year":"2021","journal-title":"Sci. Data"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"81087","DOI":"10.1007\/s11356-022-23328-7","article-title":"Types, sources, socioeconomic impacts, and control strategies of environmental noise: A review","volume":"29","author":"Farooqi","year":"2022","journal-title":"Environ. Sci. Pollut. Res."},{"key":"ref_19","unstructured":"(2023, May 28). International Organization for Standardization Lighting of Work Places\u2014Part 1: Indoor. Available online: https:\/\/www.iso.org\/standard\/28857.html,2002."},{"key":"ref_20","unstructured":"(2023, May 28). European Committee for Standardization Light and Lighting\u2014Lighting of Work Places\u2014Part 1: Indoor Work Places. Available online: https:\/\/standards.iteh.ai\/catalog\/standards\/cen\/53fc4ff7-e7df-4ebd-a730-0d5f0ea888e0\/en-12464-1-2021,2021."},{"key":"ref_21","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference On Computer Vision And Pattern Recognition, CVPR 2001, Kauai, HI, USA."},{"key":"ref_22","unstructured":"Ma, S., and Bai, L. (2016, January 26\u201328). A face detection algorithm based on Adaboost and new Haar-Like feature. Proceedings of the 2016 7th IEEE International Conference on Software Engineering And Service Science (ICSESS), Beijing, China."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2056003","DOI":"10.1142\/S0218001420560030","article-title":"Facial expression recognition method based on improved VGG convolutional neural network","volume":"34","author":"Cheng","year":"2020","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Huang, Y., Dong, C., Luo, X., and Dai, Q. (2021, January 11\u201313). Facial Expression Recognition Algorithm Based on Improved VGG16 Network. Proceedings of the 2021 6th International Symposium on Computer and Information Processing Technology (ISCIPT), Changsha, China.","DOI":"10.1109\/ISCIPT53667.2021.00103"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"032030","DOI":"10.1088\/1742-6596\/2083\/3\/032030","article-title":"Facial expression recognition based on improved VGG convolutional neural network","volume":"2083","author":"Dong","year":"2021","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_26","first-page":"1589","article-title":"Automatic facial recognition using VGG16 based transfer learning model","volume":"41","author":"Dubey","year":"2020","journal-title":"J. Inf. Optim. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1007\/s00138-022-01304-y","article-title":"A novel multi-feature fusion deep neural network using HOG and VGG-Face for facial expression classification","volume":"33","author":"Ahadit","year":"2022","journal-title":"Mach. Vis. Appl."},{"key":"ref_28","unstructured":"Sheoran, V., Joshi, S., and Bhayani, T. (2020, January 4\u20136). Age and gender prediction using deep cnns and transfer learning. Proceedings of the Computer Vision and Image Processing: 5th International Conference, CVIP 2020, Prayagraj, India."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Liu, X., Ma, J., and Wang, Q. (2022, January 10\u201312). Facial Expression Recognition based on Convolutional Neural Network with Sparse Representation. Proceedings of the 2022 8th International Conference on Systems And Informatics (ICSAI), Kunming, China.","DOI":"10.1109\/ICSAI57119.2022.10005481"},{"key":"ref_30","unstructured":"Kanade, T., Cohn, J., and Tian, Y. (2000, January 28\u201330). Comprehensive database for facial expression analysis. Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lucey, P., Cohn, J., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13\u201318). The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. Proceedings of the 2010 IEEE Computer Society Conference On Computer Vision And Pattern Recognition-workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543262"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1016\/j.patcog.2016.07.026","article-title":"Facial expression recognition with convolutional neural networks: Coping with few data and the training sample order","volume":"61","author":"Lopes","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_33","unstructured":"Ricanek, K., and Tesafaye, T. (2006, January 10\u201312). Morph: A longitudinal image database of normal adult age-progression. Proceedings of the 7th International Conference On Automatic Face And Gesture Recognition (FGR06), Southampton, UK."},{"key":"ref_34","unstructured":"Hiba, S., and Keller, Y. (2021). Hierarchical attention-based age estimation and Bias estimation. arXiv."},{"key":"ref_35","unstructured":"Gao, B., Liu, X., Zhou, H., Wu, J., and Geng, X. (2020). Learning expectation of label distribution for facial age and attractiveness estimation. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1002\/col.5080090106","article-title":"Physiological response to color: A critical review","volume":"9","author":"Kaiser","year":"1984","journal-title":"Color Res. Appl."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1037\/0096-3445.123.4.394","article-title":"Effects of color on emotions","volume":"123","author":"Valdez","year":"1994","journal-title":"J. Exp. Psychol. Gen."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1126\/science.1169144","article-title":"Blue or red? Exploring the effect of color on cognitive task performances","volume":"323","author":"Mehta","year":"2009","journal-title":"Science"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1006\/jevp.2000.0193","article-title":"Designing effective study environments","volume":"21","author":"Stone","year":"2001","journal-title":"J. Environ. Psychol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1007\/s11747-010-0245-y","article-title":"Exciting red and competent blue: The importance of color in marketing","volume":"40","author":"Labrecque","year":"2012","journal-title":"J. Acad. Mark. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1080\/13527260500247827","article-title":"Are you selling the right colour? A cross-cultural review of colour as a marketing cue","volume":"12","author":"Aslam","year":"2006","journal-title":"J. Mark. Commun."},{"key":"ref_42","first-page":"55","article-title":"Color Red: Implications for applied psychology and marketing research","volume":"49","author":"Piotrowski","year":"2012","journal-title":"Psychol.-Educ.-Interdiscip. J."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhu, J., Park, T., Isola, P., and Efros, A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference On Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"972","DOI":"10.1109\/LSP.2022.3163685","article-title":"TPE-GAN: Thumbnail preserving encryption based on GAN with key","volume":"29","author":"Chai","year":"2022","journal-title":"IEEE Signal Process. Lett."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5304\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:48:14Z","timestamp":1760125694000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5304"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,2]]},"references-count":44,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23115304"],"URL":"https:\/\/doi.org\/10.3390\/s23115304","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,2]]}}}