{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T05:08:05Z","timestamp":1774933685802,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T00:00:00Z","timestamp":1615507200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Trade, Industry &amp; Energy (MOTIE, Korea)","award":["20003519"],"award-info":[{"award-number":["20003519"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Facial emotion recognition (FER) systems play a significant role in identifying driver emotions. Accurate facial emotion recognition of drivers in autonomous vehicles reduces road rage. However, training even the advanced FER model without proper datasets causes poor performance in real-time testing. FER system performance is heavily affected by the quality of datasets than the quality of the algorithms. To improve FER system performance for autonomous vehicles, we propose a facial image threshing (FIT) machine that uses advanced features of pre-trained facial recognition and training from the Xception algorithm. The FIT machine involved removing irrelevant facial images, collecting facial images, correcting misplacing face data, and merging original datasets on a massive scale, in addition to the data-augmentation technique. The final FER results of the proposed method improved the validation accuracy by 16.95% over the conventional approach with the FER 2013 dataset. The confusion matrix evaluation based on the unseen private dataset shows a 5% improvement over the original approach with the FER 2013 dataset to confirm the real-time testing.<\/jats:p>","DOI":"10.3390\/s21062026","type":"journal-article","created":{"date-parts":[[2021,3,14]],"date-time":"2021-03-14T23:52:06Z","timestamp":1615765926000},"page":"2026","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":69,"title":["The Extensive Usage of the Facial Image Threshing Machine for Facial Emotion Recognition Performance"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3443-7468","authenticated-orcid":false,"given":"Jung Hwan","family":"Kim","sequence":"first","affiliation":[{"name":"School of Electronic and Electrical Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7100-8665","authenticated-orcid":false,"given":"Alwin","family":"Poulose","sequence":"additional","affiliation":[{"name":"School of Electronic and Electrical Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7769-0236","authenticated-orcid":false,"given":"Dong Seog","family":"Han","sequence":"additional","affiliation":[{"name":"School of Electronic and Electrical Engineering, Kyungpook National University, 80 Daehak-ro, Buk-gu, Daegu 41566, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,12]]},"reference":[{"key":"ref_1","first-page":"273","article-title":"Angry thoughts and aggressive behavior among Malaysian driver: A preliminary study to test model of accident involvement","volume":"10","author":"Ismail","year":"2009","journal-title":"Eur. J. Soc. Sci."},{"key":"ref_2","unstructured":"Carrier, P.-L., Courville, A., Goodfellow, I.J., Mirza, M., and Bengio, Y. (2013). FER-2013 Face Database, Universit de Montral."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13\u201318). The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543262"},{"key":"ref_4","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features from Tiny Images, University of Toronto. Technical Report."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sakai, A., Minoda, Y., and Morikawa, K. (September, January 31). Data augmentation methods for machine-learning-based classification of bio-signals. Proceedings of the In 2017 10th Biomedical Engineering International Conference (BMEiCON), Hokkaido, Japan.","DOI":"10.1109\/BMEiCON.2017.8229109"},{"key":"ref_7","unstructured":"Li, S., and Deng, W. (2020). Deep facial expression recognition: A survey. IEEE Trans. Affect. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"E7900","DOI":"10.1073\/pnas.1702247114","article-title":"Self-report captures 27 distinct categories of emotion bridged by continuous gradients","volume":"114","author":"Cowen","year":"2017","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Thu, K.M.M., and Thein, N.N.M. (2019, January 6\u20137). Edge based Crime Assistance System with Cloud Computing. Proceedings of the 2019 International Conference on Advanced Information Technologies (ICAIT), Yangon, Myanmar.","DOI":"10.1109\/AITC.2019.8920883"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wati, D.A.R., and Abadianto, D. (2017, January 1\u20132). Design of face detection and recognition system for smart home security application. Proceedings of the 2017 Information Systems and Electrical Engineering (ICITISEE), Yogyakarta, Indonesia.","DOI":"10.1109\/ICITISEE.2017.8285524"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wozniak, M., Silka, J., Wieczorek, M., and Alrashoud, M. (2020). Recurrent Neural Network model for IoT and networking malware threads detection. IEEE Trans. Ind. Inform.","DOI":"10.1109\/TII.2020.3021689"},{"key":"ref_12","first-page":"2101","article-title":"Body pose prediction based on motion sensor data and recurrent neural network","volume":"17","author":"Wieczorek","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"McDuff, D., Kaliouby, R., Senechal, T., Amr, M., Cohn, J., and Picard, R. (2013, January 23\u201328). Affectiva-mit facial expression dataset (am-fed): Naturalistic and spontaneous facial expressions collected. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA.","DOI":"10.1109\/CVPRW.2013.130"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"T\u00fcmen, V., S\u00f6ylemez, \u00d6.F., and Ergen, B. (2017, January 16\u201317). Facial emotion recognition on a dataset using convolutional neural network. Proceedings of the 2017 International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey.","DOI":"10.1109\/IDAP.2017.8090281"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zahara, L., Musa, P., Wibowo, E.P., Karim, I., and Musa, S.B. (2020, January 3\u20134). The Facial Emotion Recognition (FER-2013) Dataset for Prediction System of Micro-Expressions Face Using the Convolutional Neural Network (CNN) Algorithm based Raspberry Pi. Proceedings of the 2020 Fifth International Conference on Informatics and Computing (ICIC), Gorontalo, Indonesia.","DOI":"10.1109\/ICIC50835.2020.9288560"},{"key":"ref_16","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, Kauai, HI, USA."},{"key":"ref_17","first-page":"1612","article-title":"A short introduction to boosting","volume":"14","author":"Freund","year":"1999","journal-title":"J. Jpn. Soc. Artif. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"41273","DOI":"10.1109\/ACCESS.2019.2907327","article-title":"Efficient facial expression recognition algorithm based on hierarchical deep neural network structure","volume":"7","author":"Kim","year":"2019","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, S., Tang, X., and Wang, D. (2020, January 12\u201315). Facial Expression Recognition Based on Sobel Operator and Improved CNN-SVM. Proceedings of the 2020 IEEE 3rd International Conference on Information Communication and Signal Processing (ICICSP), Shanghai, China.","DOI":"10.1109\/ICICSP50920.2020.9232063"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint face detection and alignment using multitask cascaded convolutional networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_21","unstructured":"Kim, J.H., Poulose, A., and Han, D.S. (2020, January 13). Facial Image Threshing Machine for Collecting Facial Emotion Recognition Dataset. Proceedings of the Symposium of the Korean Institute of communications and Information Sciences (KICS) Fall Conference, Seoul, Korea."},{"key":"ref_22","unstructured":"Rosebrock, A. (2017). Deep Learning for Computer Vision with Python: Starter Bundle, PyImageSearch."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_24","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Wey, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"142243","DOI":"10.1109\/ACCESS.2020.3012703","article-title":"PyFER: A Facial Expression Recognizer Based on Convolutional Neural Networks","volume":"8","author":"Kabakus","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bruch, S., Wang, X., Bendersky, M., and Najork, M. (2019, January 2\u20135). An analysis of the softmax cross entropy loss for learning-to-rank with binary relevance. Proceedings of the 2019 ACM SIGIR International Conference on Theory of Information Retrieval, Santa Clara, CA, USA.","DOI":"10.1145\/3341981.3344221"},{"key":"ref_29","unstructured":"Wang, R., and Li, J. (August, January 28). Bayes test of precision, recall, and f1 measure for comparison of two natural language processing models. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/s11222-009-9153-8","article-title":"Estimation of prediction error by using K-fold cross-validation","volume":"21","author":"Fushiki","year":"2011","journal-title":"Stat. Comput."},{"key":"ref_31","unstructured":"Kim, J.H., and Han, D.S. (2020, January 12\u201316). Data Augmentation & Merging Dataset for Facial Emotion Recognition. Proceedings of the Symposium of the 1st Korea Artificial Intelligence Conference, Jeju, Korea."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2026\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:35:02Z","timestamp":1760160902000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2026"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,12]]},"references-count":31,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21062026"],"URL":"https:\/\/doi.org\/10.3390\/s21062026","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,12]]}}}