{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T10:19:24Z","timestamp":1782123564237,"version":"3.54.5"},"reference-count":38,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2020,9,21]],"date-time":"2020-09-21T00:00:00Z","timestamp":1600646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004515","name":"Universiti Kebangsaan Malaysia","doi-asserted-by":"publisher","award":["2020-018"],"award-info":[{"award-number":["2020-018"]}],"id":[{"id":"10.13039\/501100004515","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Compelling facial expression recognition (FER) processes have been utilized in very successful fields like computer vision, robotics, artificial intelligence, and dynamic texture recognition. However, the FER\u2019s critical problem with traditional local binary pattern (LBP) is the loss of neighboring pixels related to different scales that can affect the texture of facial images. To overcome such limitations, this study describes a new extended LBP method to extract feature vectors from images, detecting each image from facial expressions. The proposed method is based on the bitwise AND operation of two rotational kernels applied on LBP(8,1) and LBP(8,2) and utilizes two accessible datasets. Firstly, the facial parts are detected and the essential components of a face are observed, such as eyes, nose, and lips. The portion of the face is then cropped to reduce the dimensions and an unsharp masking kernel is applied to sharpen the image. The filtered images then go through the feature extraction method and wait for the classification process. Four machine learning classifiers were used to verify the proposed method. This study shows that the proposed multi-scale featured local binary pattern (MSFLBP), together with Support Vector Machine (SVM), outperformed the recent LBP-based state-of-the-art approaches resulting in an accuracy of 99.12% for the Extended Cohn\u2013Kanade (CK+) dataset and 89.08% for the Karolinska Directed Emotional Faces (KDEF) dataset.<\/jats:p>","DOI":"10.3390\/s20185391","type":"journal-article","created":{"date-parts":[[2020,9,21]],"date-time":"2020-09-21T08:18:01Z","timestamp":1600676281000},"page":"5391","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Development of a Robust Multi-Scale Featured Local Binary Pattern for Improved Facial Expression Recognition"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3728-4927","authenticated-orcid":false,"given":"Suraiya","family":"Yasmin","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, International Islamic University Chittagong, Chittagong-4318, Bangladesh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3773-0950","authenticated-orcid":false,"given":"Refat Khan","family":"Pathan","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, BGC Trust University Bangladesh, Chittagong-4381, Bangladesh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Munmun","family":"Biswas","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, BGC Trust University Bangladesh, Chittagong-4381, Bangladesh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mayeen Uddin","family":"Khandaker","sequence":"additional","affiliation":[{"name":"Centre for Biomedical Physics, School of Healthcare and Medical Sciences, Sunway University, Bandar Sunway 47500, Selangor, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4086-7672","authenticated-orcid":false,"given":"Mohammad Rashed Iqbal","family":"Faruque","sequence":"additional","affiliation":[{"name":"Space Science Centre (ANGKASA), Institute of Climate Change (IPI), Universiti Kebangsaan Malaysia, UKM, Bangi 43600, Selangor, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yu, Z., and Zhang, C. (2015, January 9\u201313). Image based static facial expression recognition with multiple deep network learning. Proceedings of the ICMI 2015-Proceedings of the 2015 ACM International Conference on Multimodal Interaction, Washington, DC, USA.","DOI":"10.1145\/2818346.2830595"},{"key":"ref_2","unstructured":"Kahou, S.E., Michalski, V., Konda, K., Memisevic, R., and Pal, C. (2015, January 9\u201313). Recurrent neural networks for emotion recognition in video. Proceedings of the ICMI 2015-Proceedings of the 2015 ACM International Conference on Multimodal Interaction, Washington, DC, USA."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liu, M., Li, S., Shan, S., Wang, R., and Chen, X. (2015). Deeply learning deformable facial action parts model for dynamic expression analysis. Asian Conference on Computer Vision, Proceedings of the Lecture Notes in Computer Science, Springer International Publishing.","DOI":"10.1007\/978-3-319-16817-3_10"},{"key":"ref_4","unstructured":"Nakashima, H., Aghajan, H., and Augusto, J.C. (2010). Recognizing Facial Expressions Automatically from Video. Handbook of Ambient Intelligence and Smart Environments, Springer International Publishing."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Yang, B., Cao, J., Ni, R., and Zhang, Y. (2017). Facial Expression Recognition Using Weighted Mixture Deep Neural Network Based on Double-Channel Facial Images. IEEE Access.","DOI":"10.1109\/ACCESS.2017.2784096"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, Y., Zeng, J., Shan, S., and Chen, X. (2018). Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism. IEEE Trans. Image Process.","DOI":"10.1109\/ICPR.2018.8545853"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, Y., Li, Y., Ma, X., and Song, R. (2017). Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas. Sensors, 17.","DOI":"10.20944\/preprints201701.0102.v1"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhao, Y., and Xu, J. (2019). An Improved Micro-Expression Recognition Method Based on Necessary Morphological Patches. Symmetry, 11.","DOI":"10.3390\/sym11040497"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yang, J., Wang, X., Han, S., Wang, J., Park, D.S., and Wang, Y. (2019). Improved Real-Time Facial Expression Recognition Based on a Novel Balanced and Symmetric Local Gradient Coding. Sensors, 19.","DOI":"10.3390\/s19081899"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhang, W., Shan, S., Zhang, H., Gao, W., and Chen, X. (2005). Multi-resolution Histograms of Local Variation Patterns (MHLVP) for robust face recognition. International Conference on Audio-and Video-Based Biometric Person Authentication, Proceedings of the Lecture Notes in Computer Science, Springer International Publishing.","DOI":"10.1007\/11527923_98"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1109\/TSMCC.2011.2118750","article-title":"Local binary patterns and its application to facial image analysis: A survey","volume":"41","author":"Huang","year":"2011","journal-title":"IEEE Trans. Syst. Man Cybern. Part C"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1016\/j.procs.2015.08.011","article-title":"Facial Expression Recognition: A Survey","volume":"58","author":"Kumari","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ahonen, T., Hadid, A., and Pietik\u00e4inen, M. (2006). Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2006.244"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Canedo, D., and Neves, A.J.R. (2019). Facial Expression Recognition Using Computer Vision: A Systematic Review. Appl. Sci., 9.","DOI":"10.3390\/app9214678"},{"key":"ref_15","unstructured":"Huang, D., Wang, Y., and Wang, Y. A robust method for near-infrared face recognition is based on extended local binary patterns. Proceedings of the Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer."},{"key":"ref_16","unstructured":"Guo, Z., Zhang, L., and Zhang, D. (2010). A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhao, G., and Pietik\u00e4inen, M. (2007). Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2007.1110"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sajjad, M., Shah, A., Jan, Z., Shah, S.I., Baik, S.W., and Mehmood, I. (2017). Facial appearance and texture feature-based robust facial expression recognition framework for sentiment knowledge discovery. Cluster Comput.","DOI":"10.1007\/s10586-017-0935-z"},{"key":"ref_19","unstructured":"Zhang, B., Gao, Y., Zhao, S., and Liu, J. (2010). Local derivative pattern versus local binary pattern: Face recognition with high-order local pattern descriptor. IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zangeneh, E., and Moradi, A. (2018). Facial expression recognition by using differential geometric features. Imaging Sci. J.","DOI":"10.1080\/13682199.2018.1509176"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chen, J., Takiguchi, T., and Ariki, Y. (2017). Rotation-reversal invariant HOG cascade for facial expression recognition. Signal Image Video Process.","DOI":"10.1007\/s11760-017-1111-x"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tsai, H.H., and Chang, Y.C. (2018). Facial expression recognition using a combination of multiple facial features and a support vector machine. Soft Comput.","DOI":"10.1007\/s00500-017-2634-3"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Alphonse, A.S., and Dharma, D. (2018). Novel directional patterns and a Generalized Supervised Dimension Reduction System (GSDRS) for facial emotion recognition. Multimed. Tools Appl.","DOI":"10.1007\/s11042-017-5141-8"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yu, Z., Liu, G., Liu, Q., and Deng, J. (2018). Spatio-temporal convolutional features with nested LSTM for facial expression recognition. Neurocomputing.","DOI":"10.1016\/j.neucom.2018.07.028"},{"key":"ref_25","unstructured":"Zhang, L., Gao, Q., and Zhang, D. (2008, January 23\u201328). Directional independent component analysis with tensor representation. Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Anchorage, AK, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Samara, A., Galway, L., Bond, R., and Wang, H. (2019). Affective state detection via facial expression analysis within a human-computer interaction context. J. Ambient Intell. Humaniz. Comput.","DOI":"10.1007\/s12652-017-0636-8"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Turabzadeh, S., Meng, H., Swash, R., Pleva, M., and Juhar, J. (2018). Facial Expression Emotion Detection for Real-Time Embedded Systems. Technologies, 6.","DOI":"10.3390\/technologies6010017"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez, A., Pujol, F.A., and Mora, H. (2020). Application of Texture Descriptors to Facial Emotion Recognition in Infants. Appl. Sci., 10.","DOI":"10.3390\/app10031115"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13\u201318). The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, CVPRW, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543262"},{"key":"ref_30","unstructured":"Kanade, T., Cohn, J.F., and Tian, Y. (2000, January 28\u201330). Comprehensive database for facial expression analysis. Proceedings of the Proceedings-4th IEEE International Conference on Automatic Face and Gesture Recognition, FG, Grenoble, France."},{"key":"ref_31","unstructured":"(2020, August 30). Kernel (Image Processing), n.d., para.2, Wikipedia. Available online: https:\/\/en.wikipedia.org\/w\/index.php?title=Kernel_(image_processing)."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Xiong, H., Zhang, D., Martyniuk, C.J., Trudeau, V.L., and Xia, X. (2008). Using Generalized Procrustes Analysis (GPA) for normalization of cDNA microarray data. BMC Bioinform.","DOI":"10.1186\/1471-2105-9-25"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sert, M., and Aksoy, N. (2016, January 16). Recognizing facial expressions of emotion using action unit-specific decision thresholds. Proceedings of the 2nd Workshop on Advancements in Social Signal Processing for Multimodal Interaction-ASSP4MI \u201916, Tokyo, Japan.","DOI":"10.1145\/3005467.3005471"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liliana, D.Y., Basaruddin, C., and Widyanto, M.R. (2017, January 10\u201313). Mix Emotion Recognition from Facial Expression using SVM-CRF Sequence Classifier. Proceedings of the International Conference on Algorithms, Computing and Systems-ICACS \u201917, Jeju Island, Korea.","DOI":"10.1145\/3127942.3127958"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1007\/s00521-018-3358-8","article-title":"A hybrid deep learning neural approach for emotion recognition from facial expressions for socially assistive robots","volume":"29","author":"Elshaw","year":"2018","journal-title":"Neural Comput. Appl."},{"key":"ref_36","unstructured":"Yaddaden, Y., Bouzouane, A., Adda, M., and Bouchard, B. (July, January 29). A New Approach of Facial Expression Recognition for Ambient Assisted Living. Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments-PETRA \u201916, Corfu, Greece."},{"key":"ref_37","unstructured":"FaceReader 8, Technical Specifications (2020, September 09). Noldus Information Technology. Available online: https:\/\/www.mindmetriks.com\/uploads\/4\/4\/6\/0\/44607631\/technical_specs__facereader_8.0.pdf."},{"key":"ref_38","unstructured":"Face-An AI Service that Analyzes Faces in Images (2020, September 09). Microsoft Azure. Available online: https:\/\/azure.microsoft.com\/en-us\/services\/cognitive-services\/face\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/18\/5391\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:11:54Z","timestamp":1760177514000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/18\/5391"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,21]]},"references-count":38,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["s20185391"],"URL":"https:\/\/doi.org\/10.3390\/s20185391","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,21]]}}}