{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T06:13:04Z","timestamp":1774678384785,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T00:00:00Z","timestamp":1761609600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100013355","name":"Institut Teknologi Sepuluh Nopember","doi-asserted-by":"crossref","award":["ITS-Ubaya-no 1777\/PKS\/ITS\/2023"],"award-info":[{"award-number":["ITS-Ubaya-no 1777\/PKS\/ITS\/2023"]}],"id":[{"id":"10.13039\/501100013355","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>Meme image sentiment analysis is a task of examining public opinion based on meme images posted on social media. In various fields, stakeholders often need to quickly and accurately determine the sentiment of memes from large amounts of available data. Therefore, innovation is needed in image pre-processing so that an increase in performance metrics, especially accuracy, can be obtained in improving the classification of meme image sentiment. This is because sentiment classification using human face datasets yields higher accuracy than using meme images. This research aims to develop a sentiment analysis model for meme images based on key points. The analyzed meme images contain human faces. The facial features extracted using key points are the eyebrows, eyes, and mouth. In the proposed method, key points of facial features are represented in the form of graphs, specifically directed graphs, weighted graphs, or weighted directed graphs. These graph representations of key points are then used to build a sentiment analysis model based on a Deep Neural Network (DNN) with three layers (hidden layer: i = 64, j = 64, k = 90). There are several contributions of this study, namely developing a human facial sentiment detection model using key points, representing key points as various graphs, and constructing a meme dataset with Indonesian text. The proposed model is evaluated using several metrics, namely accuracy, precision, recall, and F-1 score. Furthermore, a comparative analysis is conducted to evaluate the performance of the proposed model against existing approaches. The experimental results show that the proposed model, which utilized the directed graph representation of key points, obtained the highest accuracy at 83% and F1 score at 81%, respectively.<\/jats:p>","DOI":"10.3390\/informatics12040118","type":"journal-article","created":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T09:16:07Z","timestamp":1761642967000},"page":"118","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Sentiment Analysis of Meme Images Using Deep Neural Network Based on Keypoint Representation"],"prefix":"10.3390","volume":"12","author":[{"given":"Endah","family":"Asmawati","sequence":"first","affiliation":[{"name":"Department of Informatics Engineering, Universitas Surabaya, Surabaya 60293, Indonesia"},{"name":"Department of Informatics, Institut Teknologi Sepuluh Nopember, Surabaya 60111, Indonesia"}]},{"given":"Ahmad","family":"Saikhu","sequence":"additional","affiliation":[{"name":"Department of Informatics, Institut Teknologi Sepuluh Nopember, Surabaya 60111, Indonesia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6560-2975","authenticated-orcid":false,"given":"Daniel O.","family":"Siahaan","sequence":"additional","affiliation":[{"name":"Department of Informatics, Institut Teknologi Sepuluh Nopember, Surabaya 60111, Indonesia"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,28]]},"reference":[{"key":"ref_1","unstructured":"(2021, July 13). Tesaurus Meme, Available online: https:\/\/kbbi.kemdikbud.go.id\/."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mertiya, M., and Singh, A. (2016, January 26\u201327). Combining naive bayes and adjective analysis for sentiment detection on Twitter. Proceedings of the 2016 International Conference on Inventive Computation Technologies, Coimbatore, India.","DOI":"10.1109\/INVENTIVE.2016.7824847"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pozzi, F.A., Fersini, E., Messina, E., and Liu, B. (2017). Challenges of Sentiment Analysis in Social Networks: An Overview, Elsevier Inc.","DOI":"10.1016\/B978-0-12-804412-4.00001-2"},{"key":"ref_4","first-page":"26","article-title":"Approaches, Tools and Applications for Sentiment Analysis Implementation","volume":"125","author":"Ferri","year":"2015","journal-title":"Int. J. Comput. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chaudhry, H.N., Javed, Y., Kulsoom, F., Mehmood, Z., Khan, Z.I., Shoaib, U., and Janjua, S.H. (2021). Sentiment analysis of before and after elections: Twitter data of U.S. election 2020. Electronics, 10.","DOI":"10.3390\/electronics10172082"},{"key":"ref_6","first-page":"51","article-title":"the Application of Sentiment Analysis in Tourism Research: A Brief Review","volume":"8","author":"Hashim","year":"2020","journal-title":"IJBTS Int. J. Bus. Tour. Appl. Sci."},{"key":"ref_7","first-page":"2009","article-title":"Literature review on sentiment analysis","volume":"9","author":"Hemamalini","year":"2020","journal-title":"Int. J. Sci. Technol. Res."},{"key":"ref_8","first-page":"6","article-title":"Systematic Literature Review on Opinion Mining of Big Data for Government Intelligence","volume":"14","author":"Kumar","year":"2017","journal-title":"Webology"},{"key":"ref_9","first-page":"6971","article-title":"Lexicon-Based Approach for Sentiment Analysis to Student Feedback","volume":"19","author":"Almosawi","year":"2022","journal-title":"Webology"},{"key":"ref_10","first-page":"100174","article-title":"Sentiment analysis on images using convolutional neural networks based Inception-V3 transfer learning approach","volume":"3","author":"Meena","year":"2023","journal-title":"Int. J. Inf. Manag. Data Insights"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1016\/j.procs.2022.08.050","article-title":"Sentiment Analysis from Images using VGG19 based Transfer Learning Approach","volume":"204","author":"Meena","year":"2022","journal-title":"Procedia Comput. Sci."},{"key":"ref_12","first-page":"2588","article-title":"Ensemble-based face expression recognition approach for image sentiment analysis","volume":"12","author":"Moung","year":"2022","journal-title":"Int. J. Electr. Comput. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"90495","DOI":"10.1109\/ACCESS.2020.2993803","article-title":"Facial Sentiment Analysis Using AI Techniques: State-of-the-Art, Taxonomies, and Challenges","volume":"8","author":"Patel","year":"2020","journal-title":"IEEE Access"},{"key":"ref_14","first-page":"736","article-title":"Sentiment Analysis from Face Expressions Based on Image Processing Using Deep Learning Methods","volume":"8","author":"Aksoy","year":"2022","journal-title":"J. Adv. Res. Nat. Appl. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"de Paula, D., and Alexandre, L.A. (2022, January 4\u20136). Facial Emotion Recognition for Sentiment Analysis of Social Media Data. Proceedings of the 10th Iberian Conference, IbPRIA 2022, Aveiro, Portugal.","DOI":"10.1007\/978-3-031-04881-4_17"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"111720","DOI":"10.1016\/j.patcog.2025.111720","article-title":"Context transformer with multiscale fusion for robust facial emotion recognition","volume":"167","author":"Gan","year":"2025","journal-title":"Pattern Recognit."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"So, J., and Han, Y. (2025). Facial Landmark-Driven Keypoint Feature Extraction for Robust Facial Expression Recognition. Sensors, 25.","DOI":"10.3390\/s25123762"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Avvaru, A., and Vobilisetty, S. (2020, January 12\u201313). BERT at SemEval-2020 Task 8: Using BERT to analyse meme emotions. Proceedings of the Fourteenth Workshop on Semantic Evaluation, Barcelona, Spain.","DOI":"10.18653\/v1\/2020.semeval-1.144"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"510253","DOI":"10.1155\/2021\/5510253","article-title":"Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification","volume":"2021","author":"Aggarwal","year":"2021","journal-title":"Complexity"},{"key":"ref_20","unstructured":"Thapa, S., Shah, A., Jafri, F., Naseem, U., and Razzak, I. (, January 7\u2013December). A Multi-Modal Dataset for Hate Speech Detection on Social Media: Case-study of Russia-Ukraine Conflict. Proceedings of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE), Abu Dhabi, United Arab Emirates."},{"key":"ref_21","first-page":"151","article-title":"Hybrid Approaches Based Emotion Detection in Memes Sentiment Analysis","volume":"14","author":"Prakash","year":"2021","journal-title":"Int. J. Eng. Res. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"137063","DOI":"10.1109\/ACCESS.2024.3436107","article-title":"Sarcasm Detection in Indonesian-English Code-Mixed Text Using Multihead Attention-Based Convolutional and Bi-Directional GRU","volume":"12","author":"Rosid","year":"2024","journal-title":"IEEE Access"},{"key":"ref_23","unstructured":"Suryawanshi, S., Chakravarthi, B.R., Arcan, M., and Buitelaar, P. (2020, January 11\u201316). Multimodal Meme Dataset (MultiOFF) for Identifying Offensive Content in Image and Text. Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, Marseille, France."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"102141","DOI":"10.1016\/j.ipm.2019.102141","article-title":"Hybrid context enriched deep learning model for fine-grained sentiment analysis in textual and visual semiotic modality social data","volume":"57","author":"Kumar","year":"2020","journal-title":"Inf. Process. Manag."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Elahi, K.T., Rahman, T.B., Shahriar, S., and Sarker, S. (2023, January 13\u201315). Explainable Multimodal Sentiment Analysis on Bengali Memes. Proceedings of the 26th International Conference on Computer and Information Technology (ICCIT), Cox\u2019s Bazar, Bangladesh.","DOI":"10.1109\/ICCIT60459.2023.10441342"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Arwoko, H., Yuniarno, E.M., and Purnomo, M.H. (2022, January 9\u201311). Hand Gesture Recognition Based on Keypoint Vector. Proceedings of the IES 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Surabaya, Indonesia.","DOI":"10.1109\/IES55876.2022.9888333"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1016\/j.procs.2023.01.054","article-title":"Face Detection and Recognition Using Face Mesh and Deep Neural Network","volume":"218","author":"Hangaragi","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Albadani, B., Shi, R., Dong, J., Al-Sabri, R., and Moctard, O.B. (2022). Transformer-Based Graph Convolutional Network for Sentiment Analysis. Appl. Sci., 12.","DOI":"10.3390\/app12031316"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Liang, Y., Han, D., He, Z., Kong, B., and Wen, S. (2025). SceEmoNet: A Sentiment Analysis Model with Scene Construction Capability. Appl. Sci., 15.","DOI":"10.3390\/app15158588"},{"key":"ref_30","first-page":"895","article-title":"Cross-Modal Sentiment Analysis Based on CLIP Image-Text Attention Interaction","volume":"15","author":"Lu","year":"2024","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"24103","DOI":"10.1007\/s11042-019-7390-1","article-title":"Sentiment analysis of multimodal twitter data","volume":"78","author":"Kumar","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Asmawati, E., Saikhu, A., and Siahaan, D. (2022, January 6\u20137). Sentiment Analysis of Text Memes: A Comparison among Supervised Machine Learning Methods. Proceedings of the 9th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), Jakarta, Indonesia.","DOI":"10.23919\/EECSI56542.2022.9946506"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Rosid, M.A., Siahaan, D., and Saikhu, A. (2023, January 25\u201327). Improving Sarcasm Detection in Mash-Up Language Through Hybrid Pretrained Word Embedding. Proceedings of the IEEE 8th International Conference on Software Engineering and Computer Systems (ICSECS), Penang, Malaysia.","DOI":"10.1109\/ICSECS58457.2023.10256422"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1080\/01605682.2020.1784049","article-title":"Textual analysis and corporate bankruptcy: A financial dictionary-based sentiment approach","volume":"73","author":"Nguyen","year":"2022","journal-title":"J. Oper. Res. Soc."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s13278-022-00900-0","article-title":"Design of a personalized recommender system using sentiment analysis in social media (case study: Banking system)","volume":"12","author":"Ghobakhloo","year":"2022","journal-title":"Soc. Netw. Anal. Min."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1080\/24721735.2024.2352979","article-title":"Sentiment analysis applied to tourism: Exploring tourist-generated content in the case of a wellness tourism destination","volume":"7","author":"George","year":"2024","journal-title":"Int. J. Spa Wellness"},{"key":"ref_37","first-page":"8405623","article-title":"Sentiment Analysis of Student Texts Using the CNN-BiGRU-AT Model","volume":"2021","author":"Yan","year":"2021","journal-title":"Sci. Program."},{"key":"ref_38","first-page":"466","article-title":"Sentiment Analysis on Customer Satisfaction of Digital Banking in Indonesia","volume":"13","author":"Andrian","year":"2022","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"111429","DOI":"10.1016\/j.knosys.2024.111429","article-title":"Object aroused emotion analysis network for image sentiment analysis","volume":"286","author":"Zhang","year":"2024","journal-title":"Knowl.-Based Syst."},{"key":"ref_40","unstructured":"Ekman, P. (2003). Membaca Emosi Orang, Think Jogjakata. Available online: https:\/\/dlibrary.ittelkom-pwt.ac.id\/index.php?p=show_detail&id=1896&keywords=."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"103183","DOI":"10.1016\/j.jvcir.2021.103183","article-title":"Facial micro-expression recognition based on accordion spatio-temporal representation and random forests","volume":"79","author":"Guermazi","year":"2021","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_42","unstructured":"Cruz, R.F., and Berrol, C. (2004). Issues of vailidity and reliability in the Use of Movement Observations and Scales. Dance\/Movement Therapists in Action: A Working Guide to Research Options, Charles, C. Thomas."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"012030","DOI":"10.1088\/1742-6596\/1755\/1\/012030","article-title":"An Analysis of Multiclass Imbalanced Data Problem in Machine Learning for Network Attack Detections","volume":"1755","author":"Soon","year":"2021","journal-title":"J. Phys. Conf. Ser."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/118\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T05:35:20Z","timestamp":1761802520000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/12\/4\/118"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,28]]},"references-count":43,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["informatics12040118"],"URL":"https:\/\/doi.org\/10.3390\/informatics12040118","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,28]]}}}