{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T02:17:55Z","timestamp":1788315475180,"version":"build-2803163510"},"reference-count":39,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2021,7,15]],"date-time":"2021-07-15T00:00:00Z","timestamp":1626307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Visual sentiment analysis has become more popular than textual ones in various domains for decision-making purposes. On account of this, we develop a visual sentiment analysis system, which can classify image expression. The system classifies images by taking into account six different expressions such as anger, joy, love, surprise, fear, and sadness. In our study, we propose an expert system by integrating a Deep Learning method with a Belief Rule Base (known as the BRB-DL approach) to assess an image\u2019s overall sentiment under uncertainty. This BRB-DL approach includes both the data-driven and knowledge-driven techniques to determine the overall sentiment. Our integrated expert system outperforms the state-of-the-art methods of visual sentiment analysis with promising results. The integrated system can classify images with 86% accuracy. The system can be beneficial to understand the emotional tendency and psychological state of an individual.<\/jats:p>","DOI":"10.3390\/a14070213","type":"journal-article","created":{"date-parts":[[2021,7,15]],"date-time":"2021-07-15T02:52:34Z","timestamp":1626317554000},"page":"213","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["An Integrated Deep Learning and Belief Rule-Based Expert System for Visual Sentiment Analysis under Uncertainty"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4568-359X","authenticated-orcid":false,"given":"Sharif Noor","family":"Zisad","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, University of Chittagong, Chittagong 4331, Bangladesh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4886-2585","authenticated-orcid":false,"given":"Etu","family":"Chowdhury","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Chittagong, Chittagong 4331, Bangladesh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7473-8185","authenticated-orcid":false,"given":"Mohammad Shahadat","family":"Hossain","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Chittagong, Chittagong 4331, Bangladesh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3090-7645","authenticated-orcid":false,"given":"Raihan Ul","family":"Islam","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical and Space Engineering, Lule\u00e5 University of Technology, 971 87 Skellefte\u00e5, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0244-3561","authenticated-orcid":false,"given":"Karl","family":"Andersson","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical and Space Engineering, Lule\u00e5 University of Technology, 971 87 Skellefte\u00e5, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zhao, J., Liu, K., and Xu, L. (2016). Sentiment Analysis: Mining Opinions, Sentiments, and Emotions, Cambridge University Press.","DOI":"10.1162\/COLI_r_00259"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1109\/TAFFC.2014.2384198","article-title":"Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads","volume":"6","author":"McDuff","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_3","unstructured":"Xu, C., Cetintas, S., chih Lee, K., and Li, L.J. (2014). Visual Sentiment Prediction with Deep Convolutional Neural Networks. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Islam, J., and Zhang, Y. (2016, January 8\u201310). Visual sentiment analysis for social images using transfer learning approach. Proceedings of the 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), Atlanta, GA, USA.","DOI":"10.1109\/BDCloud-SocialCom-SustainCom.2016.29"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"You, Q., Luo, J., Jin, H., and Yang, J. (2015, January 25\u201330). Robust image sentiment analysis using progressively trained and domain transferred deep networks. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9179"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.imavis.2017.01.011","article-title":"From pixels to sentiment: Fine-tuning CNNs for visual sentiment prediction","volume":"65","author":"Campos","year":"2017","journal-title":"Image Vis. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Rahaman, S., and Hossain, M.S. (2014, January 8\u201310). A belief rule based (BRB) system to assess asthma suspicion. Proceedings of the 16th Int\u2019l Conf. Computer and Information Technology, Khulna, Bangladesh.","DOI":"10.1109\/ICCITechn.2014.6997340"},{"key":"ref_8","unstructured":"Hossain, M.S., Hossain, E., Khalid, M.S., and Haque, M.A. (2014, January 21\u201322). A belief rule-based (BRB) decision support system for assessing clinical asthma suspicion. Proceedings of the Scandinavian Conference on Health Informatics, Grimstad, Norway."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kabir, S., Islam, R.U., Hossain, M.S., and Andersson, K. (2020). An integrated approach of belief rule base and deep learning to predict air pollution. Sensors, 20.","DOI":"10.3390\/s20071956"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Uddin Ahmed, T., Jamil, M.N., Hossain, M.S., Andersson, K., and Hossain, M.S. (2020, January 26\u201329). An integrated real-time deep learning and belief rule base intelligent system to assess facial expression under uncertainty. Proceedings of the 9th International Conference on Informatics, Electronics & Vision (ICIEV), Kitakyushu, Japan.","DOI":"10.1109\/ICIEVicIVPR48672.2020.9306622"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zisad, S.N., Hossain, M.S., Hossain, M.S., and Andersson, K. (2021). An Integrated Neural Network and SEIR Model to Predict COVID-19. Algorithms, 14.","DOI":"10.3390\/a14030094"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.future.2018.06.021","article-title":"A novel fuzzy deep-learning approach to traffic flow prediction with uncertain spatial\u2013temporal data features","volume":"89","author":"Chen","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"190637","DOI":"10.1109\/ACCESS.2020.3031438","article-title":"A Deep Learning Inspired Belief Rule-Based Expert System","volume":"8","author":"Islam","year":"2020","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.knosys.2018.07.029","article-title":"Akaike Information Criterion-based conjunctive belief rule base learning for complex system modeling","volume":"161","author":"Chang","year":"2018","journal-title":"Knowl. Based Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1866","DOI":"10.1109\/TFUZZ.2019.2892348","article-title":"Generic Disjunctive Belief-Rule-Base Modeling, Inferencing, and Optimization","volume":"27","author":"Chang","year":"2019","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Siersdorfer, S., Minack, E., Deng, F., and Hare, J. (2010, January 25\u201329). Analyzing and predicting sentiment of images on the social web. Proceedings of the 18th ACM International Conference on Multimedia, Firenze, Italy.","DOI":"10.1145\/1873951.1874060"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Machajdik, J., and Hanbury, A. (2010, January 25\u201329). Affective image classification using features inspired by psychology and art theory. Proceedings of the 18th ACM International Conference on Multimedia, Firenze, Italy.","DOI":"10.1145\/1873951.1873965"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Borth, D., Ji, R., Chen, T., Breuel, T., and Chang, S.F. (2013, January 21\u201325). Large-scale visual sentiment ontology and detectors using adjective noun pairs. Proceedings of the 21st ACM International Conference on Multimedia, Barcelona, Spain.","DOI":"10.1145\/2502081.2502282"},{"key":"ref_19","unstructured":"Chen, T., Borth, D., Darrell, T., and Chang, S.F. (2014). Deepsentibank: Visual sentiment concept classification with deep convolutional neural networks. arXiv."},{"key":"ref_20","unstructured":"Gajarla, V., and Gupta, A. (2015). Emotion Detection and Sentiment Analysis of Images, Georgia Institute of Technology."},{"key":"ref_21","unstructured":"Wang, J., Fu, J., Xu, Y., and Mei, T. (2016, January 9\u201315). Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks. Proceedings of the 25th International Joint Conference on Artificial Intelligence IJCAI-16, New York, NY, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Vadicamo, L., Carrara, F., Cimino, A., Cresci, S., Dell\u2019Orletta, F., Falchi, F., and Tesconi, M. (2017, January 22\u201329). Cross-media learning for image sentiment analysis in the wild. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy.","DOI":"10.1109\/ICCVW.2017.45"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Feng-jiao, W., and Aono, M. (2018, January 14\u201317). Visual Sentiment Prediction by Merging Hand-Craft and CNN Features. Proceedings of the 2018 5th International Conference on Advanced Informatics: Concept Theory and Applications (ICAICTA), Krabi, Thailand.","DOI":"10.1109\/ICAICTA.2018.8541312"},{"key":"ref_24","unstructured":"Tan, W., Wang, J., Wang, Y., Lewis, M., and Jarrold, W. (2018). CNN Models for Classifying Emotions Evoked by Paintings, SVL Lab, Stanford University."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"871","DOI":"10.5194\/isprs-archives-XLII-2-W15-871-2019","article-title":"Deep Convolutional Neural Networks for Sentiment Analysis of Cultural Heritage","volume":"XLII-2\/W15","author":"Paolanti","year":"2019","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chowdhury, E., Chy, A.N., and Chowdhury, N.I. (2020, January 25\u201327). Exploiting Transfer Learning Ensemble for Visual Sentiment Analysis in Social Media. Proceedings of International Conference on Computational Intelligence, Data Science and Cloud Computing: IEM-ICDC 2020, Kolkata, India.","DOI":"10.1007\/978-981-33-4968-1_3"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zisad, S.N., Hossain, M.S., and Andersson, K. (2020, January 19). Speech emotion recognition in neurological disorders using convolutional neural network. Proceedings of the International Conference on Brain Informatics, Padua, Italy.","DOI":"10.1007\/978-3-030-59277-6_26"},{"key":"ref_28","unstructured":"Yu, F., Qin, Z., and Chen, X. (2018). Distilling critical paths in convolutional neural networks. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.ins.2015.12.009","article-title":"Belief rule based expert system for classification problems with new rule activation and weight calculation procedures","volume":"336","author":"Chang","year":"2016","journal-title":"Inf. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1225","DOI":"10.1109\/TSMCA.2011.2147312","article-title":"Online updating belief-rule-base using the RIMER approach","volume":"41","author":"Zhou","year":"2011","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Feng, Z., Hu, C., Zhao, F., Zhang, Y., and Hu, G. (2017, January 19\u201321). Fault detection based on belief rule base with online updating attribute weight. Proceedings of the 2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC), Hefei, China.","DOI":"10.1109\/YAC.2017.7967418"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"12845","DOI":"10.1016\/j.eswa.2011.04.077","article-title":"Inference analysis and adaptive training for belief rule based systems","volume":"38","author":"Chen","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.knosys.2019.02.007","article-title":"Online updating belief-rule-base using Bayesian estimation","volume":"171","author":"Tang","year":"2019","journal-title":"Knowl. Based Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1080\/00207543.2011.652262","article-title":"A belief-rule-based inference method for aggregate production planning under uncertainty","volume":"51","author":"Li","year":"2013","journal-title":"Int. J. Prod. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1109\/TSUSC.2017.2697768","article-title":"A belief rule based expert system for datacenter pue prediction under uncertainty","volume":"2","author":"Hossain","year":"2017","journal-title":"IEEE Trans. Sustain. Comput."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1109\/TSMCA.2005.851270","article-title":"Belief rule-base inference methodology using the evidential reasoning approach-RIMER","volume":"36","author":"Yang","year":"2006","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"e453","DOI":"10.7717\/peerj.453","article-title":"Scikit-image: Image processing in Python","volume":"2","author":"Boulogne","year":"2014","journal-title":"PeerJ"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Panda, R., Zhang, J., Li, H., Lee, J.Y., Lu, X., and Roy-Chowdhury, A.K. (2018, January 8\u201314). Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias. Proceedings of the European Conference on Computer Vision, Munich, Germany.","DOI":"10.1007\/978-3-030-01216-8_36"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Islam, R.U., Ruci, X., Hossain, M.S., Andersson, K., and Kor, A.L. (2019). Capacity management of hyperscale data centers using predictive modelling. Energies, 12.","DOI":"10.3390\/en12183438"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/7\/213\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:30:46Z","timestamp":1760164246000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/7\/213"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,15]]},"references-count":39,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["a14070213"],"URL":"https:\/\/doi.org\/10.3390\/a14070213","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,15]]}}}