{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,2]],"date-time":"2026-02-02T09:57:13Z","timestamp":1770026233833,"version":"3.49.0"},"reference-count":30,"publisher":"SAGE Publications","issue":"5","license":[{"start":{"date-parts":[[2021,3,27]],"date-time":"2021-03-27T00:00:00Z","timestamp":1616803200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"published-print":{"date-parts":[[2021,11,17]]},"abstract":"<jats:p>Human communication is not limited to verbal speech but is infinitely more complex, involving many non-verbal cues such as facial emotions and body language. This paper aims to quantitatively show the impact of non-verbal cues, with primary focus on facial emotions, on the results of multi-modal sentiment analysis. The paper works with a dataset of Spanish video reviews. The audio is available as Spanish text and is translated to English while visual features are extracted from the videos. Multiple classification models are made to analyze the sentiments at each modal stage i.e. for the Spanish and English textual datasets as well as the datasets obtained upon coalescing the English and Spanish textual data with the corresponding visual cues. The results show that the analysis of Spanish textual features combined with the visual features outperforms its English counterpart with the highest accuracy difference, thereby indicating an inherent correlation between the Spanish visual cues and Spanish text which is lost upon translation to English text.<\/jats:p>","DOI":"10.3233\/jifs-189870","type":"journal-article","created":{"date-parts":[[2021,3,30]],"date-time":"2021-03-30T14:37:39Z","timestamp":1617115059000},"page":"5487-5496","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":3,"title":["Impact of cultural-shift on multimodal sentiment analysis"],"prefix":"10.1177","volume":"41","author":[{"given":"Tulika","family":"Banerjee","sequence":"first","affiliation":[{"name":"Department of Information and Communication Technology, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India"}]},{"given":"Niraj","family":"Yagnik","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Technology, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India"}]},{"given":"Anusha","family":"Hegde","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Technology, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, India"}]}],"member":"179","published-online":{"date-parts":[[2021,3,27]]},"reference":[{"issue":"11","key":"e_1_3_1_2_2","first-page":"91","article-title":"Language and Culture,Ver. 4","volume":"22","author":"Nabi A.","year":"2017","unstructured":"NabiA., Language and Culture,Ver. 4, IOSR Journal Of Humanities And Social Science (IOSR-JHSS)22(11) (2017), 91\u201394.","journal-title":"IOSR Journal Of Humanities And Social Science (IOSR-JHSS)"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-015-0015-2"},{"key":"e_1_3_1_4_2","unstructured":"Perez-RosasV. and MihalceaR. Utterance-Level Multimodal Sentiment Analysis 51st Annual Meeting of the Association for Computational Linguistics (2013) 973\u2013982."},{"key":"e_1_3_1_5_2","unstructured":"AmosB. LudwiczukB. and SatyanarayananM. Open Face: A general-purpose face recognition library with mobile applications CMU-CS-16-118 CMU School of Computer Science Tech. Rep. (2016)."},{"key":"e_1_3_1_6_2","unstructured":"LittlewortG. WhitehillJ. WuT. FaselI. FrankM. Movel lanJ. and BartlettM. The Computer Expression Recognition Toolbox. Ninth IEEE International Conference on Automatic Face and Gesture Recognition (FG 2011) Santa Barbara CA USA (2011) 21\u201325"},{"key":"e_1_3_1_7_2","unstructured":"ThomasB.D. Investigating Facial Mimicry Responses to Emotional Stimuli in Spanish-English Bilinguals Undergraduate Honors Theses (2014) 58."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2009.06.034"},{"key":"e_1_3_1_9_2","doi-asserted-by":"crossref","unstructured":"CambriaaE. HazarikaD. SoujanyaP. HussaindA. and SubramanyambR.B.V. Benchmarking Multimodal Sentiment Analysis. CICLing 2017: Computational Linguistics and Intelligent Text Processing (2017) 166\u2013179.","DOI":"10.1007\/978-3-319-77116-8_13"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2018.2882362"},{"key":"e_1_3_1_11_2","unstructured":"GuY. YangK. FuS. ChenS. LiX. and MarsicI. Hybrid Attention based Multimodal Network for Spoken Language Classification Proceedings of the 27th International Conference on Computational Linguistics (2018) 2379\u20132390."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00530-010-0182-0"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.11591\/ijece.v10i3.pp3307-3314"},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"XiC. LuG. and YanJ. Multimodal sentiment analysis based on multi-head attention mechanism In Proceedings of the 4th International Conference on Machine Learning and Soft Computing (ICMLSC 2020). Association for Computing Machinery New York NY USA (2020) 34\u201339.","DOI":"10.1145\/3380688.3380693"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","unstructured":"PoriaS. ChaturvediI. CambriaE. and HussainA. Convolutional MKL Based Multimodal Emotion Recognition and Sentiment Analysis 2016 IEEE 16th International Conference on Data Mining (ICDM) Barcelona (2016) 439\u2013448. doi:10.1109\/ICDM.2016.0055","DOI":"10.1109\/ICDM.2016.0055"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"AlmC. RothD. and SproatR. Emotions from text Machine learning for text-based emotion prediction In Proceedings of the Conference on Empirical Methods in Natural Language Processing (2005) 347\u2013354.","DOI":"10.3115\/1220575.1220648"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","unstructured":"ChenT. and GuestrinC. XGBoost: A Scalable Tree Boosting System. KDD \u201916: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) DOI:10.1145\/2939672.2939785","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.14569\/IJACSA.2019.0100742"},{"key":"e_1_3_1_20_2","unstructured":"PlissonJ. LavracN. and MladenicD. A Rule based Approach to Word Lemmatization Proceedings of IS04. (2004). DOI:10.1.1.646.9308"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.3390\/info10040150"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/5254.708428"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/34.908962"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","unstructured":"ZengX. ChenY.-W. TaoC. and AlphenD. Feature Selection Using Recursive Feature Elimination for Handwritten Digit Recognition IIH-MSP 2009 - 2009 5th International Conference on Intelligent Information Hiding and Multimedia Signal Processing. (2009) 1205\u20131208. 10.1109\/IIH-MSP.2009.145","DOI":"10.1109\/IIH-MSP.2009.145"},{"key":"e_1_3_1_25_2","first-page":"100179","article-title":"Recursive feature elimination in random forest classification supports nanomaterial grouping","volume":"15","author":"Bahl A.","year":"2018","unstructured":"BahlA., HellackB., BalasM., DinischiotuA., WiemannM., BrinkmannJ., LuchA., RenardB.Y. and HaaseA., Recursive feature elimination in random forest classification supports nanomaterial grouping, Nano Impact15 (2018), 100179.","journal-title":"Nano Impact"},{"key":"e_1_3_1_26_2","article-title":"Facial Emotion Recognition Using Machine Learning","volume":"632","author":"Raut N.","year":"2018","unstructured":"RautN., Facial Emotion Recognition Using Machine Learning, Master\u2019s Projects632 (2018). DOI: https:\/\/doi.org\/10.31979\/etd.w5fs-s8wd","journal-title":"Master\u2019s Projects"},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","unstructured":"MorencyL.P. MihalceaR. and DoshiP. Towards multimodal sentiment analysis: Harvesting opinions from the web In Proceedings of the International Conference on Multimodal Computing Alicante Spain. (2011).","DOI":"10.1145\/2070481.2070509"},{"key":"e_1_3_1_28_2","unstructured":"SierraS. and Gonz\u00e1lezF.A. Combining Textual and Visual Representations for Multimodal Author Profiling Notebook for PAN at CLEF (2018)."},{"key":"e_1_3_1_29_2","unstructured":"GardnerJ. and BrooksC. Statistical Approaches to the Model Comparison Task in Learning Analytics MLA\/BLAC@LAK. (2017)."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.187"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-24775-3_3"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-189870","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/JIFS-189870","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-189870","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,2]],"date-time":"2026-02-02T00:57:50Z","timestamp":1769993870000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/JIFS-189870"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,27]]},"references-count":30,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2021,11,17]]}},"alternative-id":["10.3233\/JIFS-189870"],"URL":"https:\/\/doi.org\/10.3233\/jifs-189870","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,27]]}}}