{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T15:11:39Z","timestamp":1770995499523,"version":"3.50.1"},"reference-count":55,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,3,24]],"date-time":"2025-03-24T00:00:00Z","timestamp":1742774400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["71874126"],"award-info":[{"award-number":["71874126"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JTAER"],"abstract":"<jats:p>The increasing popularity of short videos has presented sellers with fresh opportunities to craft video advertisements that incorporate diverse modal information, with each modality potentially having a different influence on consumer engagement. Understanding which information is most important in attracting consumers can provide theoretical support to researchers. However, the dimensionality of the multimodal features of short video advertisements is often higher than the available data, posing specific difficulties in data analysis. Therefore, designing a multimodal analysis framework is needed to comprehensively extract and reduce the dimensionality of the different modal features of short video advertisements, thus analyzing which modal features are more important for consumer engagement. In this study, we chose TikTok as the research subject, and employed deep learning and machine learning techniques to extract features from short video advertisements, encompassing visual, acoustic, title, and speech text features. Subsequently, we introduced a method based on mixed-regularization sparse representation to select variables. Ultimately, we utilized multiblock partial least squares regression to regress the selected variables alongside additional scalar variables to calculate the block importance. The empirical analysis results indicate that visual and speech text features are the key factors influencing consumer engagement, providing theoretical support for subsequent research and offering practical insights for marketers.<\/jats:p>","DOI":"10.3390\/jtaer20020054","type":"journal-article","created":{"date-parts":[[2025,3,24]],"date-time":"2025-03-24T13:48:20Z","timestamp":1742824100000},"page":"54","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Most Significant Impact on Consumer Engagement: An Analytical Framework for the Multimodal Content of Short Video Advertisements"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9262-634X","authenticated-orcid":false,"given":"Zhipeng","family":"Zhang","sequence":"first","affiliation":[{"name":"Zhangjiagang Campus, Jiangsu University of Science and Technology, Zhangjiagang 215600, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8634-9227","authenticated-orcid":false,"given":"Liyi","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Information and Management, Wuhan University, Wuhan 430072, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1956","DOI":"10.1108\/IMDS-12-2021-0754","article-title":"The effect of advertising strategies on a short video platform: Evidence from TikTok","volume":"122","author":"Yuan","year":"2022","journal-title":"Ind. Manag. Data Syst."},{"key":"ref_2","unstructured":"QuestMobile (2024, July 01). Internet Advertising Semi-Annual Report in 2021. Available online: https:\/\/www.questmobile.com.cn\/research\/report-new\/169."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.3390\/jtaer17030054","article-title":"Why advertise on short video platforms? optimizing online advertising using advertisement quality","volume":"17","author":"Li","year":"2022","journal-title":"J. Theor. Appl. Electron. Commer. Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"105980","DOI":"10.1016\/j.chb.2019.04.004","article-title":"Computers in human behavior analyzing the effect of social support and community factors on customer engagement and its impact on loyalty behaviors toward social commerce websites","volume":"108","author":"Molinillo","year":"2019","journal-title":"Comput. Hum. Behav."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1007\/s11747-016-0485-6","article-title":"Customer engagement: The construct, antecedents, and consequences","volume":"45","author":"Pansari","year":"2017","journal-title":"J. Acad. Market. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"103121","DOI":"10.1016\/j.jretconser.2022.103121","article-title":"Effect of personal branding stereotypes on user engagement on short video platforms","volume":"69","author":"Wei","year":"2022","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"104136","DOI":"10.1016\/j.jretconser.2024.104136","article-title":"The impact of multimodal information features of short sales videos on consumer engagement behavior: A multi-method approach","volume":"82","author":"Xiao","year":"2025","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"7085","DOI":"10.1007\/s00521-024-09447-x","article-title":"Online content-based sequential recommendation considering multimodal contrastive representation and dynamic preferences","volume":"36","author":"Lu","year":"2024","journal-title":"Neural Comput. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"119452","DOI":"10.1016\/j.eswa.2023.119542","article-title":"What matters for short videos\u2019 user engagement: A multiblock model with variable screening","volume":"218","author":"Lu","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_11","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., and Zettlemoyer, L. (2018). Deep contextualized word representations. arXiv.","DOI":"10.18653\/v1\/N18-1202"},{"key":"ref_13","unstructured":"Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1002\/(SICI)1099-128X(199809\/10)12:5<301::AID-CEM515>3.0.CO;2-S","article-title":"Analysis of multiblock and hierarchical PCA and PLS models","volume":"12","author":"Westerhuis","year":"1998","journal-title":"J. Chemometr."},{"key":"ref_15","first-page":"2292759","article-title":"TikTok shop: How trust and privacy influence generation Z\u2019s purchasing behaviors","volume":"10","author":"Putri","year":"2024","journal-title":"Cogent Soc. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103170","DOI":"10.1016\/j.jretconser.2022.103170","article-title":"Exploring the factors influencing consumer engagement behavior regarding short-form video advertising: A big data perspective","volume":"70","author":"Xiao","year":"2023","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Jiang, W., and Chen, H. (2025). Can short videos work? The effects of use and gratification and social presence on purchase intention: Examining the mediating role of digital dependency. J. Theor. Appl. Electron. Commer. Res., 20.","DOI":"10.20944\/preprints202411.0736.v1"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhang, Z., and Zhang, L. (2025). How do online celebrities attract consumers? An EEG study on consumers\u2019 neural engagement in short video advertising. Electron. Commer. Res., 1\u201327.","DOI":"10.1007\/s10660-025-09956-4"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"101379","DOI":"10.1016\/j.elerap.2024.101379","article-title":"Boosting online user engagement with short video endorsement content on TikTok via the image transfer mechanism","volume":"64","author":"Haq","year":"2024","journal-title":"Electron. Commer. Res. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2319","DOI":"10.1108\/IMDS-08-2022-0510","article-title":"How doctor image features engage health science short video viewers? Investigating the age and gender bias","volume":"123","author":"Tan","year":"2023","journal-title":"Ind. Manag. Data Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e21463","DOI":"10.2196\/21463","article-title":"Factors driving citizen engagement with government TikTok accounts during the covid-19 pandemic: Model development and analysis","volume":"23","author":"Chen","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"40999","DOI":"10.1109\/ACCESS.2022.3167687","article-title":"Driving factors and moderating effects behind citizen engagement with mobile short-form videos","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1104","DOI":"10.1108\/INTR-02-2022-0121","article-title":"Short video marketing: What, when and how short-branded videos facilitate consumer engagement","volume":"34","author":"Dong","year":"2024","journal-title":"Internet Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1177\/1094670510375599","article-title":"Customer engagement behavior: Theoretical foundations and research directions","volume":"13","author":"Van","year":"2010","journal-title":"J. Serv. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/j.jretconser.2019.05.018","article-title":"Customer engagement in the service context: An empirical investigation of the construct, its antecedents and consequences","volume":"50","author":"Islam","year":"2019","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"113466","DOI":"10.1016\/j.dss.2020.113466","article-title":"Promoting or attenuating? An eye-tracking study on the role of social cues in e-commerce livestreaming","volume":"142","author":"Fei","year":"2021","journal-title":"Decis. Support Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1177\/1094670514529187","article-title":"The role of customer engagement behavior in value co-creation: A service system perspective","volume":"17","author":"Jaakkola","year":"2014","journal-title":"J. Serv. Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1002\/mar.21341","article-title":"The impact of pronoun choices on consumer engagement actions: Exploring top global brands\u2019 social media communications","volume":"37","author":"Labrecque","year":"2020","journal-title":"Psychol. Market."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Fu, Z., Wang, K., Wang, J., and Zhu, Y. (2024, January 3\u20137). Predicting sales lift of influencer-generated short video advertisements: A ladder attention-based multimodal time series forecasting framework. Proceedings of the Hawaii International Conference on System Sciences 2024, Honolulu, HI, USA.","DOI":"10.24251\/HICSS.2024.343"},{"key":"ref_30","first-page":"17","article-title":"Analyzing and predicting consumer response to short videos in e-commerce","volume":"15","author":"Guo","year":"2024","journal-title":"ACM Trans. Manag. Inf. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1287\/mksc.2021.0107","article-title":"Engagement that sells: Influencer video advertising on TikTok","volume":"44","author":"Yang","year":"2024","journal-title":"Mark. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"104005","DOI":"10.1016\/j.jretconser.2024.104005","article-title":"The effectiveness of human vs. AI voice-over in short video advertisements: A cognitive load theory perspective","volume":"81","author":"Wang","year":"2024","journal-title":"J. Retail. Consum. Serv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1108\/IMDS-06-2016-0207","article-title":"Detecting sarcasm in customer tweets: An NLP based approach","volume":"117","author":"Mukherjee","year":"2017","journal-title":"Ind. Manag. Data Syst."},{"key":"ref_34","first-page":"512","article-title":"A sparse feature extraction method based on improved quantum evolutionary algorith","volume":"40","author":"Yu","year":"2020","journal-title":"Trans. Beijing Inst. Technol."},{"key":"ref_35","first-page":"582","article-title":"Image edge feature extraction research based on quantum kernel clustering algorithm","volume":"37","author":"Tian","year":"2016","journal-title":"Acta Metrol. Sin."},{"key":"ref_36","unstructured":"Sun, Y., Wang, S., Li, Y., Feng, S., and Wu, H. (2019). ERNIE: Enhanced Representation through Knowledge Integration. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hsieh, Y.H., and Zeng, X. (2022). Sentiment analysis: An ERNIE-BiLSTM approach to bullet screen comments. Sensors, 22.","DOI":"10.3390\/s22145223"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hu, A., and Flaxman, S.R. (2018, January 19\u201323). Multimodal sentiment analysis to explore the structure of emotions. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3219853"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Gallo, I., Calefati, A., Nawaz, S., and Janjua, M.K. (2018, January 10\u201313). Image and encoded text fusion for multimodal classification. Proceedings of the 2018 IEEE Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia.","DOI":"10.1109\/DICTA.2018.8615789"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"980","DOI":"10.1002\/asi.23218","article-title":"Predicting users\u2019 domain knowledge in information retrieval using multiple regression analysis of search behaviors","volume":"66","author":"Zhang","year":"2015","journal-title":"J. Assoc. Inf. Sci. Technol."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhao, Z., and Liu, H. (2007, January 20\u201324). Spectral feature selection for supervised and unsupervised learning. Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, USA.","DOI":"10.1145\/1273496.1273641"},{"key":"ref_42","unstructured":"Li, Z., Yang, Y., Liu, J., Zhou, X., and Lu, H. (2012, January 22\u201326). Unsupervised feature selection using non-negative spectral analysis. Proceedings of the AAAI Conference on Artificial Intelligence, Toronto, ON, Canada."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Shi, L., Du, L., and Shen, Y. (2014, January 14\u201317). Robust spectral learning for unsupervised feature selection. Proceedings of the 2014 IEEE International Conference on Data Mining, Shenzhen, China.","DOI":"10.1109\/ICDM.2014.58"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2889","DOI":"10.1007\/s10489-019-01420-9","article-title":"A filter-based bare-bone particle swarm optimization algorithm for unsupervised feature selection","volume":"49","author":"Zhang","year":"2019","journal-title":"Appl. Intell."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1086\/259630","article-title":"Information and consumer behavior","volume":"72","author":"Nelson","year":"1970","journal-title":"J. Polit. Econ."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/j.intmar.2013.04.004","article-title":"Too popular to ignore: The influence of online reviews on purchase intentions of search and experience products","volume":"27","author":"Jimenez","year":"2013","journal-title":"J. Interact. Mark."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Ouzir, M., Lamrani, H.C., Bradley, R.L., and El-Moudden, I. (2024). Neuromarketing and decision-making: Classification of consumer preferences based on changes analysis in the EEG signal of brain regions. Biomed. Signal Process. Control, 87.","DOI":"10.1016\/j.bspc.2023.105469"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s10107-015-0892-3","article-title":"Coordinate descent algorithms","volume":"151","author":"Wright","year":"2015","journal-title":"Math. Program."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"11","DOI":"10.15388\/Informatica.2011.311","article-title":"From multiblock partial least squares to multiblock redundancy analysis, a continuum approach","volume":"22","author":"Bougeard","year":"2011","journal-title":"Informatica"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"103064","DOI":"10.1016\/j.nicl.2022.103064","article-title":"Cortico-cognition coupling in treatment resistant schizophrenia","volume":"35","author":"Syeda","year":"2022","journal-title":"NeuroImage Clin."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Strani, L., Vitale, R., Tanzilli, D., Bonacini, F., Perolo, A., Mantovani, E., Ferrando, A., and Cocchi, M. (2022). A Multiblock approach to fuse process and near-infrared sensors for online prediction of polymer properties. Sensors, 22.","DOI":"10.3390\/s22041436"},{"key":"ref_52","first-page":"593","article-title":"Narrative or non-narrative? The effects of short video content structure on mental simulation and resort brand attitude","volume":"32","author":"Wang","year":"2023","journal-title":"J. Hosp. Market. Manag."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1108\/LHT-09-2021-0318","article-title":"The short video usage motivation and behavior of middle-aged and old users","volume":"42","author":"Yu","year":"2024","journal-title":"Libr. Hi Tech."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1016\/j.jbusres.2015.07.001","article-title":"Big data consumer analytics and the transformation of marketing","volume":"69","author":"Erevelles","year":"2016","journal-title":"J. Bus. Res."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/j.ijresmar.2023.08.007","article-title":"Standing out from the crowd: When and why color complexity in social media images increases user engagement","volume":"41","author":"Vamsi","year":"2024","journal-title":"Int. J. Res. Mark."}],"container-title":["Journal of Theoretical and Applied Electronic Commerce Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/0718-1876\/20\/2\/54\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:59:08Z","timestamp":1760029148000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/0718-1876\/20\/2\/54"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,24]]},"references-count":55,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["jtaer20020054"],"URL":"https:\/\/doi.org\/10.3390\/jtaer20020054","relation":{},"ISSN":["0718-1876"],"issn-type":[{"value":"0718-1876","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,24]]}}}