{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T11:39:09Z","timestamp":1774438749772,"version":"3.50.1"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T00:00:00Z","timestamp":1736294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>Marketing plays a vital role in the success of a business, driving customer engagement, brand recognition, and revenue growth. Neuromarketing adds depth to this by employing insights into consumer behavior through brain activity and emotional responses to create more effective marketing strategies. Electroencephalogram (EEG) has typically been utilized by researchers for neuromarketing, whereas Eye Tracking (ET) has remained unexplored. To address this gap, we propose a novel multimodal approach to predict consumer choices by integrating EEG and ET data. Noise from EEG signals is mitigated using a bandpass filter, Artifact Subspace Reconstruction (ASR), and Fast Orthogonal Regression for Classification and Estimation (FORCE). Class imbalance is handled by employing the Synthetic Minority Over-sampling Technique (SMOTE). Handcrafted features, including statistical and wavelet features, and automated features from Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM), have been extracted and concatenated to generate a feature space representation. For ET data, preprocessing involved interpolation, gaze plots, and SMOTE, followed by feature extraction using LeNet-5 and handcrafted features like fixations and saccades. Multimodal feature space representation was generated by performing feature-level fusion for EEG and ET, which was later fed into a meta-learner-based ensemble classifier with three base classifiers, including Random Forest, Extended Gradient Boosting, and Gradient Boosting, and Random Forest as the meta-classifier, to perform classification between buy vs. not buy. The performance of the proposed approach is evaluated using a variety of performance metrics, including accuracy, precision, recall, and F1 score. Our model demonstrated superior performance compared to competitors by achieving 84.01% accuracy in predicting consumer choices and 83% precision in identifying positive consumer preferences.<\/jats:p>","DOI":"10.3389\/fncom.2024.1516440","type":"journal-article","created":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T06:26:26Z","timestamp":1736317586000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Multimodal consumer choice prediction using EEG signals and eye tracking"],"prefix":"10.3389","volume":"18","author":[{"given":"Syed Muhammad","family":"Usman","sequence":"first","affiliation":[]},{"given":"Shehzad","family":"Khalid","sequence":"additional","affiliation":[]},{"given":"Aimen","family":"Tanveer","sequence":"additional","affiliation":[]},{"given":"Ali Shariq","family":"Imran","sequence":"additional","affiliation":[]},{"given":"Muhammad","family":"Zubair","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2025,1,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"2623","DOI":"10.1145\/3292500.3330701","article-title":"\u201cOptuna: a next-generation hyperparameter optimization framework,\u201d","author":"Akiba","year":"2019","journal-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery"},{"key":"B2","doi-asserted-by":"publisher","first-page":"604639","DOI":"10.3389\/fnhum.2020.604639","article-title":"Recognition of consumer preference by analysis and classification EEG signals","volume":"14","author":"Aldayel","year":"2021","journal-title":"Front. Hum. Neurosci"},{"key":"B3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3460881.3460930","article-title":"\u201cDeep learning for neuromarketing; classification of user preference using EEG signals,\u201d","volume-title":"12th Augmented Human International Conference","author":"Alimardani","year":"2021"},{"key":"B4","first-page":"227","article-title":"\u201cLeveraging wavelets and deep CNN for sleep pattern recognition in road safety: an EEG study,\u201d","volume-title":"International Conference on Recent Trends in Image Processing and Pattern Recognition","author":"Arif","year":"2023"},{"key":"B5","volume-title":"Principles of Marketing","author":"Armstrong","year":"2014"},{"key":"B6","volume-title":"Consumer behavior and marketing action fifth edition.","author":"Assel","year":"1995"},{"key":"B7","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn"},{"key":"B8","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","article-title":"Smote: synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res"},{"key":"B9","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1145\/2939672.2939785","article-title":"\u201cXgboost: a scalable tree boosting system,\u201d","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Chen","year":"2016"},{"key":"B10","doi-asserted-by":"publisher","first-page":"72","DOI":"10.3389\/fninf.2017.00072","article-title":"Your brain on the movies: a computational approach for predicting box-office performance from viewer's brain responses to movie trailers","volume":"11","author":"Christoforou","year":"2017","journal-title":"Front. Neuroinform"},{"key":"B11","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1080\/2326263X.2017.1297192","article-title":"Riemannian geometry for eeg-based brain-computer interfaces; a primer and a review","volume":"4","author":"Congedo","year":"2017","journal-title":"Brain-Comput. Interfaces"},{"key":"B12","volume-title":"Fisch and Spehlmann's EEG Primer: Basic Principles of Digital and Analog EEG","author":"Fisch","year":"1999"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: a gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat"},{"key":"B14","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1016\/j.physbeh.2018.04.030","article-title":"Do isolated packaging variables influence consumers' attention and preferences?","volume":"200","author":"Garc\u00eda-Madariaga","year":"2019","journal-title":"Physiol. Behav"},{"key":"B15","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1186\/s40708-022-00171-7","article-title":"Rneumark: a riemannian EEG analysis framework for neuromarketing","volume":"9","author":"Georgiadis","year":"2022","journal-title":"Brain Inf"},{"key":"B16","first-page":"21","article-title":"\u201cHarneshing the potential of EEG in neuromarketing with deep learning and riemannian geometry,\u201d","volume-title":"International Conference on Brain Informatics","author":"Georgiadis","year":""},{"key":"B17","doi-asserted-by":"publisher","first-page":"508","DOI":"10.1038\/s41597-023-02392-9","article-title":"Neuma-the absolute neuromarketing dataset en route to an holistic understanding of consumer behaviour","volume":"10","author":"Georgiadis","year":"","journal-title":"Sci. Data"},{"key":"B18","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1016\/j.ijresmar.2020.10.005","article-title":"Machines learn neuromarketing: improving preference prediction from self-reports using multiple EEG measures and machine learning","volume":"38","author":"Hakim","year":"2021","journal-title":"Int. J. Res. Mark"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"B20","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1016\/0013-4694(87)90206-9","article-title":"Cerebral location of international 10-20 system electrode placement","volume":"66","author":"Homan","year":"1987","journal-title":"Electroencephalogr. Clin. Neurophysiol"},{"key":"B21","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1109\/TBME.2019.2912066","article-title":"A Riemannian geometry approach to reduced and discriminative covariance estimation in brain computer interfaces","volume":"67","author":"Kalaganis","year":"2019","journal-title":"IEEE Trans. Biomed. Eng"},{"key":"B22","doi-asserted-by":"publisher","first-page":"3803","DOI":"10.1016\/j.eswa.2012.12.095","article-title":"Consumer neuroscience: assessing the brain response to marketing stimuli using electroencephalogram (EEG) and eye tracking","volume":"40","author":"Khushaba","year":"2013","journal-title":"Expert Syst. Appl"},{"key":"B23","first-page":"1137","article-title":"\u201cA study of cross-validation and bootstrap for accuracy estimation and model selection,\u201d","volume-title":"Proceedings of the 14th international joint conference on Artificial intelligence, Volume 2","author":"Kohavi","year":"1995"},{"key":"B24","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.inffus.2018.11.001","article-title":"Fusion of EEG response and sentiment analysis of products review to predict customer satisfaction","volume":"52","author":"Kumar","year":"2019","journal-title":"Inf. Fusion"},{"key":"B25","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1146","DOI":"10.1002\/cb.2253","article-title":"Intelligent neuromarketing framework for consumers' preference prediction from electroencephalography signals and eye tracking","volume":"23","author":"Mashrur","year":"2024","journal-title":"J. Consum. Behav"},{"key":"B27","doi-asserted-by":"publisher","first-page":"205","DOI":"10.2501\/JAR-2016-017","article-title":"Toward a better understanding of advertising stimuli processing: exploring the link between consumers' eye fixation and their subconscious responses","volume":"56","author":"Matukin","year":"2016","journal-title":"J. Advert. Res"},{"key":"B28","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/CSPA.2014.6805714","article-title":"\u201cWireless EEG signals based neuromarketing system using Fast Fourier transform (FFT),\u201d","volume-title":"2014 IEEE 10th international colloquium on signal processing and its applications","author":"Murugappan","year":"2014"},{"key":"B29","volume-title":"Consumer Behaviour and Marketing Strategy. Edisi IV","author":"Peter","year":"1999"},{"key":"B30","first-page":"1","article-title":"Left lateral singular (LLS) view enhances attention at stimulus discrimination during automobile display setting revealed by EEG\/ERP and eye tracking technique","volume":"5","author":"Samsuri","year":"2016","journal-title":"Int. J. Enhanc. Res. Sci. Technol. Eng"},{"key":"B31","doi-asserted-by":"publisher","first-page":"9744","DOI":"10.3390\/s22249744","article-title":"An ensemble model for consumer emotion prediction using EEG signals for neuromarketing applications","volume":"22","author":"Shah","year":"2022","journal-title":"Sensors"},{"key":"B32","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.inffus.2016.09.003","article-title":"Combining eye tracking, pupil dilation and EEG analysis for predicting web users click intention","volume":"35","author":"Slanzi","year":"2017","journal-title":"Inf. Fusion"},{"key":"B33","doi-asserted-by":"publisher","first-page":"511","DOI":"10.1509\/jmr.13.0564","article-title":"Using EEG to predict consumers' future choices","volume":"52","author":"Telpaz","year":"2015","journal-title":"J. Mark. Res"},{"key":"B34","first-page":"1","article-title":"Fundamentals of EEG measurement","volume":"2","author":"Teplan","year":"2002","journal-title":"Meas. Sci. Rev"},{"key":"B35","unstructured":"Tobii\n              A. B.\n            \n          \n          What is eye tracking\n          \n          2024"},{"key":"B36","doi-asserted-by":"publisher","first-page":"643489","DOI":"10.1155\/2011\/643489","article-title":"On the use of EEG or MEG brain imaging tools in neuromarketing research","volume":"2011","author":"Vecchiato","year":"2011","journal-title":"Comput. Intell. Neurosci"},{"key":"B37","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1109\/TAU.1967.1161901","article-title":"The use of Fast Fourier Transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms","volume":"15","author":"Welch","year":"1967","journal-title":"IEEE Trans. Audio Electroacoust"},{"key":"B38","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural. Netw"},{"key":"B39","doi-asserted-by":"publisher","first-page":"19087","DOI":"10.1007\/s11042-017-4580-6","article-title":"Analysis of EEG signals and its application to neuromarketing","volume":"76","author":"Yadava","year":"2017","journal-title":"Multimed. Tools Appl"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1516440\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T06:26:35Z","timestamp":1736317595000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1516440\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,8]]},"references-count":39,"alternative-id":["10.3389\/fncom.2024.1516440"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2024.1516440","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,8]]},"article-number":"1516440"}}