{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T01:34:55Z","timestamp":1778808895759,"version":"3.51.4"},"reference-count":36,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2023,5,28]],"date-time":"2023-05-28T00:00:00Z","timestamp":1685232000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Plan for Science, Technology and Innovation (MAARIFAH), King Abdulaziz City for Science and Technology","award":["14-INF3139-02"],"award-info":[{"award-number":["14-INF3139-02"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>It is becoming increasingly attractive to detect human emotions using electroencephalography (EEG) brain signals. EEG is a reliable and cost-effective technology used to measure brain activities. This paper proposes an original framework for usability testing based on emotion detection using EEG signals, which can significantly affect software production and user satisfaction. This approach can provide an in-depth understanding of user satisfaction accurately and precisely, making it a valuable tool in software development. The proposed framework includes a recurrent neural network algorithm as a classifier, a feature extraction algorithm based on event-related desynchronization and event-related synchronization analysis, and a new method for selecting EEG sources adaptively for emotion recognition. The framework results are promising, achieving 92.13%, 92.67%, and 92.24% for the valence\u2013arousal\u2013dominance dimensions, respectively.<\/jats:p>","DOI":"10.3390\/s23115147","type":"journal-article","created":{"date-parts":[[2023,5,28]],"date-time":"2023-05-28T15:29:52Z","timestamp":1685287792000},"page":"5147","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Software Usability Testing Using EEG-Based Emotion Detection and Deep Learning"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6026-4257","authenticated-orcid":false,"given":"Sofien","family":"Gannouni","sequence":"first","affiliation":[{"name":"Department of Computer Science, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1455-439X","authenticated-orcid":false,"given":"Kais","family":"Belwafi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"C2PS Center, Electrical Engineering and Computer Science Department, Khalifa University, Abu Dhabi P.O. Box 127788, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arwa","family":"Aledaily","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8000-5105","authenticated-orcid":false,"given":"Hatim","family":"Aboalsamh","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4937-7453","authenticated-orcid":false,"given":"Abdelfettah","family":"Belghith","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,28]]},"reference":[{"key":"ref_1","unstructured":"Dumas, J.S., Dumas, J.S., and Redish, J. (1999). A Practical Guide to Usability Testing, Intellect Books."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jokela, T., Iivari, N., Matero, J., and Karukka, M. (2003, January 17\u201320). The standard of user-centered design and the standard definition of usability: Analyzing ISO 13407 against ISO 9241-11. Proceedings of the Latin American Conference on Human-Computer Interaction, Rio de Janeiro, Brazil.","DOI":"10.1145\/944519.944525"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1080\/00207590701396674","article-title":"Usability, aesthetics and emotions in human\u2013technology interaction","volume":"42","author":"Thuering","year":"2007","journal-title":"Int. J. Psychol."},{"key":"ref_4","unstructured":"Garrett, J.J. (2010). Elements of User Experience, the: User-Centered Design for the Web and Beyond, Pearson Education."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1515\/icom-2020-0009","article-title":"Investigating the Relationship Between Emotion Recognition Software and Usability Metrics","volume":"19","author":"Schmidt","year":"2020","journal-title":"i-com"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Landowska, A., and Miler, J. (2016, January 11\u201314). Limitations of emotion recognition in software user experience evaluation context. Proceedings of the 2016 Federated Conference on Computer Science and Information Systems (FedCSIS), Gda\u0144sk, Poland.","DOI":"10.15439\/2016F535"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1080\/10447318.2013.858460","article-title":"Emotional dimensions of user experience: A user psychological analysis","volume":"30","author":"Saariluoma","year":"2014","journal-title":"Int. J. Hum.-Comput. Interact."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"111","DOI":"10.3991\/ijoe.v15i07.10447","article-title":"A Proposed Web Based Real Time Brain Computer Interface (BCI) System for Usability Testing","volume":"15","author":"Puwakpitiyage","year":"2019","journal-title":"Int. J. Online Eng."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"do Amaral, V., Ferreira, L.A., Aquino, P.T., and de Castro, M.C.F. (2013, January 18\u201320). EEG signal classification in usability experiments. Proceedings of the 2013 ISSNIP Biosignals and Biorobotics Conference: Biosignals and Robotics for Better and Safer Living (BRC), Rio de Janeiro, Brazil.","DOI":"10.1109\/BRC.2013.6487469"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Stefancova, E., Moro, R., and Bielikova, M. (2018, January 2\u20135). Towards Detection of Usability Issues by Measuring Emotions. Proceedings of the European Conference on Advances in Databases and Information Systems, Budapest, Hungary.","DOI":"10.1007\/978-3-030-00063-9_8"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"58250","DOI":"10.1109\/ACCESS.2020.2981418","article-title":"Emotional Testing on Facebook\u2019s User Experience","volume":"8","author":"Mangion","year":"2020","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Aledaily, A., Gannoun, S., Aboalsamh, H., and Belwafi, K. (2022, January 22\u201324). A Framework for Usability Testing using EEG Signals with Emotion Recognition. Proceedings of the 5th International Conference on Intelligent Human Systems Integration (IHSI 2022) Integrating People and Intelligent Systems, Venice, Italy.","DOI":"10.54941\/ahfe1001049"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/S1071-5819(03)00019-3","article-title":"Physiological responses to different WEB page designs","volume":"59","author":"Ward","year":"2003","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_14","unstructured":"GreenTek Company (2023, April 29). GT Cap Gelfree-S3, 32\/16 EEG Channels Cap, Wuhan, China. Available online: https:\/\/www.greenteksensor.com\/products\/eeg-caps\/gt-cap-gelfree-s3\/."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Belwafi, K., Djemal, R., Ghaffari, F., Romain, O., Ouni, B., and Gannouni, S. (2016, January 21\u201323). Online Adaptive Filters to Classify Left and Right Hand Motor Imagery. Proceedings of the 9th International Joint Conference on Biomedical Engineering Systems and Technologies, Rome, Italy.","DOI":"10.5220\/0005846503350339"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"55","DOI":"10.4018\/IJSWIS.2017040104","article-title":"BCWB","volume":"13","author":"Gannouni","year":"2017","journal-title":"Int. J. Semant. Web Inf. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"Deap: A database for emotion analysis; using physiological signals","volume":"3","author":"Koelstra","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ramirez, R., and Vamvakousis, Z. (2012, January 4\u20137). Detecting emotion from EEG signals using the emotive epoc device. Proceedings of the International Conference on Brain Informatics, Macau, China.","DOI":"10.1007\/978-3-642-35139-6_17"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Gunawan, F.E., Wanandi, K., Soewito, B., Candra, S., and Sekishita, N. (2017, January 19\u201321). Detecting the early drop of attention using EEG signal. Proceedings of the 2017 4th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), Yogyakarta, Indonesia.","DOI":"10.1109\/EECSI.2017.8239175"},{"key":"ref_20","unstructured":"Arnaud Delorme, S.M. (2023, April 15). EEGLAB Wiki. Swartz Center for Computational Neuroscience. Available online: https:\/\/eeglab.org\/."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1016\/j.jad.2022.09.054","article-title":"Electroencephalography based emotion detection using ensemble classification and asymmetric brain activity","volume":"319","author":"Gannouni","year":"2022","journal-title":"J. Affect. Disord."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1016\/S0042-6989(00)00235-2","article-title":"Functional brain imaging based on ERD\/ERS","volume":"41","author":"Pfurtscheller","year":"2001","journal-title":"Vis. Res."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"7071","DOI":"10.1038\/s41598-021-86345-5","article-title":"Emotion detection using electroencephalography signals and a zero-time windowing-based epoch estimation and relevant electrode identification","volume":"11","author":"Gannouni","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_24","unstructured":"Gavin, H. (2011). The Levenberg-Marquardt Method for Nonlinear Least Squares Curve-Fitting Problems, Department of Civil and Environmental Engineering, Duke University."},{"key":"ref_25","unstructured":"Atabay, D. (2023, April 15). pyrenn: A Recurrent Neural Network Toolbox for Python and Matlab. Available online: https:\/\/pyrenn.readthedocs.io\/en\/latest\/."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Belwafi, K., Gannouni, S., and Aboalsamh, H. (2021). Embedded Brain Computer Interface: State-of-the-Art in Research. Sensors, 21.","DOI":"10.3390\/s21134293"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/s10044-016-0567-6","article-title":"Emotion recognition from EEG signals by using multivariate empirical mode decomposition","volume":"21","author":"Mert","year":"2018","journal-title":"Pattern Anal. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.eswa.2017.09.062","article-title":"Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors","volume":"93","author":"Nakisa","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_29","first-page":"355","article-title":"Emotion recognition based on EEG using LSTM recurrent neural network","volume":"8","author":"Alhagry","year":"2017","journal-title":"Emotion"},{"key":"ref_30","first-page":"1730","article-title":"Subject independent emotion recognition from EEG using VMD and deep learning","volume":"34","author":"Pandey","year":"2022","journal-title":"J. King Saud-Univ.-Comput. Inf. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"101867","DOI":"10.1016\/j.bspc.2020.101867","article-title":"Automated emotion recognition based on higher order statistics and deep learning algorithm","volume":"58","author":"Sharma","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Stickel, C., Ebner, M., Steinbach-Nordmann, S., Searle, G., and Holzinger, A. (2009, January 19\u201324). Emotion detection: Application of the valence arousal space for rapid biological usability testing to enhance universal access. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, San Diego, CA, USA.","DOI":"10.1007\/978-3-642-02707-9_70"},{"key":"ref_33","unstructured":"Ko\u0142akowska, A., Landowska, A., Szwoch, M., Szwoch, W., and Wrobel, M.R. (2014). Human-Computer Systems Interaction: Backgrounds and Applications 3, Springer."},{"key":"ref_34","first-page":"21","article-title":"The combined walkthrough: Measuring behavioral, affective, and cognitive information in usability testing","volume":"5","author":"Partala","year":"2009","journal-title":"J. Usability Stud."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1016\/j.apergo.2012.09.004","article-title":"The influence of socio-cultural background and product value in usability testing","volume":"44","author":"Sonderegger","year":"2013","journal-title":"Appl. Ergon."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1002\/mar.20196","article-title":"Toward a unified theory of consumer acceptance technology","volume":"24","author":"Kulviwat","year":"2007","journal-title":"Psychol. Mark."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5147\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:43:52Z","timestamp":1760125432000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/11\/5147"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,28]]},"references-count":36,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["s23115147"],"URL":"https:\/\/doi.org\/10.3390\/s23115147","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,28]]}}}