{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:12:36Z","timestamp":1750219956104,"version":"3.41.0"},"reference-count":87,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T00:00:00Z","timestamp":1695859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Data and Information Quality"],"published-print":{"date-parts":[[2023,9,30]]},"abstract":"<jats:p>The devices that can read Electroencephalography (EEG) signals have been widely used for Brain-Computer Interfaces (BCIs). Popularity in the field of BCIs has increased in recent years with the development of several consumer-grade EEG devices that can detect human cognitive states in real-time and deliver feedback to enhance human performance. Several previous studies have been conducted to understand the fundamentals and essential aspects of EEG in BCIs. However, the significant issue of how consumer-grade EEG devices can be used to control mechatronic systems effectively has been given less attention. In this article, we have designed and implemented an EEG BCI system using the OpenBCI Cyton headset and a user interface running a game to explore the concept of streamlining the interaction between humans and mechatronic systems with a BCI EEG-mechatronic system interface. Big Multimodal Social Data (BMSD) analytics can be applied to the high-frequency and high-volume EEG data, allowing us to explore aspects of data acquisition, data processing, and data validation and evaluate the Quality of Experience (QoE) of our system. We employ real-world participants to play a game to gather training data that was later put into multiple machine learning models, including a linear discriminant analysis (LDA), k-nearest neighbours (KNN), and a convolutional neural network (CNN). After training the machine learning models, a validation phase of the experiment took place where participants tried to play the same game but without direct control, utilising the outputs of the machine learning models to determine how the game moved. We find that a CNN trained to the specific user was able to control the game and performed with the highest activation accuracy from the machine learning models tested, along with the highest user-rated QoE, which gives us significant insight for future implementation with a mechatronic system.<\/jats:p>","DOI":"10.1145\/3597306","type":"journal-article","created":{"date-parts":[[2023,5,16]],"date-time":"2023-05-16T04:18:42Z","timestamp":1684210722000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Multimodal Social Data Analytics on the Design and Implementation of an EEG-Mechatronic System Interface"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1382-8408","authenticated-orcid":false,"given":"Cameron","family":"Aume","sequence":"first","affiliation":[{"name":"School of Engineering, Macquarie Universirty, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8784-0154","authenticated-orcid":false,"given":"Shantanu","family":"Pal","sequence":"additional","affiliation":[{"name":"School of Information Technology, Deakin University, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7818-459X","authenticated-orcid":false,"given":"Alireza","family":"Jolfaei","sequence":"additional","affiliation":[{"name":"College of Science and Engineering, Flinders University, Adelaide, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8600-5907","authenticated-orcid":false,"given":"Subhas","family":"Mukhopadhyay","sequence":"additional","affiliation":[{"name":"School of Engineering, Macquarie Universirty, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,9,28]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_3_1_2_2","DOI":"10.1016\/j.eij.2015.06.002"},{"doi-asserted-by":"crossref","unstructured":"Swati Aggarwal and Nupur Chugh. 2022. Review of machine learning techniques for EEG based brain computer interface. Archives of Computational Methods in Engineering 29 5 (2022) 3001\u20133020.","key":"e_1_3_1_3_2","DOI":"10.1007\/s11831-021-09684-6"},{"doi-asserted-by":"publisher","key":"e_1_3_1_4_2","DOI":"10.1109\/NER.2009.5109309"},{"doi-asserted-by":"crossref","unstructured":"Nikesh Bajaj. 2020. Wavelets for EEG Analysis. Retrieved from https:\/\/www.intechopen.com\/chapters\/74032.","key":"e_1_3_1_5_2","DOI":"10.5772\/intechopen.94398"},{"doi-asserted-by":"publisher","key":"e_1_3_1_6_2","DOI":"10.1109\/ICASET.2018.8376886"},{"doi-asserted-by":"publisher","key":"e_1_3_1_7_2","DOI":"10.4067\/S0716-97602007000500005"},{"doi-asserted-by":"publisher","key":"e_1_3_1_8_2","DOI":"10.1109\/SPIN.2015.7095376"},{"unstructured":"Benjamin Blankertz. 2005. BCI Competition III. Retrieved from https:\/\/www.bbci.de\/competition\/iii\/.","key":"e_1_3_1_9_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_10_2","DOI":"10.1109\/ICASSP40776.2020.9053143"},{"issue":"1","key":"e_1_3_1_11_2","first-page":"119","article-title":"A prosthetic arm based on electroencephalography by signal acquisition and processing on MATLAB","volume":"5","author":"Chaudhry Arushi","year":"2022","unstructured":"Arushi Chaudhry, Uzma Khan, Mukesh Reddy Palla, Shyam Bramhadev Singh, and Shubham Vijaykumar Deshmukh. 2022. A prosthetic arm based on electroencephalography by signal acquisition and processing on MATLAB. Int. J. Res. Eng., Sci. Manag. 5, 1 (Jan.2022), 119\u2013124. Retrieved from http:\/\/www.journals.resaim.com\/ijresm\/article\/view\/1691.","journal-title":"Int. J. Res. Eng., Sci. Manag."},{"doi-asserted-by":"publisher","key":"e_1_3_1_12_2","DOI":"10.1109\/ACCESS.2020.2969055"},{"doi-asserted-by":"publisher","key":"e_1_3_1_13_2","DOI":"10.1109\/TNSRE.2011.2174652"},{"key":"e_1_3_1_14_2","first-page":"1","volume-title":"IEEE 5th Ecuador Technical Chapters Meeting (ETCM\u201921)","author":"Chicaiza Kelvin Ortiz","year":"2021","unstructured":"Kelvin Ortiz Chicaiza and Marco E. Benalc\u00e1zar. 2021. A brain-computer interface for controlling IoT devices using EEG signals. In IEEE 5th Ecuador Technical Chapters Meeting (ETCM\u201921). IEEE, 1\u20136."},{"doi-asserted-by":"publisher","key":"e_1_3_1_15_2","DOI":"10.1109\/BIOCAS.2006.4600301"},{"doi-asserted-by":"publisher","key":"e_1_3_1_16_2","DOI":"10.1080\/0964704X.201f3.867600"},{"issue":"1","key":"e_1_3_1_17_2","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1080\/2326263X.2021.1943955","article-title":"A deep neural network-based transfer learning to enhance the performance and learning speed of BCI systems","volume":"8","author":"Dehghani Maryam","year":"2021","unstructured":"Maryam Dehghani, Ali Mobaien, and Reza Boostani. 2021. A deep neural network-based transfer learning to enhance the performance and learning speed of BCI systems. Brain-Comput. Interf. 8, 1-2 (2021), 14\u201325.","journal-title":"Brain-Comput. Interf."},{"doi-asserted-by":"publisher","key":"e_1_3_1_18_2","DOI":"10.1109\/UEMCON47517.2019.8992962"},{"key":"e_1_3_1_19_2","first-page":"65","volume-title":"International Conference on Applied Electronics","author":"Dole\u017eal J.","year":"2012","unstructured":"J. Dole\u017eal, V. \u010cern\u00fd, and J. \u0160tastn\u00fd. 2012. Online motor-imagery based BCI. In International Conference on Applied Electronics. 65\u201368."},{"doi-asserted-by":"publisher","key":"e_1_3_1_20_2","DOI":"10.1109\/ASYU48272.2019.8946364"},{"unstructured":"Emotiv. 2022. Insight Brainwear 5 Channel Wireless EEG Headset | EMOTIV. Retrieved from https:\/\/www.emotiv.com\/insight\/.","key":"e_1_3_1_21_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_22_2","DOI":"10.1109\/IST55454.2022.9827672"},{"doi-asserted-by":"publisher","key":"e_1_3_1_23_2","DOI":"10.3390\/s19061365"},{"doi-asserted-by":"publisher","key":"e_1_3_1_24_2","DOI":"10.1109\/TBME.2006.884649"},{"doi-asserted-by":"publisher","key":"e_1_3_1_25_2","DOI":"10.1016\/B978-0-12-803813-0.00009-X"},{"doi-asserted-by":"publisher","key":"e_1_3_1_26_2","DOI":"10.1109\/TBME.2019.2920711"},{"doi-asserted-by":"publisher","key":"e_1_3_1_27_2","DOI":"10.1007\/s00429-021-02274-z"},{"doi-asserted-by":"publisher","key":"e_1_3_1_28_2","DOI":"10.1109\/TBCAS.2021.3089132"},{"doi-asserted-by":"publisher","key":"e_1_3_1_29_2","DOI":"10.1109\/TCBB.2021.3052811"},{"issue":"4","key":"e_1_3_1_30_2","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1109\/TBDATA.2017.2769670","article-title":"Optimized deep learning for EEG big data and seizure prediction BCI via Internet of Things","volume":"3","author":"Hosseini Mohammad-Parsa","year":"2017","unstructured":"Mohammad-Parsa Hosseini, Dario Pompili, Kost Elisevich, and Hamid Soltanian-Zadeh. 2017. Optimized deep learning for EEG big data and seizure prediction BCI via Internet of Things. IEEE Trans. Big Data 3, 4 (2017), 392\u2013404.","journal-title":"IEEE Trans. Big Data"},{"doi-asserted-by":"publisher","key":"e_1_3_1_31_2","DOI":"10.1007\/s00381-020-04564-z"},{"doi-asserted-by":"publisher","key":"e_1_3_1_32_2","DOI":"10.1109\/ICRAE50850.2020.9310801"},{"doi-asserted-by":"publisher","key":"e_1_3_1_33_2","DOI":"10.1109\/ICISC44355.2019.9036445"},{"key":"e_1_3_1_34_2","first-page":"91","volume-title":"IEEE 12th International Symposium on Applied Machine Intelligence and Informatics (SAMI\u201914)","author":"Katona J.","year":"2014","unstructured":"J. Katona, I. Farkas, T. Ujbanyi, P. Dukan, and A. Kovari. 2014. Evaluation of the NeuroSky MindFlex EEG headset brain waves data. In IEEE 12th International Symposium on Applied Machine Intelligence and Informatics (SAMI\u201914). IEEE, 91\u201394."},{"doi-asserted-by":"publisher","key":"e_1_3_1_35_2","DOI":"10.1016\/j.compbiomed.2022.105288"},{"doi-asserted-by":"publisher","key":"e_1_3_1_36_2","DOI":"10.1016\/j.bbe.2020.02.002"},{"doi-asserted-by":"publisher","key":"e_1_3_1_37_2","DOI":"10.1109\/JBHI.2019.2951346"},{"doi-asserted-by":"publisher","key":"e_1_3_1_38_2","DOI":"10.3389\/fnhum.2021.653659"},{"doi-asserted-by":"publisher","key":"e_1_3_1_39_2","DOI":"10.3390\/s16101635"},{"key":"e_1_3_1_40_2","volume-title":"Advances in Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.), Vol. 25. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2012\/file\/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf."},{"doi-asserted-by":"publisher","key":"e_1_3_1_41_2","DOI":"10.1109\/TBME.2004.827827"},{"doi-asserted-by":"publisher","key":"e_1_3_1_42_2","DOI":"10.1109\/TCDS.2021.3098842"},{"doi-asserted-by":"publisher","key":"e_1_3_1_43_2","DOI":"10.1145\/3578709"},{"doi-asserted-by":"publisher","key":"e_1_3_1_44_2","DOI":"10.1109\/I2CACIS52118.2021.9495879"},{"doi-asserted-by":"publisher","key":"e_1_3_1_45_2","DOI":"10.3390\/s140712847"},{"doi-asserted-by":"publisher","key":"e_1_3_1_46_2","DOI":"10.1109\/SENSORS43011.2019.8956869"},{"doi-asserted-by":"publisher","key":"e_1_3_1_47_2","DOI":"10.1109\/UEMCON.2018.8796615"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1007\/978-981-15-7309-5_28","volume-title":"Advances in Mechatronics, Manufacturing, and Mechanical Engineering","author":"Kumar Jothi Letchumy Mahendra","year":"2021","unstructured":"Jothi Letchumy Mahendra Kumar, Mamunur Rashid, Rabiu Muazu Musa, Mohd Azraai Mohd Razman, Norizam Sulaiman, Rozita Jailani, and Anwar P. P. Abdul Majeed. 2021. The classification of wink-based EEG signals: The identification of significant time-domain features. In Advances in Mechatronics, Manufacturing, and Mechanical Engineering. Springer, 283\u2013291."},{"doi-asserted-by":"publisher","key":"e_1_3_1_49_2","DOI":"10.1109\/ACCESS.2017.2724555"},{"key":"e_1_3_1_50_2","first-page":"137","volume-title":"Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)","author":"Matthews Robert","year":"2007","unstructured":"Robert Matthews, Neil J. McDonald, Harini Anumula, Jamison Woodward, Peter J. Turner, Martin A. Steindorf, Kaichun Chang, and Joseph M. Pendleton. 2007. Novel hybrid bioelectrodes for ambulatory zero-prep EEG measurements using multi-channel wireless EEG system. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 4565. 137\u2013146."},{"doi-asserted-by":"publisher","key":"e_1_3_1_51_2","DOI":"10.1109\/ICMEW.2015.7169786"},{"doi-asserted-by":"publisher","key":"e_1_3_1_52_2","DOI":"10.1109\/CACIDI.2018.8584193"},{"doi-asserted-by":"publisher","key":"e_1_3_1_53_2","DOI":"10.1109\/ACT.2009.77"},{"doi-asserted-by":"publisher","key":"e_1_3_1_54_2","DOI":"10.1109\/IECBES.2016.7843462"},{"doi-asserted-by":"publisher","key":"e_1_3_1_55_2","DOI":"10.3390\/math9060606"},{"doi-asserted-by":"publisher","key":"e_1_3_1_56_2","DOI":"10.1016\/S1388-2457(00)00527-7"},{"unstructured":"OpenBCI. 2021. OpenBCI Featured Products | OpenBCI Online Store. Retrieved from https:\/\/shop.openbci.com\/collections\/frontpage.","key":"e_1_3_1_57_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_58_2","DOI":"10.1007\/978-981-16-4035-3_3"},{"key":"e_1_3_1_59_2","volume-title":"Internet of Things and Access Control: Sensing, Monitoring and Controlling Access in IoT-Enabled Healthcare Systems","author":"Pal Shantanu","year":"2021","unstructured":"Shantanu Pal. 2021. Internet of Things and Access Control: Sensing, Monitoring and Controlling Access in IoT-Enabled Healthcare Systems. Vol. 37. Springer Nature."},{"doi-asserted-by":"publisher","key":"e_1_3_1_60_2","DOI":"10.1016\/B978-0-12-818546-9.00001-4"},{"issue":"13","key":"e_1_3_1_61_2","article-title":"Fine-grained access control for smart healthcare systems in the Internet of Things","volume":"4","author":"Pal Shantanu","year":"2018","unstructured":"Shantanu Pal, Michael Hitchens, Vijay Varadharajan, and Tahiry Rabehaja. 2018. Fine-grained access control for smart healthcare systems in the Internet of Things. EAI Endors. Trans. Industr. Netw. Intell. Syst. 4, 13 (2018).","journal-title":"EAI Endors. Trans. Industr. Netw. Intell. Syst."},{"doi-asserted-by":"publisher","key":"e_1_3_1_62_2","DOI":"10.3390\/s21165554"},{"doi-asserted-by":"publisher","key":"e_1_3_1_63_2","DOI":"10.1145\/3571306.3571449"},{"doi-asserted-by":"publisher","key":"e_1_3_1_64_2","DOI":"10.1109\/INDISCON54605.2022.9862906"},{"doi-asserted-by":"publisher","key":"e_1_3_1_65_2","DOI":"10.1002\/ana.24390"},{"doi-asserted-by":"publisher","key":"e_1_3_1_66_2","DOI":"10.1109\/IEMBS.2009.5334189"},{"doi-asserted-by":"publisher","key":"e_1_3_1_67_2","DOI":"10.1109\/FORTEI-ICEE50915.2020.9249937"},{"doi-asserted-by":"publisher","key":"e_1_3_1_68_2","DOI":"10.1002\/ana.23879"},{"doi-asserted-by":"publisher","key":"e_1_3_1_69_2","DOI":"10.3390\/make3040042"},{"doi-asserted-by":"publisher","key":"e_1_3_1_70_2","DOI":"10.1145\/3570991.3570995"},{"doi-asserted-by":"publisher","key":"e_1_3_1_71_2","DOI":"10.1007\/s11760-022-02142-1"},{"doi-asserted-by":"publisher","key":"e_1_3_1_72_2","DOI":"10.1016\/B978-0-12-815553-0.00013-6"},{"key":"e_1_3_1_73_2","first-page":"2025","volume-title":"22nd European Signal Processing Conference (EUSIPCO\u201914)","author":"Samadi Mohammad Reza Haji","year":"2014","unstructured":"Mohammad Reza Haji Samadi and Neil Cooke. 2014. VOG-enhanced ICA for SSVEP response detection from consumer-grade EEG. In 22nd European Signal Processing Conference (EUSIPCO\u201914). 2025\u20132029."},{"doi-asserted-by":"publisher","key":"e_1_3_1_74_2","DOI":"10.3390\/electronics7120384"},{"key":"e_1_3_1_75_2","volume-title":"1st International Workshop on Robotic Sensing (ROSE\u201903).","author":"Siegel Mel","year":"2003","unstructured":"Mel Siegel. 2003. The sense-think-act paradigm revisited. In 1st International Workshop on Robotic Sensing (ROSE\u201903). IEEE."},{"unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-scale Image Recognition. Retrieved from https:\/\/arxiv.org\/abs\/1409.1556.","key":"e_1_3_1_76_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_77_2","DOI":"10.1007\/978-3-319-72038-8_5"},{"doi-asserted-by":"publisher","key":"e_1_3_1_78_2","DOI":"10.1016\/j.ipm.2022.102888"},{"doi-asserted-by":"publisher","key":"e_1_3_1_79_2","DOI":"10.1007\/s00062-009-9002-3"},{"doi-asserted-by":"publisher","key":"e_1_3_1_80_2","DOI":"10.1109\/ETCCE51779.2020.9350883"},{"doi-asserted-by":"publisher","key":"e_1_3_1_81_2","DOI":"10.1109\/ICASSP.2018.8462115"},{"key":"e_1_3_1_82_2","first-page":"1654","volume-title":"3rd International Conference on Computing for Sustainable Global Development (INDIACom\u201916)","author":"Tibdewal Manish N.","year":"2016","unstructured":"Manish N. Tibdewal, M. Mahadevappa, Ajoy Kumar Ray, Monika Malokar, and Himanshu R. Dey. 2016. Power line and ocular artifact denoising from EEG using notch filter and wavelet transform. In 3rd International Conference on Computing for Sustainable Global Development (INDIACom\u201916). 1654\u20131659."},{"doi-asserted-by":"publisher","key":"e_1_3_1_83_2","DOI":"10.1016\/0013-4694(93)90133-G"},{"doi-asserted-by":"publisher","key":"e_1_3_1_84_2","DOI":"10.1109\/TMECH.2022.3148141"},{"doi-asserted-by":"publisher","key":"e_1_3_1_85_2","DOI":"10.1109\/EIT.2018.8500283"},{"key":"e_1_3_1_86_2","first-page":"249","volume-title":"International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS\u201919)","author":"Yudhana Anton","year":"2019","unstructured":"Anton Yudhana, Subhas Mukhopadhyay, Ismail Rakip Karas, Ahmad Azhari, Murein Miksa Mardhia, Son Ali Akbar, Akbar Muslim, and Fathia Irbati Ammatulloh. 2019. Recognizing human emotion patterns by applying fast Fourier transform based on brainwave features. In International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS\u201919). IEEE, 249\u2013254."},{"doi-asserted-by":"publisher","key":"e_1_3_1_87_2","DOI":"10.1016\/j.cogr.2021.02.001"},{"doi-asserted-by":"publisher","key":"e_1_3_1_88_2","DOI":"10.1016\/j.bspc.2020.102144"}],"container-title":["Journal of Data and Information Quality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3597306","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3597306","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:06Z","timestamp":1750182546000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3597306"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":87,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,9,30]]}},"alternative-id":["10.1145\/3597306"],"URL":"https:\/\/doi.org\/10.1145\/3597306","relation":{},"ISSN":["1936-1955","1936-1963"],"issn-type":[{"type":"print","value":"1936-1955"},{"type":"electronic","value":"1936-1963"}],"subject":[],"published":{"date-parts":[[2023,9,28]]},"assertion":[{"value":"2022-11-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-21","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}