{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T18:22:21Z","timestamp":1771006941833,"version":"3.50.1"},"reference-count":55,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2022,8,17]],"date-time":"2022-08-17T00:00:00Z","timestamp":1660694400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>As lightweight, low-cost EEG headsets emerge, the feasibility of consumer-oriented brain\u2013computer interfaces (BCI) increases. The combination of portable smartphones and easy-to-use EEG dry electrode headbands offers intriguing new applications and methods of human\u2013computer interaction. In previous research, augmented reality (AR) scenarios have been identified to profit from additional user state information\u2014such as that provided by a BCI. In this work, we implemented a system that integrates user attentional state awareness into a smartphone application for an AR written language translator. The attentional state of the user is classified in terms of internally and externally directed attention by using the Muse 2 electroencephalography headband with four frontal electrodes. The classification results are used to adapt the behavior of the translation app, which uses the smartphone\u2019s camera to display translated text as augmented reality elements. We present the first mobile BCI system that uses a smartphone and a low-cost EEG device with few electrodes to provide attention awareness to an AR application. Our case study with 12 participants did not fully support the assumption that the BCI improves usability. However, we are able to show that the classification accuracy and ease of setup are promising paths toward mobile consumer-oriented BCI usage. For future studies, other use cases, applications, and adaptations will be tested for this setup to explore the usability.<\/jats:p>","DOI":"10.3390\/s22166160","type":"journal-article","created":{"date-parts":[[2022,8,17]],"date-time":"2022-08-17T22:53:30Z","timestamp":1660776810000},"page":"6160","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["AtAwAR Translate: Attention-Aware Language Translation Application in Augmented Reality for Mobile Phones"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0601-0950","authenticated-orcid":false,"given":"Lisa-Marie","family":"Vortmann","sequence":"first","affiliation":[{"name":"Cognitive Systems Lab, University of Bremen, 28359 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pascal","family":"Weidenbach","sequence":"additional","affiliation":[{"name":"Cognitive Systems Lab, University of Bremen, 28359 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5203-8797","authenticated-orcid":false,"given":"Felix","family":"Putze","sequence":"additional","affiliation":[{"name":"Cognitive Systems Lab, University of Bremen, 28359 Bremen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,17]]},"reference":[{"key":"ref_1","unstructured":"(2021, November 09). Camera-Based AR Translation for Mobile Phones. Available online: https:\/\/translate.google.com\/intl\/en\/about\/."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1111\/j.1469-8986.2007.00640.x","article-title":"Involvement of memory-comparison-based change detection in visual distraction","volume":"45","author":"Kimura","year":"2008","journal-title":"Psychophysiology"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e06277","DOI":"10.1016\/j.heliyon.2021.e06277","article-title":"A review of augmented reality systems and their effects on mental workload and task performance","volume":"7","author":"Jeffri","year":"2021","journal-title":"Heliyon"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1146\/annurev.psych.093008.100427","article-title":"A taxonomy of external and internal attention","volume":"62","author":"Chun","year":"2011","journal-title":"Annu. Rev. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"348","DOI":"10.3389\/fnhum.2019.00348","article-title":"EEG-based classification of internally-and externally-directed attention in an augmented reality paradigm","volume":"13","author":"Vortmann","year":"2019","journal-title":"Front. Hum. Neurosci."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Vortmann, L.M., and Putze, F. (2020, January 25\u201330). Attention-aware brain computer interface to avoid distractions in augmented reality. Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3334480.3382889"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2291","DOI":"10.1038\/s41598-022-06296-3","article-title":"Towards real-world neuroscience using mobile EEG and augmented reality","volume":"12","author":"Krugliak","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhao, G., Zhang, L., Chu, J., Zhu, W., Hu, B., He, H., and Yang, L. (2022). An Augmented Reality Based Mobile Photography Application to Improve Learning Gain, Decrease Cognitive Load, and Achieve Better Emotional State. Int. J. Hum.-Comput. Interact., 1\u201316.","DOI":"10.1080\/10447318.2022.2041911"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1109\/THMS.2022.3179954","article-title":"Design Eye-Tracking Augmented Reality Headset to Reduce Cognitive Load in Repetitive Parcel Scanning Task","volume":"52","author":"Yan","year":"2022","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tatwany, L., and Ouertani, H.C. (2017, January 19\u201321). A review on using augmented reality in text translation. Proceedings of the 2017 6th International Conference on Information and Communication Technology and Accessibility (ICTA), Muscat, Oman.","DOI":"10.1109\/ICTA.2017.8336044"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Fragoso, V., Gauglitz, S., Zamora, S., Kleban, J., and Turk, M. (2011, January 5\u20137). TranslatAR: A mobile augmented reality translator. Proceedings of the 2011 IEEE Workshop on Applications of Computer Vision (WACV), Kona, HI, USA.","DOI":"10.1109\/WACV.2011.5711545"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Toyama, T., Sonntag, D., Dengel, A., Matsuda, T., Iwamura, M., and Kise, K. (2014, January 24\u201327). A mixed reality head-mounted text translation system using eye gaze input. Proceedings of the 19th International Conference on Intelligent User Interfaces, Haifa, Israel.","DOI":"10.1145\/2557500.2557528"},{"key":"ref_13","unstructured":"Salvado, A.R.d.T. (2015). Augmented Reality Applied to Language Translation. [Ph.D. Thesis, Universidade Nova de Lisboa]."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Vortmann, L.M., Schult, M., Benedek, M., Walcher, S., and Putze, F. (2019, January 14\u201318). Real-time multimodal classification of internal and external attention. Proceedings of the Adjunct of the 2019 International Conference on Multimodal Interaction, Suzhou, China.","DOI":"10.1145\/3351529.3360658"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"625","DOI":"10.3389\/fnins.2021.664490","article-title":"Imaging Time Series of Eye Tracking Data to Classify Attentional States","volume":"15","author":"Vortmann","year":"2021","journal-title":"Front. Neurosci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e12977","DOI":"10.1111\/cogs.12977","article-title":"How Reliably Do Eye Parameters Indicate Internal Versus External Attentional Focus?","volume":"45","author":"Ceh","year":"2021","journal-title":"Cogn. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/S0167-8760(02)00107-1","article-title":"Paradox lost? Exploring the role of alpha oscillations during externally vs. internally directed attention and the implications for idling and inhibition hypotheses","volume":"47","author":"Cooper","year":"2003","journal-title":"Int. J. Psychophysiol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/j.neuropsychologia.2014.02.010","article-title":"Alpha power increases in right parietal cortex reflects focused internal attention","volume":"56","author":"Benedek","year":"2014","journal-title":"Neuropsychologia"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Putze, F., Scherer, M., and Schultz, T. (2016, January 23\u201327). Starring into the void? Classifying Internal vs. External Attention from EEG. Proceedings of the 9th Nordic Conference on Human-Computer Interaction, Gothenburg, Sweden.","DOI":"10.1145\/2971485.2971555"},{"key":"ref_20","unstructured":"(2021, November 12). Consumer-Grade EEG Headband. Available online: https:\/\/choosemuse.com\/muse-2\/."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1080\/10447318.2019.1612213","article-title":"Brain\u2013computer interface games based on consumer-grade EEG Devices: A systematic literature review","volume":"36","author":"Vasiljevic","year":"2020","journal-title":"Int. J. Hum.-Comput. Interact."},{"key":"ref_22","first-page":"60","article-title":"Concept, development and evaluation of a mind action game with the electro encephalograms as an auxiliary input","volume":"8","author":"Joselli","year":"2017","journal-title":"SBC J. Interact. Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"620391","DOI":"10.1155\/2014\/620391","article-title":"BCI sensor based environment changing system for immersion of 3D game","volume":"10","author":"Cho","year":"2014","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_24","unstructured":"Mikami, K., Kondo, K., and Kondo, K. (2017, January 2\u20133). Adaptable Game Experience Based on Player\u2019s Performance and EEG. Proceedings of the 2017 Nicograph International (NicoInt), Kyoto, Japan."},{"key":"ref_25","first-page":"6","article-title":"Wearable, wireless EEG solutions in daily life applications: What are we missing?","volume":"19","author":"Grundlehner","year":"2014","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Nam, C.S., Nijholt, A., and Lotte, F. (2018). Brain\u2013Computer Interfaces Handbook: Technological and Theoretical Advances, CRC Press.","DOI":"10.1201\/9781351231954"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"814","DOI":"10.1080\/10447318.2013.780869","article-title":"EEG-based brain-computer interfaces: A thorough literature survey","volume":"29","author":"Hwang","year":"2013","journal-title":"Int. J.-Hum.-Comput. Interact."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Galway, L., McCullagh, P., Lightbody, G., Brennan, C., and Trainor, D. (2015, January 26\u201328). The potential of the brain-computer interface for learning: A technology review. Proceedings of the 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, Liverpool, UK.","DOI":"10.1109\/CIT\/IUCC\/DASC\/PICOM.2015.234"},{"key":"ref_29","unstructured":"Soman, S., Srivastava, S., Srivastava, S., and Rajput, N. (2015). Brain computer interfaces for mobile apps: State-of-the-art and future directions. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"109","DOI":"10.3389\/fnins.2017.00109","article-title":"Choosing MUSE: Validation of a low-cost, portable EEG system for ERP research","volume":"11","author":"Krigolson","year":"2017","journal-title":"Front. Neurosci."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zgallai, W., Brown, J.T., Ibrahim, A., Mahmood, F., Mohammad, K., Khalfan, M., Mohammed, M., Salem, M., and Hamood, N. (April, January 26). Deep Learning AI Application to an EEG driven BCI Smart Wheelchair. Proceedings of the 2019 Advances in Science and Engineering Technology International Conferences (ASET), Dubai, United Arab Emirates.","DOI":"10.1109\/ICASET.2019.8714373"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zander, T.O., Kothe, C., Jatzev, S., and Gaertner, M. (2010). Enhancing human-computer interaction with input from active and passive brain-computer interfaces. Brain-Computer Interfaces, Springer.","DOI":"10.1007\/978-1-84996-272-8_11"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"08TR02","DOI":"10.1088\/1361-6579\/aad57e","article-title":"Passive BCI beyond the lab: Current trends and future directions","volume":"39","author":"Borghini","year":"2018","journal-title":"Physiol. Meas."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Roy, R.N., and Frey, J. (2016). Neurophysiological markers for passive brain\u2013computer interfaces. Brain\u2013Computer Interfaces 1: Foundations and Methods, ISTE.","DOI":"10.1002\/9781119144977.ch5"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"78","DOI":"10.3389\/fnhum.2017.00078","article-title":"Evaluation of a dry EEG system for application of passive brain-computer interfaces in autonomous driving","volume":"11","author":"Zander","year":"2017","journal-title":"Front. Hum. Neurosci."},{"key":"ref_36","unstructured":"PaddleOCR (2021, November 12). lightweight Optical Character Recognition Using Neural Networks. Available online: https:\/\/github.com\/PaddlePaddle\/PaddleOCR."},{"key":"ref_37","unstructured":"Du, Y., Li, C., Guo, R., Yin, X., Liu, W., Zhou, J., Bai, Y., Yu, Z., Yang, Y., and Dang, Q. (2020). PP-OCR: A practical ultra lightweight OCR system. arXiv."},{"key":"ref_38","unstructured":"Kaehler, A., and Bradski, G. (2016). Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library, O\u2019Reilly Media, Inc."},{"key":"ref_39","unstructured":"mindmonitor (2021, November 12). Real Time EEG Graphs from Your Interaxon Muse Headband. Available online: https:\/\/mind-monitor.com\/."},{"key":"ref_40","unstructured":"JavaOSC (2021, November 12). Library That Gives JVM Language Programs the Ability of Working with OSC Content Format. Available online: https:\/\/github.com\/hoijui\/JavaOSC."},{"key":"ref_41","unstructured":"Muse Headset (2021, November 12). Technical Specifications, Validation, and Research Use. Available online: https:\/\/images-na.ssl-images-amazon.com\/images\/I\/D1RREdoENNS.pdf."},{"key":"ref_42","unstructured":"Conrad, C.D., and Bliemel, M. (2016, January 11\u201314). Psychophysiological measures of cognitive absorption and cognitive load in e-learning applications. Proceedings of the Thirty Sixth International Conference on Information Systems, Dublin, Ireland."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Przegalinska, A., Ciechanowski, L., Magnuski, M., and Gloor, P. (2018). Muse headband: Measuring tool or a collaborative gadget?. Collaborative Innovation Networks, Springer.","DOI":"10.1007\/978-3-319-74295-3_8"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3463507","article-title":"Exploration of Person-Independent BCIs for Internal and External Attention-Detection in Augmented Reality","volume":"5","author":"Vortmann","year":"2021","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_45","unstructured":"MuseIO (2021, November 12). Muse for Developers. Available online: https:\/\/web.archive.org\/web\/20181105231756\/http:\/\/developer.choosemuse.com\/tools\/available-data#Absolute_Band_Powers."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Tan, D., and Nijholt, A. (2010). Brain-computer interfaces and human-computer interaction. Brain-Computer Interfaces, Springer.","DOI":"10.1007\/978-1-84996-272-8"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"560","DOI":"10.3389\/fpsyg.2013.00560","article-title":"Young and restless: Validation of the Mind-Wandering Questionnaire (MWQ) reveals disruptive impact of mind-wandering for youth","volume":"4","author":"Mrazek","year":"2013","journal-title":"Front. Psychol."},{"key":"ref_48","first-page":"4","article-title":"SUS-A quick and dirty usability scale","volume":"189","author":"Brooke","year":"1996","journal-title":"Usability Eval. Ind."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","article-title":"Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research","volume":"Volume 52","author":"Hart","year":"1988","journal-title":"Advances in Psychology"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"904","DOI":"10.1177\/154193120605000909","article-title":"NASA-task load index (NASA-TLX); 20 years later","volume":"Volume 50","author":"Hart","year":"2006","journal-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting"},{"key":"ref_51","unstructured":"Rummel, B. (2021, November 12). System Usability Scale\u2014Jetzt Auch auf Deutsch. Available online: https:\/\/blogs.sap.com\/2016\/02\/01\/system-usability-scale-jetzt-auch-auf-deutsch\/."},{"key":"ref_52","unstructured":"(2021, November 12). NASA-TLX. (Kurzfassung Deutsch). Available online: http:\/\/interaction-design-group.de\/toolbox\/wp-content\/uploads\/2016\/05\/NASA-TLX.pdf."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"979","DOI":"10.1111\/j.1467-9280.2005.01647.x","article-title":"Unique temporal change is the key to attentional capture","volume":"16","author":"Rempel","year":"2005","journal-title":"Psychol. Sci."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/S0896-6273(03)00689-5","article-title":"Attention-grabbing motion in the human brain","volume":"40","author":"Culham","year":"2003","journal-title":"Neuron"},{"key":"ref_55","first-page":"114","article-title":"Determining what individual SUS scores mean: Adding an adjective rating scale","volume":"4","author":"Bangor","year":"2009","journal-title":"J. Usability Stud."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6160\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:11:11Z","timestamp":1760141471000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6160"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,17]]},"references-count":55,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22166160"],"URL":"https:\/\/doi.org\/10.3390\/s22166160","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,17]]}}}