{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T11:05:26Z","timestamp":1781607926615,"version":"3.54.5"},"reference-count":70,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T00:00:00Z","timestamp":1754870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["GR 2627\/13-1"],"award-info":[{"award-number":["GR 2627\/13-1"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Virtual Real."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Neuroadaptive technology provides a promising path to enhancing immersive extended reality (XR) experiences by dynamically tuning multisensory feedback to user preferences. This study introduces a novel system employing reinforcement learning (RL) to adapt haptic rendering in XR environments based on user feedback derived either explicitly from user ratings or implicitly from neural signals measured via Electroencephalography (EEG).<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Participants interacted with virtual objects in a VR environment and rated their experience using a traditional questionnaire while their EEG data were recorded. Then, in two RL conditions, an RL agent tried to tune the haptics to the user \u2014 learning either on the rewards from explicit ratings, or on implicit neural signals decoded from EEG.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The neural decoder achieved a mean F1 score of 0.8, supporting informative yet noisy classification. Exploratory analyses revealed instability in the RL agent\u2019s behavior in both explicit and implicit feedback conditions.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>A limited number of interaction steps likely constrained exploration and contributed to convergence instability. Revisiting the interaction design to support more frequent sampling may improve robustness to EEG noise and mitigate drifts in subjective experience. By demonstrating RL\u2010based adaptation from implicit neural signals, our proof-of-concept is a step towards seamless, low-friction personalization in XR.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frvir.2025.1616442","type":"journal-article","created":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T05:23:34Z","timestamp":1754889814000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Neuroadaptive haptics: a proof-of-concept comparing reinforcement learning from explicit ratings and neural signals for adaptive XR systems"],"prefix":"10.3389","volume":"6","author":[{"given":"Lukas","family":"Gehrke","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aleksandrs","family":"Koselevs","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marius","family":"Klug","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klaus","family":"Gramann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,8,11]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1146\/annurev.neuro.25.112701.142922","article-title":"Intentional maps in posterior parietal cortex","volume":"25","author":"Andersen","year":"2002","journal-title":"Annu. Rev. Neurosci."},{"key":"B2","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1023\/a:1013689704352","article-title":"Finite-time analysis of the multiarmed bandit problem","volume":"47","author":"Auer","year":"2002","journal-title":"Mach. Learn."},{"key":"B3","doi-asserted-by":"publisher","first-page":"412","DOI":"10.1016\/j.neuroimage.2013.02.063","article-title":"The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value","volume":"76","author":"Bartra","year":"2013","journal-title":"Neuroimage"},{"key":"B4","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1016\/j.neuron.2012.10.038","article-title":"Canonical microcircuits for predictive coding","volume":"76","author":"Bastos","year":"2012","journal-title":"Neuron"},{"key":"B5","article-title":"Adapting virtual reality horror experiences using fNIRS","volume-title":"Proceedings of the fNIRS biennial meeting 2024","author":"Berger","year":"2024"},{"key":"B6","doi-asserted-by":"publisher","first-page":"814","DOI":"10.1016\/j.neuroimage.2010.06.048","article-title":"Single-trial analysis and classification of ERP components \u2014 a tutorial","volume":"56","author":"Blankertz","year":"2011","journal-title":"Neuroimage"},{"key":"B7","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1109\/38.799723","article-title":"What\u2019s real about virtual reality?","volume":"19","author":"Brooks","year":"1999","journal-title":"IEEE Comput. Graph. Appl."},{"key":"B8","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1080\/00029238.1985.11080163","article-title":"Ten percent electrode system for topographic studies of spontaneous and evoked EEG activities","volume":"25","author":"Chatrian","year":"1985","journal-title":"Am. J. EEG Technol."},{"key":"B9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3654777.3676384","article-title":"SonoHaptics: an audio-haptic cursor for gaze-based object selection in XR","volume":"8","author":"Cho","year":"2024","journal-title":"Proc. 37th Annu. ACM Symposium User Interface Softw. Technol."},{"key":"B10","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1017\/s0140525x12000477","article-title":"Whatever next? predictive brains, situated agents, and the future of cognitive science","volume":"36","author":"Clark","year":"2013","journal-title":"Behav. Brain Sci."},{"key":"B11","doi-asserted-by":"publisher","first-page":"268","DOI":"10.3389\/fnins.2020.00268","article-title":"A neuroergonomics approach to mental workload, engagement and human performance","volume":"14","author":"Dehais","year":"2020","journal-title":"Front. Neurosci."},{"key":"B12","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.jneumeth.2003.10.009","article-title":"EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis","volume":"134","author":"Delorme","year":"2004","journal-title":"J. Neurosci. Methods"},{"key":"B13","doi-asserted-by":"publisher","first-page":"066010","DOI":"10.1088\/1741-2552\/ab3bb4","article-title":"Convolutional neural networks for decoding of covert attention focus and saliency maps for EEG feature visualization","volume":"16","author":"Farahat","year":"2019","journal-title":"J. Neural Eng."},{"key":"B14","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: a unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci."},{"key":"B15","article-title":"Context informed incremental learning improves myoelectric control performance in virtual reality object manipulation tasks","author":"Gagn\u00e9","year":"2025"},{"key":"B16","first-page":"1","article-title":"Detecting visuo-haptic mismatches in virtual reality using the prediction error negativity of event-related brain potentials","author":"Gehrke","year":"2019"},{"key":"B17","doi-asserted-by":"publisher","first-page":"036002","DOI":"10.1088\/1741-2552\/ac69bc","article-title":"Neural sources of prediction errors detect unrealistic VR interactions","volume":"19","author":"Gehrke","year":"2022","journal-title":"J. Neural Eng."},{"key":"B18","doi-asserted-by":"publisher","first-page":"1411305","DOI":"10.3389\/fnrgo.2024.1411305","article-title":"Visuo-haptic prediction errors: a multimodal dataset (EEG, motion) in BIDS format indexing mismatches in haptic interaction","volume":"5","author":"Gehrke","year":"2024","journal-title":"Front. Neuroergonomics"},{"key":"B19","doi-asserted-by":"publisher","first-page":"593","DOI":"10.1515\/rns.2011.047","article-title":"Cognition in action: imaging brain\/body dynamics in Mobile humans","volume":"22","author":"Gramann","year":"2011","journal-title":"Rev. Neurosci."},{"key":"B20","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1080\/1463922021000020918","article-title":"Neuroadaptive technologies: applying neuroergonomics to the design of advanced interfaces","volume":"4","author":"Hettinger","year":"2003","journal-title":"Theor. Issues Ergon."},{"key":"B21","doi-asserted-by":"publisher","first-page":"456","DOI":"10.1016\/j.neubiorev.2020.09.036","article-title":"I tried a bunch of things: the dangers of unexpected overfitting in classification of brain data","volume":"119","author":"Hosseini","year":"2020","journal-title":"Neurosci. Biobehav. Rev."},{"key":"B22","doi-asserted-by":"publisher","first-page":"862","DOI":"10.21105\/joss.00862","article-title":"Pymer4: connecting R and python for linear mixed modeling","volume":"3","author":"Jolly","year":"2018","journal-title":"J. Open Source Softw."},{"key":"B23","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/B978-0-12-811926-6.00010-5","article-title":"Chapter 10 - mobi\u2014mobile brain\/body imaging","volume-title":"Neuroergonomics","author":"Jungnickel","year":"2019"},{"key":"B24","doi-asserted-by":"publisher","first-page":"051002","DOI":"10.1088\/1741-2552\/acfcda","article-title":"The future of wearable EEG: a review of ear-EEG technology and its applications","volume":"20","author":"Kaongoen","year":"2023","journal-title":"J. Neural Eng."},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2312.14925","article-title":"A survey of reinforcement learning from human feedback","author":"Kaufmann","year":"2023","journal-title":"arXiv"},{"key":"B26","doi-asserted-by":"publisher","first-page":"424","DOI":"10.1016\/j.neuron.2018.10.003","article-title":"Predictive processing: a canonical cortical computation","volume":"100","author":"Keller","year":"2018","journal-title":"Neuron"},{"key":"B27","doi-asserted-by":"publisher","first-page":"510051","DOI":"10.1101\/2022.09.29.510051","article-title":"The BeMoBIL pipeline for automated analyses of multimodal Mobile brain and body imaging data","volume":"2022","author":"Klug","year":"2022","journal-title":"bioRxiv"},{"key":"B28","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1016\/j.tins.2004.10.007","article-title":"The Bayesian brain: the role of uncertainty in neural coding and computation","volume":"27","author":"Knill","year":"2004","journal-title":"Trends Neurosci."},{"key":"B29","article-title":"Augmenting reinforcement learning with human feedback","author":"Knox","year":"2011"},{"key":"B30","doi-asserted-by":"crossref","DOI":"10.1145\/2856767.2856796","article-title":"RelaWorld","volume-title":"Proceedings of the 21st international conference on intelligent user interfaces","author":"Kosunen","year":"2016"},{"key":"B31","doi-asserted-by":"crossref","DOI":"10.1101\/2024.02.13.580071","volume-title":"The lab streaming layer for synchronized multimodal recording","author":"Kothe","year":"2024"},{"key":"B32","doi-asserted-by":"publisher","first-page":"012001","DOI":"10.1088\/1741-2552\/ab5bb5","article-title":"Cognitive and affective probing: a tutorial and review of active learning for neuroadaptive technology","volume":"17","author":"Krol","year":"2020","journal-title":"J. Neural Eng."},{"key":"B33","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/B978-0-12-821413-8.00007-5","article-title":"Defining neuroadaptive technology: the trouble with implicit human-computer interaction","volume-title":"Current research in neuroadaptive technology","author":"Krol","year":"2022"},{"key":"B34","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1016\/s0047-259x(03)00096-4","article-title":"A well-conditioned estimator for large-dimensional covariance matrices","volume":"88","author":"Ledoit","year":"2004","journal-title":"J. Multivar. Anal."},{"key":"B35","doi-asserted-by":"publisher","first-page":"216","DOI":"10.32614\/CRAN.package.emmeans","article-title":"Package \u2018Emmeans","volume":"34","author":"Lenth","year":"2020","journal-title":"R. package version"},{"key":"B36","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1109\/thms.2019.2912447","article-title":"Human-centered reinforcement learning: a survey","volume":"49","author":"Li","year":"2019","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"B37","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1145\/2807442.2807443","article-title":"Impacto: simulating physical impact by combining tactile stimulation with electrical muscle stimulation","volume-title":"Proceedings of the 28th annual ACM symposium on user interface software and technology","author":"Lopes","year":"2015"},{"key":"B38","doi-asserted-by":"publisher","first-page":"031005","DOI":"10.1088\/1741-2552\/aab2f2","article-title":"A review of classification algorithms for EEG-Based brain\u2013computer interfaces: a 10 year update","volume":"15","author":"Lotte","year":"2018","journal-title":"J. Neural Eng."},{"key":"B39","doi-asserted-by":"crossref","DOI":"10.1109\/BIBM.2018.8621183","article-title":"Deep reinforcement learning from error-related potentials via an EEG-Based brain-computer interface","volume-title":"IEEE international conference on bioinformatics and biomedicine (BIBM)","author":"Luo","year":"2018"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TNSRE.2025.3530154","article-title":"Using passive BCI for personalization of assistive wearable devices: a proof-of-concept study","author":"Mahmoudi","year":"2025","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"B41","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1016\/j.ijpsycho.2008.11.008","article-title":"Linking brain, mind and behavior","volume":"73","author":"Makeig","year":"2009","journal-title":"Int. J. Psychophysiol."},{"key":"B42","doi-asserted-by":"publisher","first-page":"276","DOI":"10.1016\/j.chb.2017.02.066","article-title":"Development and validation of the multimodal presence scale for virtual reality environments: a confirmatory factor analysis and item response theory approach","volume":"72","author":"Makransky","year":"2017","journal-title":"Comput. Hum. Behav."},{"key":"B43","doi-asserted-by":"publisher","first-page":"820523","DOI":"10.3389\/fnrgo.2022.820523","article-title":"Neuroadaptive training via fNIRS in flight simulators","volume":"3","author":"Mark","year":"2022","journal-title":"Front. Neuroergonomics"},{"key":"B44","doi-asserted-by":"publisher","first-page":"889","DOI":"10.3389\/fnhum.2013.00889","article-title":"Neuroergonomics: a review of applications to physical and cognitive work","volume":"7","author":"Mehta","year":"2013","journal-title":"Front. Hum. Neurosci."},{"key":"B45","doi-asserted-by":"publisher","first-page":"655","DOI":"10.1007\/s00429-010-0262-0","article-title":"Saliency, switching, attention and control: a network model of insula function","volume":"214","author":"Menon","year":"2010","journal-title":"Brain Struct. Funct."},{"key":"B46","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1146\/annurev-psych-010416-044022","article-title":"Learning from errors","volume":"68","author":"Metcalfe","year":"2017","journal-title":"Annu. Rev. Psychol."},{"key":"B47","doi-asserted-by":"publisher","first-page":"1205881","DOI":"10.3389\/fnhum.2023.1205881","article-title":"The effects of layer-wise relevance propagation-based feature selection for EEG classification: a comparative study on multiple datasets","volume":"17","author":"Nam","year":"2023","journal-title":"Front. Hum. Neurosci."},{"key":"B48","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1109\/ISMAR-Adjunct64951.2024.00022","article-title":"Exploring eye tracking to detect cognitive load in complex virtual reality training","volume-title":"2024 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct)","author":"Nasri","year":"2024"},{"key":"B49","doi-asserted-by":"publisher","first-page":"119774","DOI":"10.1016\/j.neuroimage.2022.119774","article-title":"Wireless EEG: a survey of systems and studies","volume":"269","author":"Niso","year":"2023","journal-title":"Neuroimage"},{"key":"B50","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1098\/rspl.1895.0041","article-title":"VII. Note on regression and inheritance in the case of two parents","volume":"58","author":"Pearson","year":"1895","journal-title":"Proc. R. Soc. Lond."},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1201.0490","article-title":"Scikit-learn: machine learning in python","author":"Pedregosa","year":"2012","journal-title":"arXiv [cs.LG]"},{"key":"B52","volume-title":"Mixed-Effects models in S and S-PLUS","author":"Pinheiro","year":"2006"},{"key":"B53","doi-asserted-by":"publisher","first-page":"3193","DOI":"10.1109\/tvcg.2021.3057797","article-title":"Adapting virtual embodiment through reinforcement learning","volume":"28","author":"Porssut","year":"2022","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.3389\/frsip.2022.904866","article-title":"On the relative importance of visual and spatial audio rendering on VR immersion","volume":"2","author":"Potter","year":"2022","journal-title":"Front. Signal Process"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1170","DOI":"10.1038\/nn.3495","article-title":"Probabilistic brains: knowns and unknowns","volume":"16","author":"Pouget","year":"2013","journal-title":"Nat. Neurosci."},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.3389\/frvir.2024.1358981","article-title":"Immersion in a relaxing virtual reality environment is associated with similar effects on stress and anxiety as heart rate variability biofeedback","volume":"5","author":"Pratviel","year":"2024","journal-title":"Front. Virtual Real"},{"key":"B57","first-page":"603","article-title":"Using haptics to improve immersion in virtual environments","volume-title":"Lecture notes in computer science","author":"Ramsamy","year":"2006"},{"key":"B58","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"Rao","year":"1999","journal-title":"Nat. Neurosci."},{"key":"B59","doi-asserted-by":"publisher","first-page":"1795","DOI":"10.1152\/jn.00546.2015","article-title":"Analysis of haptic information in the cerebral cortex","volume":"116","author":"Sathian","year":"2016","journal-title":"J. Neurophysiol."},{"key":"B60","first-page":"653","volume-title":"Detecting system errors in virtual reality using EEG through error-related potentials","author":"Si-mohammed","year":"2020"},{"key":"B61","doi-asserted-by":"publisher","first-page":"24617","DOI":"10.1109\/access.2018.2832089","article-title":"Visual appearance modulates prediction error in virtual reality","volume":"6","author":"Singh","year":"2018","journal-title":"IEEE Access"},{"key":"B62","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2307\/2331554","article-title":"The probable error of a mean","volume":"6","year":"1908","journal-title":"Biometrika"},{"key":"B63","first-page":"1","article-title":"Can a smartwatch move your fingers? Compact and practical electrical muscle stimulation in a smartwatch","volume-title":"Proceedings of the 37th annual ACM symposium on user interface software and technology","author":"Takahashi","year":"2024"},{"key":"B64","doi-asserted-by":"publisher","first-page":"99","DOI":"10.2307\/3001913","article-title":"Comparing individual means in the analysis of variance","volume":"5","author":"Tukey","year":"1949","journal-title":"Biometrics"},{"key":"B65","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.neucom.2020.09.017","article-title":"A review on transfer learning in EEG signal analysis","volume":"421","author":"Wan","year":"2021","journal-title":"Neurocomputing"},{"key":"B66","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1109\/MetroXRAINE62247.2024.10796452","article-title":"Design of a virtual reality-based neuroadaptive system for treatment of arachnophobia","volume-title":"2024 IEEE international conference on metrology for eXtended reality, artificial intelligence and neural engineering (MetroXRAINE)","author":"Weber","year":"2024"},{"key":"B67","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1109\/tcds.2020.3007453","article-title":"Transfer learning for EEG-Based brain\u2013computer interfaces: a review of progress made since 2016","volume":"14","author":"Wu","year":"2022","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"B68","doi-asserted-by":"publisher","first-page":"806517","DOI":"10.3389\/fnhum.2022.806517","article-title":"Error-related potentials in reinforcement learning-based brain-machine interfaces","volume":"16","author":"Xavier Fid\u00eancio","year":"2022","journal-title":"Front. Hum. Neurosci."},{"key":"B69","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/j.neucom.2021.06.064","article-title":"Accelerating reinforcement learning using EEG-Based implicit human feedback","volume":"460","author":"Xu","year":"2021","journal-title":"Neurocomputing"},{"key":"B70","doi-asserted-by":"publisher","first-page":"14898","DOI":"10.1073\/pnas.1605155114","article-title":"Neuroadaptive technology enables implicit cursor control based on medial prefrontal cortex activity","volume":"113","author":"Zander","year":"2016","journal-title":"Proc. Natl. Acad. Sci. U. S. A."}],"container-title":["Frontiers in Virtual Reality"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frvir.2025.1616442\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T05:23:38Z","timestamp":1754889818000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frvir.2025.1616442\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,11]]},"references-count":70,"alternative-id":["10.3389\/frvir.2025.1616442"],"URL":"https:\/\/doi.org\/10.3389\/frvir.2025.1616442","relation":{},"ISSN":["2673-4192"],"issn-type":[{"value":"2673-4192","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,11]]},"article-number":"1616442"}}