{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T13:55:28Z","timestamp":1777125328958,"version":"3.51.4"},"reference-count":46,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,6,14]],"date-time":"2022-06-14T00:00:00Z","timestamp":1655164800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"SurfClean Inc., Sagamihara, Kanagawa, Japan"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Motor rehabilitation is used to improve motor control skills to improve the patient\u2019s quality of life. Regular adjustments based on the effect of therapy are necessary, but this can be time-consuming for the clinician. This study proposes to use an efficient tool for high-dimensional data by considering a deep learning approach for dimensionality reduction of hand movement recorded using a wireless remote control embedded with the Oculus Rift S. This latent space is created as a visualization tool also for use in a reinforcement learning (RL) algorithm employed to provide a decision-making framework. The data collected consists of motions drawn with wireless remote control in an immersive VR environment for six different motions called \u201cCube\u201d, \u201cCylinder\u201d, \u201cHeart\u201d, \u201cInfinity\u201d, \u201cSphere\u201d, and \u201cTriangle\u201d. From these collected data, different artificial databases were created to simulate variations of the data. A latent space representation is created using an adversarial autoencoder (AAE), taking into account unsupervised (UAAE) and semi-supervised (SSAAE) training. Then, each test point is represented by a distance metric and used as a reward for two classes of Multi-Armed Bandit (MAB) algorithms, namely Boltzmann and Sibling Kalman filters. The results showed that AAE models can represent high-dimensional data in a two-dimensional latent space and that MAB agents can efficiently and quickly learn the distance evolution in the latent space. The results show that Sibling Kalman filter exploration outperforms Boltzmann exploration with an average cumulative weighted probability error of 7.9 versus 19.9 using the UAAE latent space representation and 8.0 versus 20.0 using SSAAE. In conclusion, this approach provides an effective approach to visualize and track current motor control capabilities regarding a target in order to reflect the patient\u2019s abilities in VR games in the context of DDA.<\/jats:p>","DOI":"10.3390\/s22124499","type":"journal-article","created":{"date-parts":[[2022,6,15]],"date-time":"2022-06-15T01:39:54Z","timestamp":1655257194000},"page":"4499","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Adversarial Autoencoder and Multi-Armed Bandit for Dynamic Difficulty Adjustment in Immersive Virtual Reality for Rehabilitation: Application to Hand Movement"],"prefix":"10.3390","volume":"22","author":[{"given":"Kenta","family":"Kamikokuryo","sequence":"first","affiliation":[{"name":"Department of Mechanical Systems Engineering, Tokyo University of Agriculture and Technology, Tokyo 184-0012, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takumi","family":"Haga","sequence":"additional","affiliation":[{"name":"Department of Mechanical Systems Engineering, Tokyo University of Agriculture and Technology, Tokyo 184-0012, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7767-4765","authenticated-orcid":false,"given":"Gentiane","family":"Venture","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, The University of Tokyo, Tokyo 113-8654, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6686-2410","authenticated-orcid":false,"given":"Vincent","family":"Hernandez","sequence":"additional","affiliation":[{"name":"Department of Mechanical Systems Engineering, Tokyo University of Agriculture and Technology, Tokyo 184-0012, Japan"},{"name":"Surfclean Inc., Sagamihara 252-0131, Kanagawa, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,14]]},"reference":[{"key":"ref_1","first-page":"110","article-title":"A Fully Immersive Set-Up for Remote Interaction and Neurorehabilitation Based on Virtual Body Ownership","volume":"3","author":"Solazzi","year":"2012","journal-title":"Front. Neurol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/TNSRE.2007.891388","article-title":"Telerehabilitation Using a Virtual Environment Improves Upper Extremity Function in Patients With Stroke","volume":"15","author":"Holden","year":"2007","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1080\/17434440.2018.1425613","article-title":"Virtual reality in cognitive and motor rehabilitation: Facts, fiction and fallacies","volume":"15","author":"Tieri","year":"2018","journal-title":"Expert Rev. Med. Devices"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2012\/187965","article-title":"Seven Capital Devices for the Future of Stroke Rehabilitation","volume":"2012","author":"Iosa","year":"2012","journal-title":"Stroke Res. Treat."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"415","DOI":"10.2522\/ptj.20130579","article-title":"Emergence of Virtual Reality as a Tool for Upper Limb Rehabilitation: Incorporation of Motor Control and Motor Learning Principles","volume":"95","author":"Levin","year":"2015","journal-title":"Phys. Ther."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1186\/1743-0003-1-12","article-title":"Video capture virtual reality as a flexible and effective rehabilitation tool","volume":"1","author":"Weiss","year":"2004","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/PRES_a_00078","article-title":"Virtual Rehabilitation Environment Using Principles of Intrinsic Motivation and Game Design","volume":"21","author":"Mihelj","year":"2012","journal-title":"Presence Teleoperators Virtual Environ."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kim, W.S., Cho, S., Ku, J., Kim, Y., Lee, K., Hwang, H.J., and Paik, N.J. (2020). Clinical Application of Virtual Reality for Upper Limb Motor Rehabilitation in Stroke: Review of Technologies and Clinical Evidence. J. Clin. Med., 9.","DOI":"10.3390\/jcm9103369"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3549","DOI":"10.1098\/rstb.2009.0138","article-title":"Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments","volume":"364","author":"Slater","year":"2009","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Pinto, J.F., Carvalho, H.R., Chambel, G.R.R., Ramiro, J., and Goncalves, A. (2018, January 16\u201318). Adaptive gameplay and difficulty adjustment in a gamified upper-limb rehabilitation. Proceedings of the 2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH), Vienna, Austria.","DOI":"10.1109\/SeGAH.2018.8401363"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sekhavat, Y.A. (2017, January 2\u20134). MPRL: Multiple-Periodic Reinforcement Learning for difficulty adjustment in rehabilitation games. Proceedings of the 2017 IEEE 5th International Conference on Serious Games and Applications for Health (SeGAH), Perth, Australia.","DOI":"10.1109\/SeGAH.2017.7939260"},{"key":"ref_12","first-page":"205566831664364","article-title":"The use and effect of video game design theory in the creation of game-based systems for upper limb stroke rehabilitation","volume":"3","author":"Barrett","year":"2016","journal-title":"J. Rehabil. Assist. Technol. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"632","DOI":"10.1080\/17483107.2019.1688398","article-title":"Quantitative analysis of the Oculus Rift S in controlled movement","volume":"16","author":"Jost","year":"2021","journal-title":"Disabil. Rehabil. Assist. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Monica, R., and Aleotti, J. (2022). Evaluation of the Oculus Rift S tracking system in room scale virtual reality. Virtual Real.","DOI":"10.1007\/s10055-022-00637-3"},{"key":"ref_15","unstructured":"Csikszentmihalyi, M., and Csikzentmihaly, M. (1990). Flow: The Psychology of Optimal Experience, Harper & Row."},{"key":"ref_16","unstructured":"Andrade, G., Ramalho, G., Gomes, A., and Corruble, V. (2006, January 20\u201323). Dynamic Game Balancing: An Evaluation of User Satisfaction. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Marina del Rey, CA, USA."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1109\/TCIAIG.2011.2158434","article-title":"Dynamic Game Difficulty Scaling Using Adaptive Behavior-Based AI","volume":"3","author":"Tan","year":"2011","journal-title":"IEEE Trans. Comput. Intell. Games"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hunicke, R., and Chapman, V. (2004). AI for Dynamic Difficulty Adjustment in Games, Northwestern University.","DOI":"10.1145\/1178477.1178573"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1186\/1743-0003-7-48","article-title":"Neurorehabilitation using the virtual reality based Rehabilitation Gaming System: Methodology, design, psychometrics, usability and validation","volume":"7","author":"Badia","year":"2010","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_20","first-page":"387","article-title":"Using artificial intelligence to control and adapt level of difficulty in computer-based, cognitive therapy\u2014An explorative study","volume":"4","author":"Wilms","year":"2011","journal-title":"J. Cyberther. Rehabil."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Pirovano, M., Mainetti, R., Baud-Bovy, G., Lanzi, P.L., and Borghese, N.A. (2012, January 11\u201314). Self-adaptive games for rehabilitation at home. Proceedings of the 2012 IEEE Conference on Computational Intelligence and Games (CIG), Granada, Spain.","DOI":"10.1109\/CIG.2012.6374154"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Andrade, K.d.O., Pasqual, T.B., Caurin, G.A.P., and Crocomo, M.K. (2016, January 11\u201313). Dynamic difficulty adjustment with Evolutionary Algorithm in games for rehabilitation robotics. Proceedings of the 2016 IEEE International Conference on Serious Games and Applications for Health (SeGAH), Orlando, FL, USA.","DOI":"10.1109\/SeGAH.2016.7586277"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3180657","article-title":"Evolutionary Algorithms for a Better Gaming Experience in Rehabilitation Robotics","volume":"16","author":"Andrade","year":"2018","journal-title":"Comput. Entertain."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"109684","DOI":"10.1016\/j.jbiomech.2020.109684","article-title":"Adversarial autoencoder for visualization and classification of human activity: Application to a low-cost commercial force plate","volume":"103","author":"Hernandez","year":"2020","journal-title":"J. Biomech."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Boudreault, M., Bouchard, B., Bouchard, K., and Gaboury, S. (2018, January 28\u201330). Maximizing Player Engagement in a Global Warming Sensitization Video Game Through Reinforcement Learning. Proceedings of the 4th EAI International Conference on Smart Objects and Technologies for Social Good, Bologna, Italy.","DOI":"10.1145\/3284869.3284920"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1080\/02687030444000048","article-title":"Outcomes of computer-provided treatment for aphasia","volume":"18","author":"Wertz","year":"2004","journal-title":"Aphasiology"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1080\/713755526","article-title":"Do Specific Attention Deficits Need Specific Training?","volume":"7","author":"Sturm","year":"1997","journal-title":"Neuropsychol. Rehabil."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, J., Monroe, W., Ritter, A., Galley, M., Gao, J., and Jurafsky, D. (2016). Deep Reinforcement Learning for Dialogue Generation. arXiv.","DOI":"10.18653\/v1\/D16-1127"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Vatavu, R.D., Anthony, L., and Wobbrock, J.O. (2012, January 22\u201326). Gestures as point clouds: A $P recognizer for user interface prototypes. Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI\u201912, Santa Monica, CA, USA.","DOI":"10.1145\/2388676.2388732"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1550147717707417","DOI":"10.1177\/1550147717707417","article-title":"Biomechanical parameter assessment for classification of Parkinson\u2019s disease on clinical scale","volume":"13","author":"Butt","year":"2017","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_31","unstructured":"Makhzani, A., Shlens, J., Jaitly, N., Goodfellow, I., and Frey, B. (2015). Adversarial Autoencoders. arXiv."},{"key":"ref_32","unstructured":"Doersch, C. (2016). Tutorial on variational autoencoders. arXiv."},{"key":"ref_33","first-page":"252","article-title":"Convolutional Neural Network Hyper-Parameters Optimization based on Genetic Algorithms","volume":"9","author":"Loussaief","year":"2018","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_34","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., and Devin, M. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv."},{"key":"ref_35","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press. [2nd ed.]."},{"key":"ref_36","unstructured":"Slivkins, A. (2022). Introduction to Multi-Armed Bandits. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1287\/stsy.2019.0033","article-title":"Optimal Exploration\u2013Exploitation in a Multi-armed Bandit Problem with Non-stationary Rewards","volume":"9","author":"Besbes","year":"2019","journal-title":"Stoch. Syst."},{"key":"ref_38","unstructured":"Burtini, G., Loeppky, J., and Lawrence, R. (2015). A Survey of Online Experiment Design with the Stochastic Multi-Armed Bandit. arXiv."},{"key":"ref_39","unstructured":"Garivier, A., and Moulines, E. (2008). On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1111\/tops.12145","article-title":"Uncertainty and exploration in a restless bandit problem","volume":"7","author":"Speekenbrink","year":"2015","journal-title":"Top. Cogn. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Granmo, O.C., and Berg, S. (2010, January 1\u20134). Solving Non-Stationary Bandit Problems by Random Sampling from Sibling Kalman Filters. Proceedings of the 23rd International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, IEA\/AIE 2010, Cordoba, Spain.","DOI":"10.1007\/978-3-642-13033-5_21"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A New Approach to Linear Filtering and Prediction Problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1115\/1.3658902","article-title":"New Results in Linear Filtering and Prediction Theory","volume":"83","author":"Kalman","year":"1961","journal-title":"J. Basic Eng."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/biomet\/25.3-4.285","article-title":"On the Likelihood that One Unknown Probability Exceeds Another in View of the Evidence of Two Samples","volume":"25","author":"Thompson","year":"1933","journal-title":"Biometrika"},{"key":"ref_45","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_46","first-page":"953","article-title":"Artificial neural networks for small dataset analysis","volume":"7","author":"Pasini","year":"2015","journal-title":"J. Thorac. Dis."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4499\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:31:09Z","timestamp":1760139069000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4499"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,14]]},"references-count":46,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22124499"],"URL":"https:\/\/doi.org\/10.3390\/s22124499","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,14]]}}}