{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T16:24:43Z","timestamp":1775665483727,"version":"3.50.1"},"reference-count":67,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T00:00:00Z","timestamp":1747785600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Neurological tremors, prevalent among a large population, are one of the most rampant movement disorders. Biomechanical loading and exoskeletons show promise in enhancing patient well-being, but traditional control algorithms limit their efficacy in dynamic movements and personalized interventions. Furthermore, a pressing need exists for more comprehensive and robust validation methods to ensure the effectiveness and generalizability of proposed solutions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This paper proposes a physical simulation approach modeling multiple arm joints and tremor propagation. This study also introduces a novel adaptable reinforcement learning environment tailored for disorders with tremors. We present a deep reinforcement learning-based encoder-actor controller for Parkinson\u2019s tremors in various shoulder and elbow joint axes displayed in dynamic movements.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Our findings suggest that such a control strategy offers a viable solution for tremor suppression in real-world scenarios.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>By overcoming the limitations of traditional control algorithms, this work takes a new step in adapting biomechanical loading into the everyday life of patients. This work also opens avenues for more adaptive and personalized interventions in managing movement disorders.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frobt.2025.1537470","type":"journal-article","created":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T00:10:47Z","timestamp":1747786247000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Learning to suppress tremors: a deep reinforcement learning-enabled soft exoskeleton for Parkinson\u2019s patients"],"prefix":"10.3389","volume":"12","author":[{"given":"Tam\u00e1s","family":"Endrei","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"S\u00e1ndor","family":"F\u00f6ldi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"\u00c1d\u00e1m","family":"Makk","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gy\u00f6rgy","family":"Cserey","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,5,21]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"3623","DOI":"10.1007\/s00415-023-11705-2","article-title":"Longitudinal brain functional connectivity changes induced by neurosurgical thalamotomy for tremor in Parkinson\u2019s disease: a preliminary study","volume":"270","author":"Albano","year":"2023","journal-title":"J. Neurology"},{"key":"B2","doi-asserted-by":"publisher","first-page":"4109","DOI":"10.1039\/d0bm00809e","article-title":"Nanomaterial based drug delivery systems for the treatment of neurodegenerative diseases","volume":"8","author":"Asil","year":"2020","journal-title":"Biomaterials Sci."},{"key":"B3","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1109\/RoboSoft48309.2020.9115994","article-title":"A novel soft glove for hand tremor suppression: evaluation of layer jamming actuator placement","volume-title":"2020 3rd IEEE international conference on soft robotics (RoboSoft)","author":"Awantha","year":"2020"},{"key":"B4","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1002\/mds.27121","article-title":"Consensus statement on the classification of tremors. from the task force on tremor of the international Parkinson and movement disorder society","volume":"33","author":"Bhatia","year":"2018","journal-title":"Mov. Disord."},{"key":"B5","article-title":"Openai gym","author":"Brockman","year":"2016"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1123188","DOI":"10.3389\/fphar.2023.1123188","article-title":"Role of traditional Chinese medicine in ameliorating mitochondrial dysfunction via non-coding rna signaling: implication in the treatment of neurodegenerative diseases","volume":"14","author":"Cai","year":"2023","journal-title":"Front. Pharmacol."},{"key":"B7","doi-asserted-by":"publisher","first-page":"0810011","DOI":"10.1115\/1.4043442","article-title":"Simulated tremor propagation in the upper limb: from muscle activity to joint displacement","volume":"141","author":"Corie","year":"2019","journal-title":"J. biomechanical Eng."},{"key":"B8","article-title":"Pybullet, a python module for physics simulation for games, robotics and machine learning","author":"Coumans","year":"2016"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1133","DOI":"10.1007\/s10439-016-1765-5","article-title":"Fundamental principles of tremor propagation in the upper limb","volume":"45","author":"Davidson","year":"2017","journal-title":"Ann. Biomed. Eng."},{"key":"B10","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1002\/mds.870131303","article-title":"Consensus statement of the movement disorder society on tremor","volume":"13","author":"Deuschl","year":"1998","journal-title":"Mov. Disord."},{"key":"B11","doi-asserted-by":"publisher","first-page":"178","DOI":"10.3389\/fnins.2017.00178","article-title":"Electrical stimulation of afferent pathways for the suppression of pathological tremor","volume":"11","author":"Dideriksen","year":"2017","journal-title":"Front. Neurosci."},{"key":"B12","doi-asserted-by":"publisher","first-page":"eaar5438","DOI":"10.1126\/scirobotics.aar5438","article-title":"Human-in-the-loop optimization of hip assistance with a soft exosuit during walking","volume":"3","author":"Ding","year":"2018","journal-title":"Sci. robotics"},{"key":"B13","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1109\/tnsre.2014.2328296","article-title":"Online tremor suppression using electromyography and low-level electrical stimulation","volume":"23","author":"Dosen","year":"2014","journal-title":"IEEE Trans. Neural Syst. Rehabilitation Eng."},{"key":"B14","article-title":"Pink noise is all you need: colored noise exploration in deep reinforcement learning","author":"Eberhard","year":"2023"},{"key":"B15","doi-asserted-by":"publisher","first-page":"104410","DOI":"10.1016\/j.bspc.2022.104410","article-title":"Adaptive multi symptoms control of Parkinson\u2019s disease by deep reinforcement learning","volume":"80","author":"Faraji","year":"2023","journal-title":"Biomed. Signal Process. Control"},{"key":"B16","article-title":"Revisiting fundamentals of experience replay","author":"Fedus","year":"2020"},{"key":"B17","unstructured":"For sale: state-action representation learning for deep reinforcement learning\n          \n          61573\n          61624\n          \n            \n              Fujimoto\n              S.\n            \n            \n              Chang\n              W.-D.\n            \n            \n              Smith\n              E.\n            \n            \n              Gu\n              S. S.\n            \n            \n              Precup\n              D.\n            \n            \n              Meger\n              D.\n            \n          \n          Advances in neural information processing systems\n          36\n          2023"},{"key":"B18","first-page":"20132","article-title":"A minimalist approach to offline reinforcement learning","volume":"34","author":"Fujimoto","year":"2021","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B19","first-page":"1587","article-title":"Addressing function approximation error in actor-critic methods","author":"Fujimoto","year":"2018"},{"key":"B20","first-page":"14219","article-title":"An equivalence between loss functions and non-uniform sampling in experience replay","volume":"33","author":"Fujimoto","year":"2020","journal-title":"Adv. neural Inf. Process. Syst."},{"key":"B21","doi-asserted-by":"publisher","first-page":"676","DOI":"10.1186\/s12891-020-03665-9","article-title":"Shoulder range of movement in the general population: age and gender stratified normative data using a community-based cohort","volume":"21","author":"Gill","year":"2020","journal-title":"BMC Musculoskelet. Disord."},{"key":"B22","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1504\/ijhfms.2012.051092","article-title":"Maximum elbow joint torques for digital human models","volume":"3","author":"G\u00fcnzkofer","year":"2012","journal-title":"Int. J. Hum. Factors Model. Simul."},{"key":"B23","doi-asserted-by":"crossref","first-page":"eadi8022","DOI":"10.1126\/scirobotics.adi8022","article-title":"Learning agile soccer skills for a bipedal robot with deep reinforcement learning","volume":"9","author":"Haarnoja","year":"2024","journal-title":"Science Robotics"},{"key":"B24","first-page":"1861","article-title":"Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor","author":"Haarnoja","year":"2018"},{"key":"B25","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1038\/539179a","article-title":"Neurodegenerative diseases","volume":"539","author":"Heemels","year":"2016","journal-title":"Nature"},{"key":"B26","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1186\/s13287-022-03050-4","article-title":"The potential use of mesenchymal stem cells and their exosomes in Parkinson\u2019s disease treatment","volume":"13","author":"Heris","year":"2022","journal-title":"Stem Cell. Res. and Ther."},{"key":"B27","doi-asserted-by":"publisher","first-page":"29","DOI":"10.3389\/fbioe.2016.00029","article-title":"Voluntary-driven elbow orthosis with speed-controlled tremor suppression","volume":"4","author":"Herrnstadt","year":"2016","journal-title":"Front. Bioeng. Biotechnol."},{"key":"B28","doi-asserted-by":"publisher","first-page":"1198","DOI":"10.1038\/s42256-022-00576-3","article-title":"High-speed quadrupedal locomotion by imitation-relaxation reinforcement learning","volume":"4","author":"Jin","year":"2022","journal-title":"Nat. Mach. Intell."},{"key":"B29","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1016\/j.jns.2017.08.3246","article-title":"Tremor\u2019s glove-an innovative electrical muscle stimulation therapy for intractable tremor in Parkinson\u2019s disease: a randomized sham-controlled trial","volume":"381","author":"Jitkritsadakul","year":"2017","journal-title":"J. Neurological Sci."},{"key":"B30","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A new approach to linear filtering and prediction problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng"},{"key":"B31","doi-asserted-by":"publisher","first-page":"982","DOI":"10.1038\/s41586-023-06419-4","article-title":"Champion-level drone racing using deep reinforcement learning","volume":"620","author":"Kaufmann","year":"2023","journal-title":"Nature"},{"key":"B32","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1049\/htl.2013.0030","article-title":"Inverse dynamics modelling of upper-limb tremor, with cross-correlation analysis","volume":"1","author":"Ketteringham","year":"2014","journal-title":"Healthc. Technol. Lett."},{"key":"B33","doi-asserted-by":"publisher","first-page":"1851","DOI":"10.3390\/ijms23031851","article-title":"A review of the common neurodegenerative disorders: current therapeutic approaches and the potential role of nanotherapeutics","volume":"23","author":"Lamptey","year":"2022","journal-title":"Int. J. Mol. Sci."},{"key":"B34","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1056\/nejm199810083391506","article-title":"Medical progress: Parkinson\u2019s disease (first of two parts)","volume":"339","author":"Lang","year":"","journal-title":"N. Engl. J. Med."},{"key":"B35","doi-asserted-by":"publisher","first-page":"1130","DOI":"10.1056\/nejm199810153391607","article-title":"Parkinson\u2019s disease","volume":"339","author":"Lang","year":"","journal-title":"N. Engl. J. Med."},{"key":"B36","doi-asserted-by":"publisher","first-page":"700600","DOI":"10.3389\/fneur.2021.700600","article-title":"A review on wearable technologies for tremor suppression","volume":"12","author":"Lora-Millan","year":"2021","journal-title":"Front. neurology"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1201","DOI":"10.1001\/archneur.1995.00540360079019","article-title":"Differences in the prevalence of essential tremor among elderly african Americans, whites, and hispanics in northern manhattan, NY","volume":"52","author":"Louis","year":"1995","journal-title":"Archives Neurology"},{"key":"B38","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1186\/s12984-023-01147-2","article-title":"Robust walking control of a lower limb rehabilitation exoskeleton coupled with a musculoskeletal model via deep reinforcement learning","volume":"20","author":"Luo","year":"2023","journal-title":"J. neuroengineering rehabilitation"},{"key":"B39","doi-asserted-by":"publisher","first-page":"702845","DOI":"10.3389\/frobt.2021.702845","article-title":"Reinforcement learning and control of a lower extremity exoskeleton for squat assistance","volume":"8","author":"Luo","year":"2021","journal-title":"Front. Robotics AI"},{"key":"B40","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1038\/s41586-024-07382-4","article-title":"Experiment-free exoskeleton assistance via learning in simulation","volume":"630","author":"Luo","year":"2024","journal-title":"Nature"},{"key":"B41","article-title":"Playing atari with deep reinforcement learning","author":"Mnih","year":"2013"},{"key":"B42","doi-asserted-by":"publisher","first-page":"5313","DOI":"10.1007\/s00415-023-11873-1","article-title":"Machine learning for adaptive deep brain stimulation in Parkinson\u2019s disease: closing the loop","volume":"270","author":"Oliveira","year":"2023","journal-title":"J. Neurology"},{"key":"B43","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1177\/036354659001800201","article-title":"Torque production in the shoulder of the normal young adult male: the interaction of function, dominance, joint angle, and angular velocity","volume":"18","author":"Otis","year":"1990","journal-title":"Am. J. sports Med."},{"key":"B44","doi-asserted-by":"publisher","first-page":"1768","DOI":"10.1109\/tbme.2020.3015572","article-title":"Intramuscular stimulation of muscle afferents attains prolonged tremor reduction in essential tremor patients","volume":"68","author":"Pascual-Valdunciel","year":"2020","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B45","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3197517.3201311","article-title":"Deepmimic: example-guided deep reinforcement learning of physics-based character skills","volume":"37","author":"Peng","year":"","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"B46","first-page":"3803","article-title":"Sim-to-real transfer of robotic control with dynamics randomization","author":"Peng","year":""},{"key":"B47","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1080\/02701367.1983.10605290","article-title":"Anatomical data for analyzing human motion","volume":"54","author":"Plagenhoef","year":"1983","journal-title":"Res. Q. Exerc. sport"},{"key":"B48","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/s0165-0270(97)02263-2","article-title":"Adaptive fourier modeling for quantification of tremor","volume":"74","author":"Riviere","year":"1997","journal-title":"J. Neurosci. methods"},{"key":"B49","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1109\/tnsre.2007.903917","article-title":"Design and validation of a rehabilitation robotic exoskeleton for tremor assessment and suppression","volume":"15","author":"Rocon","year":"","journal-title":"IEEE Trans. neural Syst. rehabilitation Eng."},{"key":"B50","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1080\/14734220601103037","article-title":"Mechanical suppression of essential tremor","volume":"6","author":"Rocon","year":"","journal-title":"Cerebellum"},{"key":"B51","unstructured":"Cad2rl: real single-image flight without a single real image\n          \n          \n            \n              Sadeghi\n              F.\n            \n            \n              Levine\n              S.\n            \n          \n          arXiv preprint arXiv:1611.04201\n          \n          2016"},{"key":"B52","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017"},{"key":"B53","first-page":"387","article-title":"Deterministic policy gradient algorithms","author":"Silver","year":"2014"},{"key":"B54","doi-asserted-by":"publisher","first-page":"456","DOI":"10.1038\/s41551-022-00984-1","article-title":"Opportunities and challenges in the development of exoskeletons for locomotor assistance","volume":"7","author":"Siviy","year":"2023","journal-title":"Nat. Biomed. Eng."},{"key":"B55","first-page":"889","article-title":"An experimental investigation of essential hand tremor suppression via a soft exoskeletal glove","author":"Skaramagkas","year":"2020"},{"key":"B56","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1038\/s41586-022-05191-1","article-title":"Personalizing exoskeleton assistance while walking in the real world","volume":"610","author":"Slade","year":"2022","journal-title":"Nature"},{"key":"B57","volume-title":"Reinforcement learning: an introduction","author":"Sutton","year":"2018"},{"key":"B58","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1109\/tnsre.2013.2295034","article-title":"Robust controller for tremor suppression at musculoskeletal level in human wrist","volume":"22","author":"Taheri","year":"2013","journal-title":"IEEE Trans. neural Syst. rehabilitation Eng."},{"key":"B59","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1109\/iros.2017.8202133","article-title":"Domain randomization for transferring deep neural networks from simulation to the real world","author":"Tobin","year":"2017","journal-title":"2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10295","article-title":"Deep reinforcement learning with double q-learning","volume":"30","author":"Van Hasselt","year":"2016","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"B61","doi-asserted-by":"publisher","first-page":"3693","DOI":"10.1109\/lra.2023.3270747","article-title":"Jamming enabled variable stiffness wrist exoskeleton for tremor suppression","volume":"8","author":"Wang","year":"2023","journal-title":"IEEE Robotics Automation Lett."},{"key":"B62","unstructured":"An introduction to the kalman filter\n          \n          \n            \n              Welch\n              G.\n            \n            \n              Bishop\n              G.\n            \n          \n          Chapel Hill, NC, USA\n          University of North Carolina at Chapel Hill\n          \n          1995"},{"key":"B63","doi-asserted-by":"publisher","first-page":"3971","DOI":"10.1109\/tmech.2024.3375911","article-title":"A tremor-suppressing strategy based on the equivalent-input-disturbance approach","volume":"29","author":"Xie","year":"2024","journal-title":"IEEE\/ASME Trans. Mechatronics"},{"key":"B64","first-page":"1115","article-title":"A novel exoskeleton system based on magnetorheological fluid for tremor suppression of wrist joints","author":"Yi","year":"2019"},{"key":"B65","doi-asserted-by":"publisher","first-page":"432","DOI":"10.1089\/soro.2019.0194","article-title":"A soft exoskeleton for tremor suppression equipped with flexible semiactive actuator","volume":"8","author":"Zahedi","year":"2021","journal-title":"Soft Robot."},{"key":"B66","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1016\/j.robot.2016.12.009","article-title":"Design and validation of a novel mechatronic transmission system for a wearable tremor suppression device","volume":"91","author":"Zhou","year":"2017","journal-title":"Robotics Aut. Syst."},{"key":"B67","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1177\/1758573217728711","article-title":"Normative values and affecting factors for the elbow range of motion","volume":"11","author":"Zwerus","year":"2019","journal-title":"Shoulder and Elb."}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1537470\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T00:10:51Z","timestamp":1747786251000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2025.1537470\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,21]]},"references-count":67,"alternative-id":["10.3389\/frobt.2025.1537470"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2025.1537470","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-3824299\/v1","asserted-by":"object"},{"id-type":"doi","id":"10.21203\/rs.3.rs-3824299\/v2","asserted-by":"object"}]},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,21]]},"article-number":"1537470"}}