{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T03:36:59Z","timestamp":1784691419559,"version":"3.55.0"},"reference-count":157,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,4,5]],"date-time":"2023-04-05T00:00:00Z","timestamp":1680652800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Robotic manipulation challenges, such as grasping and object manipulation, have been tackled successfully with the help of deep reinforcement learning systems. We give an overview of the recent advances in deep reinforcement learning algorithms for robotic manipulation tasks in this review. We begin by outlining the fundamental ideas of reinforcement learning and the parts of a reinforcement learning system. The many deep reinforcement learning algorithms, such as value-based methods, policy-based methods, and actor\u2013critic approaches, that have been suggested for robotic manipulation tasks are then covered. We also examine the numerous issues that have arisen when applying these algorithms to robotics tasks, as well as the various solutions that have been put forth to deal with these issues. Finally, we highlight several unsolved research issues and talk about possible future directions for the subject.<\/jats:p>","DOI":"10.3390\/s23073762","type":"journal-article","created":{"date-parts":[[2023,4,6]],"date-time":"2023-04-06T01:10:27Z","timestamp":1680743427000},"page":"3762","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":226,"title":["A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation"],"prefix":"10.3390","volume":"23","author":[{"given":"Dong","family":"Han","sequence":"first","affiliation":[{"name":"School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beni","family":"Mulyana","sequence":"additional","affiliation":[{"name":"School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1075-2420","authenticated-orcid":false,"given":"Vladimir","family":"Stankovic","sequence":"additional","affiliation":[{"name":"Department of Electronic and Electrical Engineering, University of Straclyde, Glasglow G1 1XW, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5439-1137","authenticated-orcid":false,"given":"Samuel","family":"Cheng","sequence":"additional","affiliation":[{"name":"School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s12599-014-0334-4","article-title":"Industry 4.0","volume":"6","author":"Lasi","year":"2014","journal-title":"Bus. Inf. Syst. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sigov, A., Ratkin, L., Ivanov, L.A., and Xu, L.D. (2022). Emerging enabling technologies for Industry 4.0 and beyond. Inf. Syst. Front., 1\u201311.","DOI":"10.1007\/s10796-021-10213-w"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hua, J., Zeng, L., Li, G., and Ju, Z. (2021). Learning for a robot: Deep reinforcement learning, imitation learning, transfer learning. Sensors, 21.","DOI":"10.3390\/s21041278"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1146\/annurev-control-060117-104848","article-title":"Toward Robotic Manipulation","volume":"1","author":"Mason","year":"2018","journal-title":"Annu. Rev. Control. Robot. Auton. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2651","DOI":"10.32604\/csse.2023.031720","article-title":"Reinforcement Learning with an Ensemble of Binary Action Deep Q-Networks","volume":"46","author":"Hafiz","year":"2023","journal-title":"Comput. Syst. Sci. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hafiz, A.M., Hassaballah, M., and Binbusayyis, A. (2023). Formula-Driven Supervised Learning in Computer Vision: A Literature Survey. Appl. Sci., 13.","DOI":"10.3390\/app13020723"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1007\/s11370-021-00398-z","article-title":"A survey on deep learning and deep reinforcement learning in robotics with a tutorial on deep reinforcement learning","volume":"14","author":"Morales","year":"2021","journal-title":"Intell. Serv. Robot."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/MCS.2022.3216653","article-title":"Shared Control of Robot Manipulators With Obstacle Avoidance: A Deep Reinforcement Learning Approach","volume":"43","author":"Rubagotti","year":"2023","journal-title":"IEEE Control. Syst. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"ref_10","unstructured":"Zejnullahu, F., Moser, M., and Osterrieder, J. (2022). Applications of Reinforcement Learning in Finance\u2014Trading with a Double Deep Q-Network. arXiv."},{"key":"ref_11","unstructured":"Ramamurthy, R., Ammanabrolu, P., Brantley, K., Hessel, J., Sifa, R., Bauckhage, C., Hajishirzi, H., and Choi, Y. (2023). Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"eaap7885","DOI":"10.1126\/sciadv.aap7885","article-title":"Deep reinforcement learning for de novo drug design","volume":"4","author":"Popova","year":"2018","journal-title":"Sci. Adv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kiran, B.R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A.A.A., Yogamani, S., and P\u00e9rez, P. (2021). Deep Reinforcement Learning for Autonomous Driving: A Survey. arXiv.","DOI":"10.1109\/TITS.2021.3054625"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"102517","DOI":"10.1016\/j.rcim.2022.102517","article-title":"A review on reinforcement learning for contact-rich robotic manipulation tasks","volume":"81","author":"Chrysostomou","year":"2023","journal-title":"Robot. Comput. Integr. Manuf."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1109\/JAS.2016.7471613","article-title":"Where does AlphaGo go: From church-turing thesis to AlphaGo thesis and beyond","volume":"3","author":"Wang","year":"2016","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_17","unstructured":"Rummery, G.A., and Niranjan, M. (1994). On-Line Q-Learning Using Connectionist Systems, University of Cambridge, Department of Engineering."},{"key":"ref_18","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, the, Phoenix Convention Center, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_20","unstructured":"Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 19\u201324). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York City, NY, USA."},{"key":"ref_21","unstructured":"Sutton, R.S., McAllester, D.A., Singh, S.P., and Mansour, Y. (2000, January 28\u201330). Policy gradient methods for reinforcement learning with function approximation. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_22","unstructured":"Schulman, J., Levine, S., Abbeel, P., Jordan, M., and Moritz, P. (2015, January 6\u201311). Trust region policy optimization. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_23","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_24","unstructured":"Konda, V.R., and Tsitsiklis, J.N. (2000, January 28\u201330). Actor-critic algorithms. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_25","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous methods for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York City, NY, USA."},{"key":"ref_26","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_28","unstructured":"Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. (2014, January 21\u201326). Deterministic Policy Gradient Algorithms. Proceedings of the International Conference on Machine Learning, Beijing, China."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1103\/PhysRev.36.823","article-title":"On the theory of the Brownian motion","volume":"36","author":"Uhlenbeck","year":"1930","journal-title":"Phys. Rev."},{"key":"ref_30","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing function approximation error in actor-critic methods. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_31","unstructured":"Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., Kumar, V., Zhu, H., Gupta, A., and Abbeel, P. (2018). Soft actor-critic algorithms and applications. arXiv."},{"key":"ref_32","unstructured":"Haarnoja, T., Tang, H., Abbeel, P., and Levine, S. (2017, January 6\u201311). Reinforcement learning with deep energy-based policies. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_33","unstructured":"Pomerleau, D.A. (1989). Artificial Intelligence and Psychology, Carnegie-Mellon University. Technical Report."},{"key":"ref_34","unstructured":"Christiano, P., Leike, J., Brown, T.B., Martic, M., Legg, S., and Amodei, D. (2017). Deep reinforcement learning from human preferences. arXiv."},{"key":"ref_35","unstructured":"Ng, A.Y., and Russell, S.J. (July, January 29). Algorithms for inverse reinforcement learning. Proceedings of the International Conference on Machine Learning, Stanford, CA, USA."},{"key":"ref_36","unstructured":"Ziebart, B.D., Maas, A.L., Bagnell, J.A., and Dey, A.K. (2008, January 13\u201317). Maximum entropy inverse reinforcement learning. Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence, Chicago, IL, USA."},{"key":"ref_37","unstructured":"Ramachandran, D., and Amir, E. (2007, January 6\u201312). Bayesian Inverse Reinforcement Learning. Proceedings of the International Joint Conference on Artificial Intelligence, Hyderabad, India."},{"key":"ref_38","first-page":"4565","article-title":"Generative adversarial imitation learning","volume":"29","author":"Ho","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_39","unstructured":"Ding, Y., Florensa, C., Phielipp, M., and Abbeel, P. (2019). Goal-conditioned imitation learning. arXiv."},{"key":"ref_40","unstructured":"Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., and Zaremba, W. (2017). Hindsight experience replay. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009, January 14\u201318). Curriculum learning. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.","DOI":"10.1145\/1553374.1553380"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3732","DOI":"10.1109\/TNNLS.2019.2934906","article-title":"Teacher\u2013Student curriculum learning","volume":"31","author":"Matiisen","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_43","unstructured":"Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., and Fergus, R. (2017). Intrinsic motivation and automatic curricula via asymmetric self-play. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artif. Intell."},{"key":"ref_45","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Rumelhart, D.E., Hinton, G.E., and Williams, R.J. (1985). Learning Internal Representations by Error Propagation, Institute for Cognitive Science, California University of San Diego. Technical Report.","DOI":"10.21236\/ADA164453"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1109\/72.572108","article-title":"Supervised neural networks for the classification of structures","volume":"8","author":"Sperduti","year":"1997","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_49","unstructured":"Morris, C., Ritzert, M., Fey, M., Hamilton, W.L., Lenssen, J.E., Rattan, G., and Grohe, M. (February, January 27). Weisfeiler and leman go neural: Higher-order graph neural networks. Proceedings of the the Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_50","unstructured":"Hamilton, W.L., Ying, R., and Leskovec, J. (2017, January 4\u20139). Inductive representation learning on large graphs. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_51","unstructured":"Li, Y., Tarlow, D., Brockschmidt, M., and Zemel, R. (2015). Gated graph sequence neural networks. arXiv."},{"key":"ref_52","unstructured":"Veli\u010dkovi\u0107, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (2017). Graph attention networks. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: A survey","volume":"32","author":"Kober","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Peng, X.B., Andrychowicz, M., Zaremba, W., and Abbeel, P. (2018, January 21\u201326). Sim-to-real transfer of robotic control with dynamics randomization. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/0278364919887447","article-title":"Learning dexterous in-hand manipulation","volume":"39","author":"Andrychowicz","year":"2020","journal-title":"Int. J. Robot. Res."},{"key":"ref_56","unstructured":"Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Wiele, T., Mnih, V., Heess, N., and Springenberg, J.T. (2018, January 10\u201315). Learning by playing solving sparse reward tasks from scratch. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_57","unstructured":"Vecerik, M., Hester, T., Scholz, J., Wang, F., Pietquin, O., Piot, B., Heess, N., Roth\u00f6rl, T., Lampe, T., and Riedmiller, M. (2017). Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards. arXiv."},{"key":"ref_58","unstructured":"Kilinc, O., Hu, Y., and Montana, G. (2019). Reinforcement learning for robotic manipulation using simulated locomotion demonstrations. arXiv."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Chen, H. (2021, January 2\u20139). Robotic Manipulation with Reinforcement Learning, State Representation Learning, and Imitation Learning (Student Abstract). Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence, Online.","DOI":"10.1609\/aaai.v35i18.17881"},{"key":"ref_60","unstructured":"Zhang, M., Jian, P., Wu, Y., Xu, H., and Wang, X. (2021). DAIR: Disentangled Attention Intrinsic Regularization for Safe and Efficient Bimanual Manipulation. arXiv."},{"key":"ref_61","unstructured":"Yamada, J., Lee, Y., Salhotra, G., Pertsch, K., Pflueger, M., Sukhatme, G.S., Lim, J.J., and Englert, P. (2020). Motion planner augmented reinforcement learning for robot manipulation in obstructed environments. arXiv."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Yang, X., Ji, Z., Wu, J., and Lai, Y.K. (2021). An Open-Source Multi-Goal Reinforcement Learning Environment for Robotic Manipulation with Pybullet. arXiv.","DOI":"10.1007\/978-3-030-89177-0_2"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"2194","DOI":"10.1109\/LRA.2021.3061308","article-title":"Improved learning of robot manipulation tasks via tactile intrinsic motivation","volume":"6","author":"Vulin","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_64","unstructured":"Silver, T., Allen, K., Tenenbaum, J., and Kaelbling, L. (2018). Residual policy learning. arXiv."},{"key":"ref_65","first-page":"57","article-title":"Learning to control a low-cost manipulator using data-efficient reinforcement learning","volume":"Volume 7","author":"Deisenroth","year":"2011","journal-title":"Robotics: Science and Systems VII"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Li, R., Jabri, A., Darrell, T., and Agrawal, P. (August, January 31). Towards practical multi-object manipulation using relational reinforcement learning. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Online.","DOI":"10.1109\/ICRA40945.2020.9197468"},{"key":"ref_67","unstructured":"Popov, I., Heess, N., Lillicrap, T., Hafner, R., Barth-Maron, G., Vecerik, M., Lampe, T., Tassa, Y., Erez, T., and Riedmiller, M. (2017). Data-efficient deep reinforcement learning for dexterous manipulation. arXiv."},{"key":"ref_68","unstructured":"Rusu, A.A., Ve\u010der\u00edk, M., Roth\u00f6rl, T., Heess, N., Pascanu, R., and Hadsell, R. (2017, January 13\u201315). Sim-to-real robot learning from pixels with progressive nets. Proceedings of the Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., and Abbeel, P. (2018, January 21\u201326). Overcoming exploration in reinforcement learning with demonstrations. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8463162"},{"key":"ref_70","unstructured":"OpenAI, Plappert, M., Sampedro, R., Xu, T., Akkaya, I., Kosaraju, V., Welinder, P., D\u2019Sa, R., Petron, A., and Pinto, H.P.d.O. (2021). Asymmetric self-play for automatic goal discovery in robotic manipulation. arXiv."},{"key":"ref_71","unstructured":"Zhan, A., Zhao, P., Pinto, L., Abbeel, P., and Laskin, M. (2020). A Framework for Efficient Robotic Manipulation. arXiv."},{"key":"ref_72","unstructured":"Franceschetti, A., Tosello, E., Castaman, N., and Ghidoni, S. (2020, January 29\u201331). Robotic arm control and task training through deep reinforcement learning. Intelligent Autonomous Systems 16, Proceedings of the 16th International Conference IAS-16, Singapore."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"012026","DOI":"10.1088\/1742-6596\/2216\/1\/012026","article-title":"A Method of Robot Grasping Based on Reinforcement Learning","volume":"2216","author":"Lu","year":"2022","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"4488","DOI":"10.1109\/LRA.2022.3150024","article-title":"Residual learning from demonstration: Adapting dmps for contact-rich manipulation","volume":"7","author":"Davchev","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_75","doi-asserted-by":"crossref","unstructured":"Zhang, X., Jin, S., Wang, C., Zhu, X., and Tomizuka, M. (2022, January 23\u201327). Learning insertion primitives with discrete-continuous hybrid action space for robotic assembly tasks. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811973"},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Zhao, T.Z., Luo, J., Sushkov, O., Pevceviciute, R., Heess, N., Scholz, J., Schaal, S., and Levine, S. (2022, January 23\u201327). Offline meta-reinforcement learning for industrial insertion. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9812312"},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1177\/09544054221100004","article-title":"Impedance control and parameter optimization of surface polishing robot based on reinforcement learning","volume":"237","author":"Ding","year":"2023","journal-title":"Proc. Inst. Mech. Eng. Part B J. Eng. Manuf."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"104006","DOI":"10.1016\/j.autcon.2021.104006","article-title":"Robotic architectural assembly with tactile skills: Simulation and optimization","volume":"133","author":"Belousov","year":"2022","journal-title":"Autom. Constr."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"1387","DOI":"10.1109\/LRA.2021.3140127","article-title":"Manipulation planning from demonstration via goal-conditioned prior action primitive decomposition and alignment","volume":"7","author":"Lin","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"829437","DOI":"10.3389\/fnbot.2022.829437","article-title":"Reinforcement learning with vision-proprioception model for robot planar pushing","volume":"16","author":"Cong","year":"2022","journal-title":"Front. Neurorobot."},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1007\/s11370-021-00402-6","article-title":"Object manipulation system based on image-based reinforcement learning","volume":"15","author":"Kim","year":"2022","journal-title":"Intell. Serv. Robot."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Nasiriany, S., Liu, H., and Zhu, Y. (2022, January 23\u201327). Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9812140"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Anand, A.S., Myrestrand, M.H., and Gravdahl, J.T. (2022, January 9\u201312). Evaluation of variable impedance-and hybrid force\/motioncontrollers for learning force tracking skills. Proceedings of the 2022 IEEE\/SICE International Symposium on System Integration (SII), Online.","DOI":"10.1109\/SII52469.2022.9708826"},{"key":"ref_84","unstructured":"Deisenroth, M., and Rasmussen, C.E. (July, January 28). PILCO: A model-based and data-efficient approach to policy search. Proceedings of the 28th International Conference on Machine Learning (ICML-11), Bellevue, WA, USA."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.neunet.2022.04.003","article-title":"Context meta-reinforcement learning via neuromodulation","volume":"152","author":"Dick","year":"2022","journal-title":"Neural Netw."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1146\/annurev-control-042920-020211","article-title":"Safe learning in robotics: From learning-based control to safe reinforcement learning","volume":"5","author":"Brunke","year":"2022","journal-title":"Annu. Rev. Control. Robot. Auton. Syst."},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Wabersich, K.P., and Zeilinger, M.N. (2018, January 17\u201319). Linear model predictive safety certification for learning-based control. Proceedings of the 2018 IEEE Conference on Decision and Control (CDC), Miami Beach, FL, USA.","DOI":"10.1109\/CDC.2018.8619829"},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Beyene, S.W., and Han, J.H. (2022). Prioritized Hindsight with Dual Buffer for Meta-Reinforcement Learning. Electronics, 11.","DOI":"10.3390\/electronics11244192"},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Shao, Q., Qi, J., Ma, J., Fang, Y., Wang, W., and Hu, J. (2020). Object detection-based one-shot imitation learning with an RGB-D camera. Appl. Sci., 10.","DOI":"10.3390\/app10030803"},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Ho, D., Rao, K., Xu, Z., Jang, E., Khansari, M., and Bai, Y. (June, January 30). Retinagan: An object-aware approach to sim-to-real transfer. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561157"},{"key":"ref_91","unstructured":"Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsell, R. (2016). Progressive neural networks. arXiv."},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Sadeghi, F., Toshev, A., Jang, E., and Levine, S. (2017). Sim2real view invariant visual servoing by recurrent control. arXiv.","DOI":"10.1109\/CVPR.2018.00493"},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Gu, S., Holly, E., Lillicrap, T., and Levine, S. (June, January 29). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Marina Bay Sands, Singapore.","DOI":"10.1109\/ICRA.2017.7989385"},{"key":"ref_94","doi-asserted-by":"crossref","unstructured":"Sun, C., Orbik, J., Devin, C., Yang, B., Gupta, A., Berseth, G., and Levine, S. (2021). Fully Autonomous Real-World Reinforcement Learning for Mobile Manipulation. arXiv.","DOI":"10.1109\/ICDL49984.2021.9515637"},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Ding, Z., Tsai, Y.Y., Lee, W.W., and Huang, B. (2021). Sim-to-Real Transfer for Robotic Manipulation with Tactile Sensory. arXiv.","DOI":"10.1109\/IROS51168.2021.9636259"},{"key":"ref_96","unstructured":"Duan, Y., Andrychowicz, M., Stadie, B.C., Ho, J., Schneider, J., Sutskever, I., Abbeel, P., and Zaremba, W. (2017). One-shot imitation learning. arXiv."},{"key":"ref_97","unstructured":"Finn, C., Yu, T., Zhang, T., Abbeel, P., and Levine, S. (2017, January 13\u201315). One-shot visual imitation learning via meta-learning. Proceedings of the Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_98","doi-asserted-by":"crossref","unstructured":"Yu, T., Finn, C., Xie, A., Dasari, S., Zhang, T., Abbeel, P., and Levine, S. (2018). One-shot imitation from observing humans via domain-adaptive meta-learning. arXiv.","DOI":"10.15607\/RSS.2018.XIV.002"},{"key":"ref_99","unstructured":"Finn, C., Abbeel, P., and Levine, S. (2017, January 6\u201311). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_100","unstructured":"Wang, Z., Merel, J., Reed, S., Wayne, G., de Freitas, N., and Heess, N. (2017). Robust imitation of diverse behaviors. arXiv."},{"key":"ref_101","doi-asserted-by":"crossref","unstructured":"Zhou, A., Kim, M.J., Wang, L., Florence, P., and Finn, C. (2023). NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis. arXiv.","DOI":"10.1109\/CVPR52729.2023.01717"},{"key":"ref_102","doi-asserted-by":"crossref","unstructured":"Li, K., Chappell, D., and Rojas, N. (2023). Immersive Demonstrations are the Key to Imitation Learning. arXiv.","DOI":"10.1109\/ICRA48891.2023.10160560"},{"key":"ref_103","unstructured":"Tong, D., Choi, A., Terzopoulos, D., Joo, J., and Jawed, M.K. (2023). Deep Learning of Force Manifolds from the Simulated Physics of Robotic Paper Folding. arXiv."},{"key":"ref_104","doi-asserted-by":"crossref","unstructured":"Zhang, D., Fan, W., Lloyd, J., Yang, C., and Lepora, N.F. (2022). One-Shot Domain-Adaptive Imitation Learning via Progressive Learning Applied to Robotic Pouring. arXiv.","DOI":"10.1109\/TASE.2022.3220728"},{"key":"ref_105","doi-asserted-by":"crossref","unstructured":"Yi, J.B., Kim, J., Kang, T., Song, D., Park, J., and Yi, S.J. (2022). Anthropomorphic Grasping of Complex-Shaped Objects Using Imitation Learning. Appl. Sci., 12.","DOI":"10.3390\/app122412861"},{"key":"ref_106","first-page":"414","article-title":"An adaptive imitation learning framework for robotic complex contact-rich insertion tasks","volume":"8","author":"Wang","year":"2022","journal-title":"Front. Robot."},{"key":"ref_107","unstructured":"von Hartz, J.O., Chisari, E., Welschehold, T., and Valada, A. (2022). Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation. arXiv."},{"key":"ref_108","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Aytar, Y., and Bousmalis, K. (2021). Manipulator-independent representations for visual imitation. arXiv.","DOI":"10.15607\/RSS.2021.XVII.002"},{"key":"ref_109","doi-asserted-by":"crossref","unstructured":"Jung, E., and Kim, I. (2021). Hybrid imitation learning framework for robotic manipulation tasks. Sensors, 21.","DOI":"10.3390\/s21103409"},{"key":"ref_110","doi-asserted-by":"crossref","unstructured":"Bong, J.H., Jung, S., Kim, J., and Park, S. (2022). Standing Balance Control of a Bipedal Robot Based on Behavior Cloning. Biomimetics, 7.","DOI":"10.3390\/biomimetics7040232"},{"key":"ref_111","unstructured":"Shafiullah, N.M.M., Cui, Z.J., Altanzaya, A., and Pinto, L. (2022). Behavior Transformers: Cloning k modes with one stone. arXiv."},{"key":"ref_112","unstructured":"Piche, A., Pardinas, R., Vazquez, D., Mordatch, I., and Pal, C. (2022). Implicit Offline Reinforcement Learning via Supervised Learning. arXiv."},{"key":"ref_113","unstructured":"Shridhar, M., Manuelli, L., and Fox, D. (2022). Perceiver-actor: A multi-task transformer for robotic manipulation. arXiv."},{"key":"ref_114","unstructured":"Wang, Q., McCarthy, R., Bulens, D.C., and Redmond, S.J. (2023). Winning Solution of Real Robot Challenge III. arXiv."},{"key":"ref_115","unstructured":"Finn, C., Levine, S., and Abbeel, P. (2016, January 19\u201324). Guided cost learning: Deep inverse optimal control via policy optimization. Proceedings of the International Conference on Machine Learning, New York City, NY, USA."},{"key":"ref_116","doi-asserted-by":"crossref","unstructured":"Zhao, X., Xia, L., Zhang, L., Ding, Z., Yin, D., and Tang, J. (2018, January 2). Deep reinforcement learning for page-wise recommendations. Proceedings of the 12th ACM Conference on Recommender Systems, Vancouver, BC, Canada.","DOI":"10.1145\/3240323.3240374"},{"key":"ref_117","unstructured":"Li, X., Ma, Y., and Belta, C. (2018). Automata guided reinforcement learning with demonstrations. arXiv."},{"key":"ref_118","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1080\/01691864.2018.1509018","article-title":"Hierarchical reinforcement learning of multiple grasping strategies with human instructions","volume":"32","author":"Osa","year":"2018","journal-title":"Adv. Robot."},{"key":"ref_119","unstructured":"Zhang, J., Yu, H., and Xu, W. (2021). Hierarchical reinforcement learning by discovering intrinsic options. arXiv."},{"key":"ref_120","unstructured":"Baram, N., Anschel, O., Caspi, I., and Mannor, S. (2017, January 6\u201311). End-to-end differentiable adversarial imitation learning. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_121","unstructured":"Merel, J., Tassa, Y., TB, D., Srinivasan, S., Lemmon, J., Wang, Z., Wayne, G., and Heess, N. (2017). Learning human behaviors from motion capture by adversarial imitation. arXiv."},{"key":"ref_122","doi-asserted-by":"crossref","first-page":"104264","DOI":"10.1016\/j.robot.2022.104264","article-title":"Goal-aware generative adversarial imitation learning from imperfect demonstration for robotic cloth manipulation","volume":"158","author":"Tsurumine","year":"2022","journal-title":"Robot. Auton. Syst."},{"key":"ref_123","unstructured":"Zolna, K., Reed, S., Novikov, A., Colmenarejo, S.G., Budden, D., Cabi, S., Denil, M., de Freitas, N., and Wang, Z. (2021, January 8\u201311). Task-relevant adversarial imitation learning. Proceedings of the Conference on Robot Learning, London, UK."},{"key":"ref_124","doi-asserted-by":"crossref","unstructured":"Yang, X., Ji, Z., Wu, J., and Lai, Y.K. (2022, January 1\u20133). Abstract demonstrations and adaptive exploration for efficient and stable multi-step sparse reward reinforcement learning. Proceedings of the 2022 27th International Conference on Automation and Computing (ICAC), Bristol, UK.","DOI":"10.1109\/ICAC55051.2022.9911100"},{"key":"ref_125","doi-asserted-by":"crossref","unstructured":"Li, Y., Kong, T., Li, L., Li, Y., and Wu, Y. (2021). Learning to Design and Construct Bridge without Blueprint. arXiv.","DOI":"10.1109\/IROS51168.2021.9636280"},{"key":"ref_126","doi-asserted-by":"crossref","unstructured":"Puang, E.Y., Tee, K.P., and Jing, W. (December, January 25). Kovis: Keypoint-based visual servoing with zero-shot sim-to-real transfer for robotics manipulation. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Online.","DOI":"10.1109\/IROS45743.2020.9341370"},{"key":"ref_127","doi-asserted-by":"crossref","unstructured":"Yuan, C., Shi, Y., Feng, Q., Chang, C., Liu, M., Chen, Z., Knoll, A.C., and Zhang, J. (2022, January 5\u20139). Sim-to-Real Transfer of Robotic Assembly with Visual Inputs Using CycleGAN and Force Control. Proceedings of the 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO), Xishuangbanna, China.","DOI":"10.1109\/ROBIO55434.2022.10011878"},{"key":"ref_128","unstructured":"Tiboni, G., Arndt, K., and Kyrki, V. (2022). DROPO: Sim-to-Real Transfer with Offline Domain Randomization. arXiv."},{"key":"ref_129","doi-asserted-by":"crossref","first-page":"8964","DOI":"10.1109\/LRA.2022.3189156","article-title":"Randomized-to-Canonical Model Predictive Control for Real-World Visual Robotic Manipulation","volume":"7","author":"Yamanokuchi","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_130","unstructured":"Julian, R., Swanson, B., Sukhatme, G.S., Levine, S., Finn, C., and Hausman, K. (2020, January 13\u201318). Efficient adaptation for end-to-end vision-based robotic manipulation. Proceedings of the 4th Lifelong Machine Learning Workshop at ICML, Online."},{"key":"ref_131","unstructured":"Rammohan, S., Yu, S., He, B., Hsiung, E., Rosen, E., Tellex, S., and Konidaris, G. (2021). Value-Based Reinforcement Learning for Continuous Control Robotic Manipulation in Multi-Task Sparse Reward Settings. arXiv."},{"key":"ref_132","unstructured":"Wang, D., and Walters, R. (2022, January 25\u201329). So (2) equivariant reinforcement learning. Proceedings of the International Conference on Learning Representations, Online."},{"key":"ref_133","doi-asserted-by":"crossref","unstructured":"Deng, Y., Guo, D., Guo, X., Zhang, N., Liu, H., and Sun, F. (2020). MQA: Answering the question via robotic manipulation. arXiv.","DOI":"10.15607\/RSS.2021.XVII.044"},{"key":"ref_134","doi-asserted-by":"crossref","unstructured":"Imtiaz, M.B., Qiao, Y., and Lee, B. (2023). Prehensile and Non-Prehensile Robotic Pick-and-Place of Objects in Clutter Using Deep Reinforcement Learning. Sensors, 23.","DOI":"10.3390\/s23031513"},{"key":"ref_135","doi-asserted-by":"crossref","unstructured":"Sarantopoulos, I., Kiatos, M., Doulgeri, Z., and Malassiotis, S. (August, January 31). Split deep q-learning for robust object singulation. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Online.","DOI":"10.1109\/ICRA40945.2020.9196647"},{"key":"ref_136","doi-asserted-by":"crossref","unstructured":"Hsu, H.L., Huang, Q., and Ha, S. (2022, January 23\u201327). Improving safety in deep reinforcement learning using unsupervised action planning. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9812181"},{"key":"ref_137","doi-asserted-by":"crossref","unstructured":"Iriondo, A., Lazkano, E., Susperregi, L., Urain, J., Fernandez, A., and Molina, J. (2019). Pick and place operations in logistics using a mobile manipulator controlled with deep reinforcement learning. Appl. Sci., 9.","DOI":"10.3390\/app9020348"},{"key":"ref_138","doi-asserted-by":"crossref","first-page":"eaau5872","DOI":"10.1126\/scirobotics.aau5872","article-title":"Learning agile and dynamic motor skills for legged robots","volume":"4","author":"Hwangbo","year":"2019","journal-title":"Sci. Robot."},{"key":"ref_139","unstructured":"Clegg, A., Yu, W., Tan, J., Kemp, C.C., Turk, G., and Liu, C.K. (2017). Learning human behaviors for robot-assisted dressing. arXiv."},{"key":"ref_140","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."},{"key":"ref_141","doi-asserted-by":"crossref","unstructured":"Wang, Q., Sanchez, F.R., McCarthy, R., Bulens, D.C., McGuinness, K., O\u2019Connor, N., W\u00fcthrich, M., Widmaier, F., Bauer, S., and Redmond, S.J. (2022). Dexterous robotic manipulation using deep reinforcement learning and knowledge transfer for complex sparse reward-based tasks. Expert Syst., e13205.","DOI":"10.1111\/exsy.13205"},{"key":"ref_142","doi-asserted-by":"crossref","first-page":"51996","DOI":"10.1109\/ACCESS.2021.3069975","article-title":"Hindsight Goal Ranking on Replay Buffer for Sparse Reward Environment","volume":"9","author":"Luu","year":"2021","journal-title":"IEEE Access"},{"key":"ref_143","doi-asserted-by":"crossref","unstructured":"Eppe, M., Magg, S., and Wermter, S. (2019, January 19\u201322). Curriculum goal masking for continuous deep reinforcement learning. Proceedings of the 2019 Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), Norway.","DOI":"10.1109\/DEVLRN.2019.8850721"},{"key":"ref_144","doi-asserted-by":"crossref","unstructured":"Sehgal, A., La, H., Louis, S., and Nguyen, H. (2019, January 25\u201327). Deep reinforcement learning using genetic algorithm for parameter optimization. Proceedings of the 2019 Third IEEE International Conference on Robotic Computing (IRC), Naples, Italy.","DOI":"10.1109\/IRC.2019.00121"},{"key":"ref_145","doi-asserted-by":"crossref","unstructured":"Sehgal, A., Ward, N., La, H., and Louis, S. (2022). Automatic parameter optimization using genetic algorithm in deep reinforcement learning for robotic manipulation tasks. arXiv.","DOI":"10.1109\/IRC55401.2022.00022"},{"key":"ref_146","unstructured":"Nair, A.V., Pong, V., Dalal, M., Bahl, S., Lin, S., and Levine, S. (2018). Visual reinforcement learning with imagined goals. arXiv."},{"key":"ref_147","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1109\/JSAC.2022.3221993","article-title":"Sampling, communication, and prediction co-design for synchronizing the real-world device and digital model in metaverse","volume":"41","author":"Meng","year":"2022","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_148","doi-asserted-by":"crossref","first-page":"1115","DOI":"10.1017\/S0263574722001527","article-title":"Discrete soft actor-critic with auto-encoder on vascular robotic system","volume":"41","author":"Li","year":"2022","journal-title":"Robotica"},{"key":"ref_149","unstructured":"Wang, D., Jia, M., Zhu, X., Walters, R., and Platt, R. (2022, January 14\u201318). On-robot learning with equivariant models. Proceedings of the Conference on Robot Learning, Auckland, New Zealand."},{"key":"ref_150","doi-asserted-by":"crossref","unstructured":"Jian, P., Yang, C., Guo, D., Liu, H., and Sun, F. (June, January 30). Adversarial Skill Learning for Robust Manipulation. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561379"},{"key":"ref_151","unstructured":"Janisch, J., Pevn\u1ef3, T., and Lis\u1ef3, V. (2020). Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks. arXiv."},{"key":"ref_152","unstructured":"Almasan, P., Su\u00e1rez-Varela, J., Badia-Sampera, A., Rusek, K., Barlet-Ros, P., and Cabellos-Aparicio, A. (2019). Deep reinforcement learning meets graph neural networks: Exploring a routing optimization use case. arXiv."},{"key":"ref_153","doi-asserted-by":"crossref","unstructured":"Lin, Y., Wang, A.S., Undersander, E., and Rai, A. (2021). Efficient and interpretable robot manipulation with graph neural networks. arXiv.","DOI":"10.1109\/LRA.2022.3143518"},{"key":"ref_154","unstructured":"Sieb, M., Xian, Z., Huang, A., Kroemer, O., and Fragkiadaki, K. (2020, January 16\u201318). Graph-structured visual imitation. Proceedings of the Conference on Robot Learning, Online."},{"key":"ref_155","first-page":"2327","article-title":"Deep imitation learning for bimanual robotic manipulation","volume":"33","author":"Xie","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_156","unstructured":"Liang, J., and Boularias, A. (2022). Learning Category-Level Manipulation Tasks from Point Clouds with Dynamic Graph CNNs. arXiv."},{"key":"ref_157","doi-asserted-by":"crossref","unstructured":"Oliva, M., Banik, S., Josifovski, J., and Knoll, A. (2022, January 18\u201323). Graph Neural Networks for Relational Inductive Bias in Vision-based Deep Reinforcement Learning of Robot Control. Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy.","DOI":"10.1109\/IJCNN55064.2022.9892101"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3762\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:10:40Z","timestamp":1760123440000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3762"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,5]]},"references-count":157,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23073762"],"URL":"https:\/\/doi.org\/10.3390\/s23073762","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,5]]}}}