{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T15:10:35Z","timestamp":1776179435924,"version":"3.50.1"},"reference-count":28,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["RETTL-2118380"],"award-info":[{"award-number":["RETTL-2118380"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this study, we developed a new haptic\u2013mixed reality intravenous (HMR-IV) needle insertion simulation system, providing a bimanual haptic interface integrated into a mixed reality system with programmable variabilities considering real clinical environments. The system was designed for nursing students or healthcare professionals to practice IV needle insertion into a virtual arm with unlimited attempts under various changing insertion conditions (e.g., skin: color, texture, stiffness, friction; vein: size, shape, location depth, stiffness, friction). To achieve accurate hand\u2013eye coordination under dynamic mixed reality scenarios, two different haptic devices (Dexmo and Geomagic Touch) and a standalone mixed reality system (HoloLens 2) were integrated and synchronized through multistep calibration for different coordinate systems (real world, virtual world, mixed reality world, haptic interface world, HoloLens camera). In addition, force-profile-based haptic rendering proposed in this study was able to successfully mimic the real tactile feeling of IV needle insertion. Further, a global hand-tracking method, combining two depth sensors (HoloLens and Leap Motion), was developed to accurately track a haptic glove and simulate grasping a virtual hand with force feedback. We conducted an evaluation study with 20 participants (9 experts and 11 novices) to measure the usability of the HMR-IV simulation system with user performance under various insertion conditions. The quantitative results from our own metric and qualitative results from the NASA Task Load Index demonstrate the usability of our system.<\/jats:p>","DOI":"10.3390\/s23156697","type":"journal-article","created":{"date-parts":[[2023,7,27]],"date-time":"2023-07-27T02:14:48Z","timestamp":1690424088000},"page":"6697","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Bimanual Intravenous Needle Insertion Simulation Using Nonhomogeneous Haptic Device Integrated into Mixed Reality"],"prefix":"10.3390","volume":"23","author":[{"given":"Jin Woo","family":"Kim","sequence":"first","affiliation":[{"name":"Computer Science, Kent State University, 800 E Summit St, Kent, OH 44240, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jeremy","family":"Jarzembak","sequence":"additional","affiliation":[{"name":"College of Nursing, Kent State University, 800 E Summit St, Kent, OH 44240, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6830-4655","authenticated-orcid":false,"given":"Kwangtaek","family":"Kim","sequence":"additional","affiliation":[{"name":"Computer Science, Kent State University, 800 E Summit St, Kent, OH 44240, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1097\/01.NAJ.0000541440.91369.00","article-title":"Making It Stick","volume":"118","author":"Ehrhardt","year":"2018","journal-title":"AJN Am. J. Nurs."},{"key":"ref_2","unstructured":"Josephson, D.L., and Keegan, L. (1999). Intravenous Infusion Therapy for Nurses, Cengage Learning."},{"key":"ref_3","unstructured":"Wang, X., and Fenster, A. (2004, January 15\u201318). A Virtual Reality Based 3D Real-Time Interactive Brachytherapy Simulation of Needle Insertion and Seed Implantation. Proceedings of the IEEE International Symposium on Biomedical Imaging: Nano to Macro, Arlington, VA, USA."},{"key":"ref_4","first-page":"36","article-title":"A Virtual Reality Simulator for Ultrasound-Guided Biopsy Training","volume":"31","author":"Ni","year":"2009","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Shin, S., Park, W., Cho, H., Park, S., and Kim, L. (2011, January 9\u201314). Needle Insertion Simulator with Haptic Feedback. Proceedings of the Human-Computer Interaction Techniques and Environments: 14th International Conference, HCI International 2011, Orlando, FL, USA.","DOI":"10.1007\/978-3-642-21605-3_13"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1108\/01445151111172961","article-title":"Design and Implementation of a Haptic-Based Virtual Assembly System","volume":"31","author":"Xia","year":"2011","journal-title":"Assem. Autom."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3009","DOI":"10.1109\/TBME.2012.2236091","article-title":"An Augmented Reality Haptic Training Simulator for Spinal Needle Procedures","volume":"60","author":"Sutherland","year":"2013","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"101731","DOI":"10.1016\/j.compmedimag.2020.101731","article-title":"PI\u00d1ATA: Pinpoint Insertion of Intravenous Needles via Augmented Reality Training Assistance","volume":"82","author":"Mendes","year":"2020","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1109\/TOH.2011.32","article-title":"Integrating Haptics with Augmented Reality in a Femoral Palpation and Needle Insertion Training Simulation","volume":"4","author":"Coles","year":"2011","journal-title":"IEEE Trans. Haptics"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1109\/TOH.2015.2445768","article-title":"Direct Visuo-Haptic 4D Volume Rendering Using Respiratory Motion Models","volume":"8","author":"Fortmeier","year":"2015","journal-title":"IEEE Trans. Haptics"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5783","DOI":"10.1007\/s00500-019-04341-4","article-title":"Development and Assessment of a Haptic-Enabled Holographic Surgical Simulator for Renal Biopsy Training","volume":"24","author":"Guo","year":"2020","journal-title":"Soft Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"18620","DOI":"10.1038\/s41598-020-75676-4","article-title":"Augmented Reality Improves Procedural Efficiency and Reduces Radiation Dose for CT-Guided Lesion Targeting: A Phantom Study Using HoloLens 2","volume":"10","author":"Park","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_13","first-page":"260","article-title":"Augmenting the Training Space of an Epidural Needle Insertion Simulator with HoloLens","volume":"10","author":"Costa","year":"2021","journal-title":"Comput. Methods Biomech. Biomed. Eng. Imaging Vis."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1007\/s11548-021-02341-0","article-title":"Psychomotor Skills Development for Veress Needle Placement Using a Virtual Reality and Haptics-Based Simulator","volume":"16","author":"Vece","year":"2021","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hart, S.G. (2006, January 16\u201320). Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, USA.","DOI":"10.1037\/e577632012-009"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Si, W., Liao, X., Wang, Q., and Heng, P.-A. (2018, January 30). Augmented Reality-Based Personalized Virtual Operative Anatomy for Neurosurgical Guidance and Training. Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Tuebingen\/Reutlingen, Germany.","DOI":"10.1109\/VR.2018.8446450"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Vaughan, N., John, N., and Rees, N. (2019, January 2\u20134). ParaVR: Paramedic Virtual Reality Training Simulator. Proceedings of the 2019 International Conference on Cyberworlds, Kyoto, Japan.","DOI":"10.1109\/CW.2019.00012"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ortegon, T., Acosta, D.A., Salgado, S., Mino, W., Moo-Young, J., Luk, D., Smiley, C., Tsiliopoulos, T., Yang, J., and Avil\u00e9s, O.F. (2019, January 11\u201313). Prototyping Interactive Multimodal vr Epidural Administration. Proceedings of the IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA.","DOI":"10.1109\/ICCE.2019.8662013"},{"key":"ref_19","first-page":"1","article-title":"Physically-based shading at Disney","volume":"2012","author":"Burley","year":"2012","journal-title":"ACM Siggraph"},{"key":"ref_20","unstructured":"Itkowitz, B.D., Handley, J., and Zhu, W. (2005, January 18\u201320). The OpenHapticsTM Toolkit: A Library for Adding 3D TouchTM Navigation and Haptics to Graphics Applications. Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference, Pisa, Italy."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press.","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1016\/0009-2614(85)80574-1","article-title":"A Broyden\u2014Fletcher\u2014Goldfarb\u2014Shanno Optimization Procedure for Molecular Geometries","volume":"122","author":"Head","year":"1985","journal-title":"Chem. Phys. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gu, X., Zhang, Y., Sun, W., Bian, Y., Zhou, D., and Kristensson, P.O. (2016, January 7\u201312). Dexmo. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA.","DOI":"10.1145\/2858036.2858487"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ruspini, D., Kolarov, K.D., and Khatib, O. (1997, January 3\u20138). The Haptic Display of Complex Graphical Environments. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA.","DOI":"10.1145\/258734.258878"},{"key":"ref_25","unstructured":"Zilles, C.B., and Salisbury, J.K. (1995, January 5\u20139). A Constraint-Based God-Object Method for Haptic Display. Proceedings of the 1995 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Pittsburgh, PA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Cohen, I., Huang, Y., Chen, J., Benesty, J., Benesty, J., Chen, J., Huang, Y., and Cohen, I. (2009). Noise Reduction in Speech Processing, Springer.","DOI":"10.1007\/978-3-642-00296-0"},{"key":"ref_27","unstructured":"Engen, T. (1988). States of Brain and Mind, Birkh\u00e4user."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1097\/NAN.0000000000000100","article-title":"Accepted but unacceptable: Peripheral IV catheter failure","volume":"38","author":"Helm","year":"2015","journal-title":"J. Infus. Nurs."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6697\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:19:32Z","timestamp":1760127572000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6697"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,26]]},"references-count":28,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["s23156697"],"URL":"https:\/\/doi.org\/10.3390\/s23156697","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,26]]}}}