{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T11:12:24Z","timestamp":1775819544027,"version":"3.50.1"},"reference-count":44,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T00:00:00Z","timestamp":1731456000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000968","name":"American Heart Association","doi-asserted-by":"publisher","award":["23CDA1040585"],"award-info":[{"award-number":["23CDA1040585"]}],"id":[{"id":"10.13039\/100000968","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>In developing medical interventions using untethered milli- and microrobots, ensuring safety and effectiveness relies on robust methods for real-time robot detection, tracking, and precise localization within the body. The inherent non-transparency of human tissues significantly challenges these efforts, as traditional imaging systems like fluoroscopy often lack crucial anatomical details, potentially compromising intervention safety and efficacy. To address this technological gap, in this study, we build a virtual reality environment housing an exact digital replica (digital twin) of the operational workspace and a robot avatar. We synchronize the virtual and real workspaces and continuously send the robot position data derived from the image stream into the digital twin with short average delay time around 20\u201325 ms. This allows the operator to steer the robot by tracking its avatar within the digital twin with near real-time temporal resolution. We demonstrate the feasibility of this approach with millirobots steered in confined phantoms. Our concept demonstration herein can pave the way for not only improved procedural safety by complementing fluoroscopic guidance with virtual reality enhancement, but also provides a platform for incorporating various additional real-time derivative data, e.g., instantaneous robot velocity, intraoperative physiological data obtained from the patient, e.g., blood flow rate, and pre-operative physical simulation models, e.g., periodic body motions, to further refine robot control capacity.<\/jats:p>","DOI":"10.3389\/frobt.2024.1495445","type":"journal-article","created":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T11:29:03Z","timestamp":1731497343000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["X-ray fluoroscopy guided localization and steering of miniature robots using virtual reality enhancement"],"prefix":"10.3389","volume":"11","author":[{"given":"Husnu Halid","family":"Alabay","sequence":"first","affiliation":[]},{"given":"Tuan-Anh","family":"Le","sequence":"additional","affiliation":[]},{"given":"Hakan","family":"Ceylan","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,11,13]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"10865","DOI":"10.1021\/acsnano.0c05530","article-title":"Medical imaging of microrobots: toward in vivo applications","volume":"14","author":"Aziz","year":"2020","journal-title":"ACS Nano"},{"key":"B2","doi-asserted-by":"publisher","first-page":"14082","DOI":"10.1038\/s41598-021-93323-4","article-title":"Navigation of a magnetic micro-robot through a cerebral aneurysm phantom with magnetic particle imaging","volume":"11","author":"Bakenecker","year":"2021","journal-title":"Sci. Rep."},{"key":"B3","doi-asserted-by":"publisher","first-page":"798","DOI":"10.1109\/tro.2012.2188165","article-title":"Visually servoing magnetic intraocular microdevices","volume":"28","author":"Bergeles","year":"2012","journal-title":"IEEE Trans. Robotics"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1705","DOI":"10.1039\/c7lc00064b","article-title":"Mobile microrobots for bioengineering applications","volume":"17","author":"Ceylan","year":"2017","journal-title":"Lab a Chip"},{"key":"B5","doi-asserted-by":"publisher","first-page":"012002","DOI":"10.1088\/2516-1091\/ab22d5","article-title":"Translational prospects of untethered medical microrobots","volume":"1","author":"Ceylan","year":"","journal-title":"Prog. Biomed. Eng."},{"key":"B6","doi-asserted-by":"publisher","first-page":"3353","DOI":"10.1021\/acsnano.8b09233","article-title":"3D-Printed biodegradable microswimmer for theranostic cargo delivery and release","volume":"13","author":"Ceylan","year":"","journal-title":"ACS Nano"},{"key":"B7","doi-asserted-by":"publisher","first-page":"2300793","DOI":"10.1002\/aisy.202300793","article-title":"Virtual reality-enabled intuitive magnetic manipulation of microrobots and nanoparticles","volume":"6","author":"Chowdhury","year":"2024","journal-title":"Adv. Intell. Syst."},{"key":"B8","doi-asserted-by":"publisher","first-page":"5889","DOI":"10.1038\/s41467-023-41557-3","article-title":"Ultrasound trapping and navigation of microrobots in the mouse brain vasculature","volume":"14","author":"Del Campo Fonseca","year":"2023","journal-title":"Nat. Commun."},{"key":"B9","doi-asserted-by":"publisher","first-page":"2204016","DOI":"10.1002\/smll.202204016","article-title":"Remotely guided immunobots engaged in anti-tumorigenic phenotypes for targeted cancer immunotherapy","volume":"18","author":"Dogan","year":"2022","journal-title":"Small"},{"key":"B10","doi-asserted-by":"publisher","first-page":"eabi8017","DOI":"10.1126\/scirobotics.abi8017","article-title":"A decade retrospective of medical robotics research from 2010 to 2020","volume":"6","author":"Dupont","year":"2021","journal-title":"Sci. Robotics"},{"key":"B11","article-title":"Roboflow (version 1.0)","author":"Dwyer","year":"2024","journal-title":"Every. you need to build and deploy comp. vis. model"},{"key":"B12","doi-asserted-by":"publisher","first-page":"2305205","DOI":"10.1002\/adfm.202305205","article-title":"A soft biodegradable chitosan\u2010based medical microrobot with optimized structural design and X\u2010ray visibility for targeted vessel chemoembolization","volume":"33","author":"Go","year":"2023","journal-title":"Adv. Funct. Mater."},{"key":"B13","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-3-030-87266-3_1","volume-title":"Anesthesiology in-training exam review","author":"Hausman","year":"2022"},{"key":"B14","doi-asserted-by":"publisher","first-page":"eabe8436","DOI":"10.1126\/sciadv.abe8436","article-title":"Magnetic soft micromachines made of linked microactuator networks","volume":"7","author":"Hu","year":"2021","journal-title":"Sci. Adv."},{"key":"B15","doi-asserted-by":"publisher","first-page":"e1900709","DOI":"10.1002\/smll.201900709","article-title":"High-resolution SPECT imaging of stimuli-responsive soft microrobots","volume":"15","author":"Iacovacci","year":"","journal-title":"Small"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1900709","DOI":"10.1002\/smll.201900709","article-title":"High\u2010resolution SPECT imaging of stimuli\u2010responsive soft microrobots","volume":"15","author":"Iacovacci","year":"","journal-title":"Small"},{"key":"B17","unstructured":"Ultralytics YOLO\n          \n          \n            \n              Jocher\n              G.\n            \n            \n              Chaurasia\n              A.\n            \n            \n              Qiu\n              J.\n            \n          \n          \n          2023"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1112","DOI":"10.1109\/lra.2018.2792156","article-title":"Mechanical rubbing of blood clots using helical robots under ultrasound guidance","volume":"3","author":"Khalil","year":"2018","journal-title":"Ieee Robotics Automation Lett."},{"key":"B19","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.snb.2019.04.145","article-title":"PDMS double casting method enabled by plasma treatment and alcohol passivation","volume":"293","author":"Kim","year":"2019","journal-title":"Sensors Actuators B Chem."},{"key":"B20","first-page":"290","volume-title":"Sonography principles and instruments","author":"Kremkau","year":"2015"},{"key":"B21","doi-asserted-by":"publisher","first-page":"12759","DOI":"10.1021\/acsami.0c18221","article-title":"3D-Printed multi-stimuli-responsive mobile micromachines","volume":"13","author":"Lee","year":"2021","journal-title":"ACS Appl. Mater. and Interfaces"},{"key":"B22","doi-asserted-by":"publisher","first-page":"eadh8702","DOI":"10.1126\/scirobotics.adh8702","article-title":"Human-scale navigation of magnetic microrobots in hepatic arteries","volume":"9","author":"Li","year":"2024","journal-title":"Sci. Robotics"},{"key":"B23","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1038\/s44172-024-00215-2","article-title":"Ex vivo validation of magnetically actuated intravascular untethered robots in a clinical setting","volume":"3","author":"Ligtenberg","year":"2024","journal-title":"Commun. Eng."},{"key":"B24","doi-asserted-by":"publisher","first-page":"882","DOI":"10.1038\/s41467-021-21232-1","article-title":"High-speed three-dimensional photoacoustic computed tomography for preclinical research and clinical translation","volume":"12","author":"Lin","year":"2021","journal-title":"Nat. Commun."},{"key":"B25","doi-asserted-by":"publisher","first-page":"eadh2479","DOI":"10.1126\/scirobotics.adh2479","article-title":"Magnetic soft microfiberbots for robotic embolization","volume":"9","author":"Liu","year":"2024","journal-title":"Sci. Robot."},{"key":"B26","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1109\/SVR.2019.00049","volume-title":"2019 21st symposium on virtual and augmented reality (SVR)","author":"Melo","year":"2019"},{"key":"B27","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1002\/jmri.27411","article-title":"Real-time magnetic resonance imaging","volume":"55","author":"Nayak","year":"2022","journal-title":"J. Magn. Reson Imaging"},{"key":"B28","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1146\/annurev-bioeng-010510-103409","article-title":"Microrobots for minimally invasive medicine","volume":"12","author":"Nelson","year":"2010","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"B29","first-page":"186","volume-title":"Resolution in ultrasound imaging","author":"Ng","year":"2011"},{"key":"B30","doi-asserted-by":"publisher","first-page":"20160296","DOI":"10.1259\/bjr.20160296","article-title":"Fiducial marker guided prostate radiotherapy: a review","volume":"89","author":"O'Neill","year":"2016","journal-title":"Br. J. Radiol."},{"key":"B31","doi-asserted-by":"publisher","first-page":"1800575","DOI":"10.1002\/admt.201800575","article-title":"Imaging technologies for biomedical micro\u2010 and nanoswimmers","volume":"4","author":"Pan\u00e9","year":"2018","journal-title":"Adv. Mater. Technol."},{"key":"B32","doi-asserted-by":"publisher","first-page":"10901","DOI":"10.1073\/pnas.0403918101","article-title":"MRI detection of single particles for cellular imaging","volume":"101","author":"Shapiro","year":"2004","journal-title":"Proc. Natl. Acad. Sci. U. S. A."},{"key":"B33","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1109\/jproc.2014.2385105","article-title":"Biomedical applications of untethered mobile milli\/microrobots","volume":"103","author":"Sitti","year":"2015","journal-title":"Proc. IEEE"},{"key":"B34","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/j.pacs.2019.05.001","article-title":"Photoacoustic clinical imaging","volume":"14","author":"Steinberg","year":"2019","journal-title":"Photoacoustics"},{"key":"B35","doi-asserted-by":"publisher","first-page":"20552076221149651","DOI":"10.1177\/20552076221149651","article-title":"Digital twin in healthcare: recent updates and challenges","volume":"9","author":"Sun","year":"2023","journal-title":"Digit. Health"},{"key":"B36","doi-asserted-by":"publisher","first-page":"6982","DOI":"10.1109\/lra.2022.3179509","article-title":"Deep learning-based 3D magnetic microrobot tracking using 2D MR images","volume":"7","author":"Tiryaki","year":"2022","journal-title":"Ieee Robotics Automation Lett."},{"key":"B37","doi-asserted-by":"publisher","first-page":"1424931","DOI":"10.3389\/fphys.2024.1424931","article-title":"Model-driven engineering for digital twins: a graph model-based patient simulation application","volume":"15","author":"Trevena","year":"2024","journal-title":"Front. Physiol."},{"key":"B38","doi-asserted-by":"publisher","first-page":"1220","DOI":"10.1021\/acsnano.7b07220","article-title":"Medical imaging for the tracking of micromotors","volume":"12","author":"Vilela","year":"2018","journal-title":"ACS Nano"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1443","DOI":"10.1007\/s00247-023-05597-7","article-title":"SPECT\/CT, PET\/CT and PET\/MRI: oncologic and infectious applications and protocol considerations","volume":"53","author":"Voss","year":"2023","journal-title":"Pediatr. Radiol."},{"key":"B40","doi-asserted-by":"publisher","first-page":"4465","DOI":"10.1038\/s41467-022-32059-9","article-title":"Adaptive wireless millirobotic locomotion into distal vasculature","volume":"13","author":"Wang","year":"2022","journal-title":"Nat. Commun."},{"key":"B41","doi-asserted-by":"publisher","first-page":"eaax0613","DOI":"10.1126\/scirobotics.aax0613","article-title":"A microrobotic system guided by photoacoustic computed tomography for targeted navigation in intestines in vivo","volume":"4","author":"Wu","year":"2019","journal-title":"Sci. Robot."},{"key":"B42","doi-asserted-by":"publisher","first-page":"898426","DOI":"10.3389\/fphys.2022.898426","article-title":"Applications of magnetic particle imaging in biomedicine: advancements and prospects","volume":"13","author":"Yang","year":"2022","journal-title":"Front. Physiol."},{"key":"B43","doi-asserted-by":"publisher","first-page":"1808992","DOI":"10.1002\/adfm.201808992","article-title":"3D-Printed microrobotic transporters with recapitulated stem cell niche for programmable and active cell delivery","volume":"29","author":"Yasa","year":"2019","journal-title":"Adv. Funct. Mater."},{"key":"B44","doi-asserted-by":"publisher","first-page":"2108266","DOI":"10.1002\/adma.202108266","article-title":"Treatment of ruptured and nonruptured aneurysms using a semisolid iodinated embolic agent","volume":"34","author":"Zhang","year":"2022","journal-title":"Adv. Mater."}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1495445\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T11:29:08Z","timestamp":1731497348000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1495445\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,13]]},"references-count":44,"alternative-id":["10.3389\/frobt.2024.1495445"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1495445","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,13]]},"article-number":"1495445"}}