{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,3]],"date-time":"2026-08-03T17:43:34Z","timestamp":1785779014297,"version":"3.56.0"},"reference-count":51,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T00:00:00Z","timestamp":1717632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Surg."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Surgical robots are gaining increasing popularity because of their capability to improve the precision of pedicle screw placement. However, current surgical robots rely on unimodal computed tomography (CT) images as baseline images, limiting their visualization to vertebral bone structures and excluding soft tissue structures such as intervertebral discs and nerves. This inherent limitation significantly restricts the applicability of surgical robots. To address this issue and further enhance the safety and accuracy of robot-assisted pedicle screw placement, this study will develop a software system for surgical robots based on multimodal image fusion. Such a system can extend the application range of surgical robots, such as surgical channel establishment, nerve decompression, and other related operations.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Initially, imaging data of the patients included in the study are collected. Professional workstations are employed to establish, train, validate, and optimize algorithms for vertebral bone segmentation in CT and magnetic resonance (MR) images, intervertebral disc segmentation in MR images, nerve segmentation in MR images, and registration fusion of CT and MR images. Subsequently, a spine application model containing independent modules for vertebrae, intervertebral discs, and nerves is constructed, and a software system for surgical robots based on multimodal image fusion is designed. Finally, the software system is clinically validated.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>We will develop a software system based on multimodal image fusion for surgical robots, which can be applied to surgical access establishment, nerve decompression, and other operations not only for robot-assisted nail placement. The development of this software system is important. First, it can improve the accuracy of pedicle screw placement, percutaneous vertebroplasty, percutaneous kyphoplasty, and other surgeries. Second, it can reduce the number of fluoroscopies, shorten the operation time, and reduce surgical complications. In addition, it would be helpful to expand the application range of surgical robots by providing key imaging data for surgical robots to realize surgical channel establishment, nerve decompression, and other operations.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fsurg.2024.1389244","type":"journal-article","created":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T19:25:36Z","timestamp":1718133936000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Development of a software system for surgical robots based on multimodal image fusion: study protocol"],"prefix":"10.3389","volume":"11","author":[{"given":"Shuo","family":"Yuan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruiyuan","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Zang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aobo","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ning","family":"Fan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"Du","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Xi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianyi","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,6,6]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"E123","DOI":"10.1093\/ons\/opy200","article-title":"Robotic-Assisted spine surgery using the mazor XTM system: 2-dimensional operative video","volume":"16","author":"Tan","year":"2019","journal-title":"Oper Neurosurg (Hagerstown)"},{"key":"B2","doi-asserted-by":"publisher","first-page":"367","DOI":"10.14245\/ns.2143126.563","article-title":"Learning curve of ROSA ONE spine system for transpedicular screw placement","volume":"19","author":"Hsu","year":"2022","journal-title":"Neurospine"},{"key":"B3","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1007\/s11684-019-0716-6","article-title":"New spinal robotic technologies","volume":"13","author":"Jiang","year":"2019","journal-title":"Front Med"},{"key":"B4","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1186\/s13018-022-03271-6","article-title":"Safety and risk factors of TINAVI robot-assisted percutaneous pedicle screw placement in spinal surgery","volume":"17","author":"Zhang","year":"2022","journal-title":"J Orthop Surg Res"},{"key":"B5","doi-asserted-by":"publisher","first-page":"86","DOI":"10.21037\/atm-2020-ioi-07","article-title":"The current state of navigation in robotic spine surgery","volume":"9","author":"Huang","year":"2021","journal-title":"Ann Transl Med"},{"key":"B6","doi-asserted-by":"publisher","first-page":"S10","DOI":"10.14444\/8136","article-title":"The clinical impact of image guidance and robotics in spinal surgery: a review of safety, accuracy, efficiency, and complication reduction","volume":"15","author":"Campbell","year":"2021","journal-title":"Int J Spine Surg"},{"key":"B7","doi-asserted-by":"publisher","first-page":"824","DOI":"10.21037\/atm-20-1106","article-title":"Accuracy of robot-assisted versus conventional freehand pedicle screw placement in spine surgery: a systematic review and meta-analysis of randomized controlled trials","volume":"8","author":"Peng","year":"2020","journal-title":"Ann Transl Med"},{"key":"B8","doi-asserted-by":"publisher","first-page":"E10","DOI":"10.3171\/2021.10.Focus21509","article-title":"Evaluating robotic pedicle screw placement against conventional modalities: a systematic review and network meta-analysis","volume":"52","author":"Naik","year":"2022","journal-title":"Neurosurg Focus"},{"key":"B9","doi-asserted-by":"publisher","first-page":"101421","DOI":"10.1016\/j.eclinm.2022.101421","article-title":"Association of robot-assisted techniques with the accuracy rates of pedicle screw placement: a network pooling analysis","volume":"48","author":"Wei","year":"2022","journal-title":"EClinicalMedicine"},{"key":"B10","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1186\/s12891-022-05206-y","article-title":"Accuracy and safety of robot-assisted cortical bone trajectory screw placement: a comparison of robot-assisted technique with fluoroscopy-assisted approach","volume":"23","author":"Li","year":"2022","journal-title":"BMC Musculoskelet Disord"},{"key":"B11","doi-asserted-by":"publisher","first-page":"E14","DOI":"10.3171\/2017.3.Focus179","article-title":"Accuracy of robot-guided versus freehand fluoroscopy-assisted pedicle screw insertion in thoracolumbar spinal surgery","volume":"42","author":"Molliqaj","year":"2017","journal-title":"Neurosurg Focus"},{"key":"B12","doi-asserted-by":"publisher","first-page":"E1692","DOI":"10.1097\/brs.0000000000003701","article-title":"Risk of postoperative complications and revision surgery following robot-assisted posterior lumbar spinal fusion","volume":"45","author":"Yang","year":"2020","journal-title":"Spine (Phila Pa 1976)"},{"key":"B13","doi-asserted-by":"publisher","first-page":"53","DOI":"10.19723\/j.issn.1671-167X.2019.01.010","article-title":"Multimodal image fusion technology for diagnosis and treatment of the skull base-infratemporal tumors","volume":"51","author":"Yang","year":"2019","journal-title":"Beijing Da Xue Xue Bao Yi Xue Ban"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1007\/s11548-021-02429-7","article-title":"A convenient and stable vertebrae instance segmentation method for transforaminal endoscopic surgery planning","volume":"16","author":"Peng","year":"2021","journal-title":"Int J Comput Assist Radiol Surg"},{"key":"B15","author":"Lessmann","year":"2020"},{"key":"B16","first-page":"105","author":"Ledig","year":"2017"},{"key":"B17","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1097\/01.brs.0000197188.76369.13","article-title":"Complications in spinal fusion for adolescent idiopathic scoliosis in the new millennium. A report of the scoliosis research society morbidity and mortality committee","volume":"31","author":"Coe","year":"2006","journal-title":"Spine (Phila Pa 1976)"},{"key":"B18","doi-asserted-by":"publisher","first-page":"594","DOI":"10.1007\/s00586-002-0469-8","article-title":"Complications of pedicle screws in lumbar and lumbosacral fusions in 105 consecutive primary operations","volume":"11","author":"Jutte","year":"2002","journal-title":"Eur Spine J"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1320","DOI":"10.1097\/00007632-199606010-00008","article-title":"Accuracy of pedicle screw placement in lumbar vertebrae","volume":"21","author":"Castro","year":"1996","journal-title":"Spine (Phila Pa 1976)"},{"key":"B20","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1007\/s00586-009-1091-9","article-title":"Image-guided spine surgery: state of the art and future directions","volume":"19","author":"Tjardes","year":"2010","journal-title":"Eur Spine J"},{"key":"B21","doi-asserted-by":"publisher","first-page":"E8","DOI":"10.3171\/2011.7.Focus11168","article-title":"Clinically relevant complications related to pedicle screw placement in thoracolumbar surgery and their management: a literature review of 35,630 pedicle screws","volume":"31","author":"Gautschi","year":"2011","journal-title":"Neurosurg Focus"},{"key":"B22","doi-asserted-by":"publisher","first-page":"452","DOI":"10.4103\/0019-5413.159670","article-title":"Robot assisted navigated drilling for percutaneous pedicle screw placement: a preliminary animal study","volume":"49","author":"Wang","year":"2015","journal-title":"Indian J Orthop"},{"key":"B23","doi-asserted-by":"publisher","first-page":"e429","DOI":"10.1016\/j.wneu.2019.01.097","article-title":"Robot-Assisted versus fluoroscopy-guided pedicle screw placement in transforaminal lumbar interbody fusion for lumbar degenerative disease","volume":"125","author":"Zhang","year":"2019","journal-title":"World Neurosurg"},{"key":"B24","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1007\/s00586-015-3758-8","article-title":"Robot-assisted spine surgery: feasibility study through a prospective case-matched analysis","volume":"25","author":"Lonjon","year":"2016","journal-title":"Eur Spine J"},{"key":"B25","doi-asserted-by":"publisher","first-page":"E111","DOI":"10.1097\/brs.0000000000003193","article-title":"Accuracy of pedicle screw placement and clinical outcomes of robot-assisted technique versus conventional freehand technique in spine surgery from nine randomized controlled trials: a meta-analysis","volume":"45","author":"Li","year":"2020","journal-title":"Spine (Phila Pa 1976)"},{"key":"B26","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1016\/j.jot.2019.09.002","article-title":"A preliminary study of a novel robotic system for pedicle screw fixation: a randomised controlled trial","volume":"20","author":"Li","year":"2020","journal-title":"J Orthop Translat"},{"key":"B27","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1007\/s00586-011-1729-2","article-title":"Perioperative course and accuracy of screw positioning in conventional, open robotic-guided and percutaneous robotic-guided, pedicle screw placement","volume":"20","author":"Kantelhardt","year":"2011","journal-title":"Eur Spine J"},{"key":"B28","doi-asserted-by":"publisher","first-page":"e625","DOI":"10.1016\/j.wneu.2020.05.257","article-title":"Comparing cortical bone trajectories for pedicle screw insertion using robotic guidance and three-dimensional computed tomography navigation","volume":"141","author":"Khan","year":"2020","journal-title":"World Neurosurg"},{"key":"B29","doi-asserted-by":"publisher","first-page":"e10970","DOI":"10.1097\/md.0000000000010970","article-title":"Accuracy of pedicle screw placement comparing robot-assisted technology and the free-hand with fluoroscopy-guided method in spine surgery: an updated meta-analysis","volume":"97","author":"Fan","year":"2018","journal-title":"Medicine (Baltimore)"},{"key":"B30","doi-asserted-by":"publisher","first-page":"620","DOI":"10.1177\/2192568220960406","article-title":"Comparison of accuracy and clinical outcomes of robot-assisted versus fluoroscopy-guided pedicle screw placement in posterior cervical surgery","volume":"12","author":"Su","year":"2022","journal-title":"Global Spine J"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1256","DOI":"10.14245\/ns.2346610.305","article-title":"Spine surgical robotics: current status and recent clinical applications","volume":"20","author":"Wang","year":"2023","journal-title":"Neurospine"},{"key":"B32","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1007\/s10143-023-02027-y","article-title":"Clinical application of spinal robot in cervical spine surgery: safety and accuracy of posterior pedicle screw placement in comparison with conventional freehand methods","volume":"46","author":"Zhou","year":"2023","journal-title":"Neurosurg Rev"},{"key":"B33","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1186\/s13018-021-02211-0","article-title":"Effectiveness of tirobot-assisted vertebroplasty in treating thoracolumbar osteoporotic compression fracture","volume":"16","author":"Wang","year":"2021","journal-title":"J Orthop Surg Res"},{"key":"B34","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s00264-022-05580-1","article-title":"TiRobot-assisted percutaneous kyphoplasty in the management of multilevel (more than three levels) osteoporotic vertebral compression fracture","volume":"47","author":"Lin","year":"2023","journal-title":"Int Orthop"},{"key":"B35","first-page":"49","article-title":"Percutaneous endoscopic lumbar discectomy: indications and complications","volume":"23","author":"Pan","year":"2020","journal-title":"Pain Physician"},{"key":"B36","first-page":"3121","article-title":"Two-dimensional fluoroscopy-guided robot-assisted percutaneous endoscopic transforaminal discectomy: a retrospective cohort study","volume":"14","author":"Yang","year":"2022","journal-title":"Am J Transl Res"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1067","DOI":"10.1002\/jso.26999","article-title":"Experience in utilizing a novel 3D digital model with CT and MRI fusion data in sarcoma evaluation and surgical planning","volume":"126","author":"Du","year":"2022","journal-title":"J Surg Oncol"},{"key":"B38","doi-asserted-by":"publisher","first-page":"e18187","DOI":"10.7759\/cureus.18187","article-title":"The utility of augmented reality in spinal decompression surgery using CT\/MRI fusion image","volume":"13","author":"Aoyama","year":"2021","journal-title":"Cureus"},{"key":"B39","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1007\/s00256-009-0788-5","article-title":"3-D MRI\/CT fusion imaging of the lumbar spine","volume":"39","author":"Yamanaka","year":"2010","journal-title":"Skeletal Radiol"},{"key":"B40","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1016\/j.media.2019.02.005","article-title":"Iterative fully convolutional neural networks for automatic vertebra segmentation and identification","volume":"53","author":"Lessmann","year":"2019","journal-title":"Med Image Anal"},{"key":"B41","doi-asserted-by":"publisher","first-page":"115010","DOI":"10.1088\/1361-6560\/aac226","article-title":"Sparse intervertebral fence composition for 3D cervical vertebra segmentation","volume":"63","author":"Liu","year":"2018","journal-title":"Phys Med Biol"},{"key":"B42","doi-asserted-by":"publisher","first-page":"1663","DOI":"10.1109\/tmi.2015.2443912","article-title":"Statistical interspace models (SIMs): application to robust 3D spine segmentation","volume":"34","author":"Castro-Mateos","year":"2015","journal-title":"IEEE Trans Med Imaging"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1676","DOI":"10.1109\/tmi.2015.2392054","article-title":"Multi-modality vertebra recognition in arbitrary views using 3D deformable hierarchical model","volume":"34","author":"Cai","year":"2015","journal-title":"IEEE Trans Med Imaging"},{"key":"B44","doi-asserted-by":"publisher","first-page":"2375","DOI":"10.1109\/tbme.2013.2256460","article-title":"Simultaneous localization of lumbar vertebrae and intervertebral discs with SVM-based MRF","volume":"60","author":"Oktay","year":"2013","journal-title":"IEEE Trans Biomed Eng"},{"key":"B45","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1007\/s10916-019-1357-7","article-title":"Intervertebral disc segmentation and diagnostic application based on wavelet denoising and AAM model in human spine image","volume":"43","author":"Yang","year":"2019","journal-title":"J Med Syst"},{"key":"B46","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.media.2018.01.004","article-title":"3D multi-scale FCN with random modality voxel dropout learning for intervertebral disc localization and segmentation from multi-modality MR images","volume":"45","author":"Li","year":"2018","journal-title":"Med Image Anal"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1012","DOI":"10.3390\/biom10071012","article-title":"Nerve segmentation with deep learning from label-free endoscopic images obtained using coherent anti-stokes Raman scattering","volume":"10","author":"Yamato","year":"2020","journal-title":"Biomolecules"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1771","DOI":"10.1007\/s00586-010-1430-x","article-title":"MR\/CT image fusion of the spine after spondylodesis: a feasibility study","volume":"19","author":"Karlo","year":"2010","journal-title":"Eur Spine J"},{"key":"B49","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.compbiomed.2018.09.003","article-title":"Multi-segmental spine image registration supporting image-guided interventions of spinal metastases","volume":"102","author":"Hille","year":"2018","journal-title":"Comput Biol Med"},{"key":"B50","doi-asserted-by":"publisher","first-page":"2413","DOI":"10.1109\/tmi.2016.2576360","article-title":"MIND demons: symmetric diffeomorphic deformable registration of MR and CT for image-guided spine surgery","volume":"35","author":"Reaungamornrat","year":"2016","journal-title":"IEEE Trans Med Imaging"},{"key":"B51","doi-asserted-by":"publisher","first-page":"1762","DOI":"10.1002\/jmri.28735","article-title":"LSW-Net: lightweight deep neural network based on small-world properties for spine MR image segmentation","volume":"58","author":"He","year":"2023","journal-title":"J Magn Reson Imaging"}],"container-title":["Frontiers in Surgery"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fsurg.2024.1389244\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T19:25:42Z","timestamp":1718133942000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fsurg.2024.1389244\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,6]]},"references-count":51,"alternative-id":["10.3389\/fsurg.2024.1389244"],"URL":"https:\/\/doi.org\/10.3389\/fsurg.2024.1389244","relation":{},"ISSN":["2296-875X"],"issn-type":[{"value":"2296-875X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,6]]},"article-number":"1389244"}}