{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T20:21:04Z","timestamp":1781900464585,"version":"3.54.5"},"reference-count":56,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2022,5,25]],"date-time":"2022-05-25T00:00:00Z","timestamp":1653436800000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"China's National Key Research and Development Program","award":["2019YFB1311300"],"award-info":[{"award-number":["2019YFB1311300"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,6,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Surgical tool detection is a key technology in computer-assisted surgery, and can help surgeons to obtain more comprehensive visual information. Currently, a data shortage problem still exists in surgical tool detection. In addition, some surgical tool detection methods may not strike a good balance between detection accuracy and speed. Given the above problems, in this study a new Cholec80-tool6 dataset was manually annotated, which provided a better validation platform for surgical tool detection methods. We propose an enhanced feature-fusion network (EFFNet) for real-time surgical tool detection. FENet20 is the backbone of the network and performs feature extraction more effectively. EFFNet is the feature-fusion part and performs two rounds of feature fusion to enhance the utilization of low-level and high-level feature information. The latter part of the network contains the weight fusion and predictor responsible for the output of the prediction results. The performance of the proposed method was tested using the ATLAS Dione and Cholec80-tool6 datasets, yielding mean average precision values of 97.0% and 95.0% with 21.6 frames per second, respectively. Its speed met the real-time standard and its accuracy outperformed that of other detection methods.<\/jats:p>","DOI":"10.1093\/jcde\/qwac049","type":"journal-article","created":{"date-parts":[[2022,5,25]],"date-time":"2022-05-25T16:24:56Z","timestamp":1653495896000},"page":"1123-1134","source":"Crossref","is-referenced-by-count":14,"title":["Real-time surgical tool detection in computer-aided surgery based on enhanced feature-fusion convolutional neural network"],"prefix":"10.1093","volume":"9","author":[{"given":"Kaidi","family":"Liu","sequence":"first","affiliation":[{"name":"School of Control Science and Engineering, Shandong University , Jinan 250061, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zijian","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University , Jinan 250061, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pan","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University , Jinan 250061, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feng","family":"Li","sequence":"additional","affiliation":[{"name":"Department of General Surgery, Qilu Hospital of Shandong University , Jinan 250012, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"He","family":"Song","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University , Jinan 250061, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,5,24]]},"reference":[{"key":"2022062215491960900_bib1","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/j.media.2018.05.001","article-title":"Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks","volume":"47","author":"Al\u00a0Hajj","year":"2018","journal-title":"Medical Image Analysis"},{"key":"2022062215491960900_bib2","first-page":"245","article-title":"Surgical tool detection and tracking in retinal microsurgery","volume-title":"Proceedings of SPIE","author":"Alsheakhali","year":"2015"},{"key":"2022062215491960900_bib3","article-title":"YOLOv4: Optimal speed and accuracy of object detection","author":"Bochkovskiy","year":"2020"},{"key":"2022062215491960900_bib4","first-page":"1756","article-title":"Surgical-tools detection based on convolutional neural network in laparoscopic robot-assisted surgery","volume-title":"Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS","author":"Choi","year":"2017"},{"issue":"5","key":"2022062215491960900_bib5","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1109\/TMI.2017.2787672","article-title":"Articulated multi-instrument 2-D pose estimation using fully convolutional networks","volume":"37","author":"Du","year":"2018","journal-title":"IEEE Transactions on Medical Imaging"},{"issue":"5","key":"2022062215491960900_bib6","doi-asserted-by":"crossref","first-page":"594","DOI":"10.1097\/00005537-199705000-00008","article-title":"Image-guided endoscopic surgery: Results of accuracy and performance in a multicenter clinical study using an electromagnetic tracking system","volume":"107","author":"Fried","year":"1997","journal-title":"The Laryngoscope"},{"key":"2022062215491960900_bib7","article-title":"Localizing dexterous surgical tools in X-ray for image-based navigation","author":"Gao","year":"2019"},{"issue":"2","key":"2022062215491960900_bib8","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1109\/TPAMI.2018.2844175","article-title":"Mask R-CNN","volume":"42","author":"He","year":"2017","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2022062215491960900_bib9","article-title":"Gaussian error linear units (GELUs)","author":"Hendrycks","year":"2016"},{"issue":"8","key":"2022062215491960900_bib10","doi-asserted-by":"crossref","first-page":"2011","DOI":"10.1109\/TPAMI.2019.2913372","article-title":"Squeeze-and-excitation networks","volume":"42","author":"Hu","year":"2020","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2022062215491960900_bib11","first-page":"646","article-title":"Deep networks with stochastic depth","volume-title":"Computer Vision \u2013 ECCV 2016. ECCV 2016. Lecture Notes in Computer Science","author":"Huang","year":"2016"},{"key":"2022062215491960900_bib12","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1109\/WACV.2018.00081","article-title":"Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks","author":"Jin","year":"2018","journal-title":"2018 IEEE Winter Conference on Applications of Computer Vision (WACV)"},{"issue":"5","key":"2022062215491960900_bib13","doi-asserted-by":"crossref","first-page":"1114","DOI":"10.1109\/TMI.2017.2787657","article-title":"SV-RCNet: Workflow recognition from surgical videos using recurrent convolutional network","volume":"37","author":"Jin","year":"2018","journal-title":"IEEE Transactions on Medical Imaging"},{"key":"2022062215491960900_bib14","doi-asserted-by":"crossref","first-page":"101572","DOI":"10.1016\/j.media.2019.101572","article-title":"Multi-task recurrent convolutional network with correlation loss for surgical video analysis","volume":"59","author":"Jin","year":"2020","journal-title":"Medical Image Analysis"},{"issue":"6","key":"2022062215491960900_bib15","doi-asserted-by":"crossref","first-page":"271","DOI":"10.3109\/10929089809148148","article-title":"FRACAS: A system for computer-aided image-guided long bone fracture surgery","volume":"3","author":"Joskowicz","year":"1998","journal-title":"Computer Aided Surgery"},{"key":"2022062215491960900_bib16","first-page":"1097","article-title":"ImageNet classification with deep convolutional neural networks","volume":"2","author":"Krizhevsky","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"5","key":"2022062215491960900_bib17","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1109\/TRA.2003.817086","article-title":"Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing","volume":"19","author":"Krupa","year":"2003","journal-title":"IEEE Transactions on Robotics and Automation"},{"key":"2022062215491960900_bib18","first-page":"505","article-title":"Simultaneous recognition and pose estimation of instruments in minimally invasive surgery","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2212 MICCAI 2017. MICCAI 2017. Lecture Notes in Computer Science","author":"Kurmann","year":"2017"},{"key":"2022062215491960900_bib19","first-page":"664","article-title":"Concurrent segmentation and localization for tracking of surgical instruments","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2212 MICCAI 2017. MICCAI 2017. Lecture Notes in Computer Science","author":"Laina","year":"2017"},{"key":"2022062215491960900_bib20","first-page":"88","article-title":"Image analysis for automated tracking in robot-assisted endoscopic surgery","volume-title":"Proceedings of 12th International Conference on Pattern Recognition","author":"Lee","year":"1994"},{"issue":"5","key":"2022062215491960900_bib21","doi-asserted-by":"crossref","first-page":"1407","DOI":"10.1093\/jcde\/qwab053","article-title":"Detection and tracking for the awareness of surroundings of a ship based on deep learning","volume":"8","author":"Lee","year":"2021","journal-title":"Journal of Computational Design and Engineering"},{"issue":"2","key":"2022062215491960900_bib22","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1109\/TPAMI.2018.2858826","article-title":"Focal loss for dense object detection","volume":"42","author":"Lin","year":"2020","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2022062215491960900_bib24","first-page":"21","article-title":"SSD: Single shot multibox detector","volume-title":"Computer Vision \u2013 ECCV 2016. ECCV 2016. Lecture Notes in Computer Science","author":"Liu","year":"2016"},{"key":"2022062215491960900_bib23","article-title":"Learning spatial fusion for single-shot object detection","author":"Liu","year":"2019"},{"key":"2022062215491960900_bib25","doi-asserted-by":"crossref","first-page":"78193","DOI":"10.1109\/ACCESS.2020.2989807","article-title":"An anchor-free convolutional neural network for real-time surgical tool detection in robot-assisted surgery","volume":"8","author":"Liu","year":"2020","journal-title":"IEEE Access"},{"key":"2022062215491960900_bib26","first-page":"2233","article-title":"Learning latent temporal connectionism of deep residual visual abstractions for identifying surgical tools in laparoscopy procedures","volume-title":"IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","author":"Mishra","year":"2017"},{"key":"2022062215491960900_bib27","first-page":"483","article-title":"Stacked hourglass networks for human pose estimation","volume-title":"Computer Vision \u2013 ECCV 2016. ECCV 2016. Lecture Notes in Computer Science","author":"Newell","year":"2016"},{"key":"2022062215491960900_bib28","first-page":"5735","article-title":"RASNet: Segmentation for tracking surgical instruments in surgical videos using refined attention segmentation network","volume-title":"Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS","author":"Ni","year":"2019"},{"issue":"6","key":"2022062215491960900_bib29","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1007\/s11548-019-01958-6","article-title":"Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos","volume":"14","author":"Nwoye","year":"2019","journal-title":"International Journal for Computer Assisted Radiology and Surgery"},{"key":"2022062215491960900_bib30","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1109\/CVPR.2016.91","article-title":"You only look once: Unified, real-time object detection","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Redmon","year":"2016"},{"key":"2022062215491960900_bib31","doi-asserted-by":"crossref","first-page":"6517","DOI":"10.1109\/CVPR.2017.690","article-title":"YOLO9000: Better, faster, stronger","volume-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Redmon","year":"2017"},{"key":"2022062215491960900_bib32","article-title":"YOLOv3: An incremental improvement","author":"Redmon","year":"2018"},{"issue":"6","key":"2022062215491960900_bib33","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2022062215491960900_bib34","doi-asserted-by":"crossref","first-page":"658","DOI":"10.1109\/CVPR.2019.00075","article-title":"Generalized intersection over union: A metric and a loss for bounding box regression","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Rezatofighi","year":"2019"},{"key":"2022062215491960900_bib35","first-page":"234","article-title":"U-net: Convolutional networks for biomedical image segmentation","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science","author":"Ronneberger","year":"2015"},{"issue":"7","key":"2022062215491960900_bib36","doi-asserted-by":"crossref","first-page":"1542","DOI":"10.1109\/TMI.2017.2665671","article-title":"Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection","volume":"36","author":"Sarikaya","year":"2017","journal-title":"IEEE Transactions on Medical Imaging"},{"issue":"4","key":"2022062215491960900_bib37","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TPAMI.2016.2572683","article-title":"Fully convolutional networks for semantic segmentation","volume":"39","author":"Shelhamer","year":"2017","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2022062215491960900_bib38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ACCESS.2020.3046258","article-title":"Real-time surgical tool detection in minimally invasive surgery based on attention-guided convolutional neural network","volume":"8","author":"Shi","year":"2020","journal-title":"IEEE Access"},{"issue":"2","key":"2022062215491960900_bib39","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1007\/s10439-011-0441-z","article-title":"Surgical vision","volume":"40","author":"Stoyanov","year":"2012","journal-title":"Annals of Biomedical Engineering"},{"key":"2022062215491960900_bib40","article-title":"EfficientNetV2: Smaller models and faster training","author":"Tan","year":"2021"},{"key":"2022062215491960900_bib41","doi-asserted-by":"crossref","first-page":"9626","DOI":"10.1109\/ICCV.2019.00972","article-title":"FCOS: Fully convolutional one-stage object detection","volume-title":"2019 IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Tian","year":"2019"},{"issue":"1","key":"2022062215491960900_bib42","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1109\/TMI.2016.2593957","article-title":"EndoNet: A deep architecture for recognition tasks on laparoscopic videos","volume":"36","author":"Twinanda","year":"2017","journal-title":"IEEE Transactions on Medical Imaging"},{"issue":"4","key":"2022062215491960900_bib43","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1093\/jcde\/qwab030","article-title":"Intervertebral disc instance segmentation using a multistage optimization mask-RCNN (MOM-RCNN)","volume":"8","author":"Vania","year":"2021","journal-title":"Journal of Computational Design and Engineering"},{"issue":"2","key":"2022062215491960900_bib44","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.jcde.2018.05.002","article-title":"Automatic spine segmentation from CT images using convolutional neural network via redundant generation of class labels","volume":"6","author":"Vania","year":"2019","journal-title":"Journal of Computational Design and Engineering"},{"key":"2022062215491960900_bib45","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1007\/978-3-030-01364-6_19","article-title":"Weakly-supervised learning for tool localization in laparoscopic videos","volume-title":"Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis","author":"Vardazaryan","year":"2018"},{"key":"2022062215491960900_bib47","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1109\/ISBI.2017.7950597","article-title":"Deep learning based multi-label classification for surgical tool presence detection in laparoscopic videos","volume-title":"2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017)","author":"Wang","year":"2017"},{"key":"2022062215491960900_bib46","doi-asserted-by":"crossref","first-page":"1781","DOI":"10.1109\/CVPR.2019.00188","article-title":"Adaptively connected neural networks","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Wang","year":"2019"},{"key":"2022062215491960900_bib48","first-page":"467","article-title":"Graph convolutional nets for tool presence detection in surgical videos","volume-title":"Information Processing in Medical Imaging. IPMI 2019. Lecture Notes in Computer Science","author":"Wang","year":"2019"},{"issue":"8","key":"2022062215491960900_bib49","doi-asserted-by":"crossref","first-page":"3092","DOI":"10.1109\/TMAG.2009.2019116","article-title":"A six-dimensional magnetic localization algorithm for a rectangular magnet objective based on a particle swarm optimizer","volume":"45","author":"Yang","year":"2009","journal-title":"IEEE Transactions on Magnetics"},{"key":"2022062215491960900_bib50","first-page":"18","article-title":"An efficient one-stage detector for real-time surgical tools detection in robot-assisted surgery","volume-title":"Medical Image Understanding and Analysis.\u00a0MIUA 2021. Lecture Notes in Computer Science","author":"Yang","year":"2021"},{"key":"2022062215491960900_bib51","article-title":"Less is more: Surgical phase recognition with less annotations through self-supervised pre-training of CNN-LSTM networks","author":"Yengera","year":"2018"},{"key":"2022062215491960900_bib52","doi-asserted-by":"crossref","first-page":"4203","DOI":"10.1109\/CVPR.2018.00442","article-title":"Single-shot refinement neural network for object detection","volume-title":"2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhang","year":"2018"},{"issue":"sup1","key":"2022062215491960900_bib56","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1080\/24699322.2017.1378777","article-title":"Tracking-by-detection of surgical instruments in minimally invasive surgery via the convolutional neural network deep learning-based method","volume":"22","author":"Zhao","year":"2017","journal-title":"Computer Assisted Surgery"},{"issue":"6","key":"2022062215491960900_bib53","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1049\/htl.2019.0064","article-title":"Real-time surgical instrument detection in robot-assisted surgery using a convolutional neural network cascade","volume":"6","author":"Zhao","year":"2019","journal-title":"Healthcare Technology Letters"},{"issue":"14","key":"2022062215491960900_bib55","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1049\/joe.2018.9401","article-title":"Surgical tool tracking based on two CNNs: From coarse to fine","volume":"2019","author":"Zhao","year":"2019","journal-title":"Journal of Engineering"},{"key":"2022062215491960900_bib57","article-title":"Objects as points","author":"Zhou","year":"2019"}],"container-title":["Journal of Computational Design and Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jcde\/advance-article-pdf\/doi\/10.1093\/jcde\/qwac049\/43815560\/qwac049.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/9\/3\/1123\/44167460\/qwac049.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/9\/3\/1123\/44167460\/qwac049.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,22]],"date-time":"2022-06-22T15:50:45Z","timestamp":1655913045000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jcde\/article\/9\/3\/1123\/6591578"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,24]]},"references-count":56,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,6,22]]}},"URL":"https:\/\/doi.org\/10.1093\/jcde\/qwac049","relation":{},"ISSN":["2288-5048"],"issn-type":[{"value":"2288-5048","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,6]]},"published":{"date-parts":[[2022,5,24]]}}}