{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T16:32:50Z","timestamp":1775665970830,"version":"3.50.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T00:00:00Z","timestamp":1630281600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T00:00:00Z","timestamp":1630281600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Context-aware system (CAS) is a system that can understand the context of a given situation and either share this context with other systems for their response or respond by itself. In surgery, these systems are intended to assist surgeons enhance the scheduling productivity of operating rooms (OR) and surgical teams, and promote a comprehensive perception and consciousness of the OR. Furthermore, the automated surgical tool classification in medical images is a real-time computerized assistance to the surgeons in conducting different operations. Moreover, deep learning has embroiled in every facet of life due to the availability of large datasets and the emergence of convolutional neural networks (CNN) that have paved the way for the development of different image related processes. The aim of this paper is to resolve the problem of unbalanced data in the publicly available Cholec80 laparoscopy video dataset, using multiple data augmentation techniques. Furthermore, we implement a fine-tuned CNN to tackle the automatic tool detection during a surgery, with prospective use in the teaching field, evaluating surgeons, and surgical quality assessment (SQA). The proposed method is evaluated on a dataset of 80 cholecystectomy videos (Cholec80 dataset). A mean average precision of 93.75% demonstrates the effectiveness of the proposed method, outperforming the other models significantly.<\/jats:p>","DOI":"10.1186\/s40537-021-00509-8","type":"journal-article","created":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T12:03:09Z","timestamp":1630324989000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Towards more efficient CNN-based surgical tools classification using transfer learning"],"prefix":"10.1186","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8825-5427","authenticated-orcid":false,"given":"Jaafar","family":"Jaafari","sequence":"first","affiliation":[]},{"given":"Samira","family":"Douzi","sequence":"additional","affiliation":[]},{"given":"Khadija","family":"Douzi","sequence":"additional","affiliation":[]},{"given":"Badr","family":"Hssina","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,8,30]]},"reference":[{"issue":"5","key":"509_CR1","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1001\/jamasurg.2014.4052","volume":"150","author":"T Xu","year":"2015","unstructured":"Tim Xu, Hutfless Susan M, Cooper Michol A, et al. Hospital cost implications of increased use of minimally invasive surgery. JAMA Surg. 2015;150(5):489.","journal-title":"JAMA Surg"},{"key":"509_CR2","doi-asserted-by":"publisher","DOI":"10.1007\/s11605-018-3883-x","author":"Q Chen","year":"2018","unstructured":"Chen Q, Merath K, Bagante F, Akgul O, Dillhoff M, Cloyd J, Pawlik TM. A comparison of open and minimally invasive surgery for hepatic and pancreatic resections among the medicare population. J Gastrointest Surg. 2018. https:\/\/doi.org\/10.1007\/s11605-018-3883-x.","journal-title":"J Gastrointest Surg"},{"key":"509_CR3","doi-asserted-by":"crossref","unstructured":"Ee WWG, Lau WLJ, Yeo W, Bing VY, Yue WM. Does minimally invasive surgery have a lower risk of surgical site infections compared with open spinal surgery? Clinical. 2013.","DOI":"10.1007\/s11999-013-3158-5"},{"issue":"3","key":"509_CR4","doi-asserted-by":"publisher","first-page":"828","DOI":"10.1016\/j.jsurg.2017.09.027","volume":"75","author":"P Mota","year":"2018","unstructured":"Mota P, Carvalho N, Carvalho-Dias E, Jo\u00e3ao Costa M, Correia-Pinto J, Lima E. Video-based surgical learning: improving trainee education and preparation for surgery. J Surg Edu. 2018;75(3):828\u201335. https:\/\/doi.org\/10.1016\/j.jsurg.2017.09.027.","journal-title":"J Surg Edu"},{"key":"509_CR5","doi-asserted-by":"publisher","first-page":"2909","DOI":"10.1007\/s00464-012-2284-6","volume":"26","author":"KR Henken","year":"2012","unstructured":"Henken KR, Jansen FW, Klein J, Stassen LPS, Dankelman J, van den Dobbelsteen JJ. Implications of the law on video recording in clinical practice. Surg Endosc. 2012;26:2909\u201316. https:\/\/doi.org\/10.1007\/s00464-012-2284-6.","journal-title":"Surg Endosc"},{"key":"509_CR6","doi-asserted-by":"publisher","unstructured":"Li L, Huang H, Jin X. AE-CNN Classification of Pulmonary Tuberculosis Based on CT Images. 2018 9th International Conference on Information Technology in Medicine and Education (ITME); 2018. https:\/\/doi.org\/10.1109\/itme.2018.00020.","DOI":"10.1109\/itme.2018.00020"},{"key":"509_CR7","doi-asserted-by":"publisher","unstructured":"Xiao Z, Huang R, Ding Y, Lan T, Dong F, Qin Z, Wang W. A deep learning-based segmentation method for brain tumor in MR images. 2016 IEEE 6th International Conference on Computational Advances in Bio and Medical Sciences (ICCABS); 2016. https:\/\/doi.org\/10.1109\/iccabs.2016.7802771.","DOI":"10.1109\/iccabs.2016.7802771"},{"key":"509_CR8","doi-asserted-by":"publisher","unstructured":"Joshi S, Gore S. Ishemic Stroke Lesion Segmentation by Analyzing MRI Images Using Dilated and Transposed Convolutions in Convolutional Neural Networks. 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA); 2018. https:\/\/doi.org\/10.1109\/iccubea.2018.8697545.","DOI":"10.1109\/iccubea.2018.8697545"},{"key":"509_CR9","doi-asserted-by":"publisher","unstructured":"Ye J, Luo Y, Zhu C, Liu F, Zhang Y. Breast cancer image classification on WSI with spatial correlations. ICASSP 2019\u20132019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2019. https:\/\/doi.org\/10.1109\/icassp.2019.8682560.","DOI":"10.1109\/icassp.2019.8682560"},{"key":"509_CR10","doi-asserted-by":"crossref","unstructured":"Kiruthika M, Swapna TR, Kumar SC, Peeyush KP. Artery and Vein classification for hypertensive retinopathy 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI).","DOI":"10.1109\/ICOEI.2019.8862719"},{"key":"509_CR11","doi-asserted-by":"publisher","unstructured":"Kletz S, Schoeffmann K, Benois-Pineau J, Husslein H. Identifying Surgical Instruments in aparoscopy using deep learning instance segmentation. 2019 International Conference on Content-Based Multimedia Indexing (CBMI); 2019. https:\/\/doi.org\/10.1109\/cbmi.2019.8877379.","DOI":"10.1109\/cbmi.2019.8877379"},{"key":"509_CR12","doi-asserted-by":"crossref","unstructured":"Jin A et al. Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE; 2018","DOI":"10.1109\/WACV.2018.00081"},{"key":"509_CR13","doi-asserted-by":"crossref","unstructured":"da Costa Rocha C, Padoy N, Benoit R. International Conference on Robotics and\nAutomation (ICRA) Palais des congres de Montreal, Montreal, Canada, 20\u201324. Self-Supervised\nSurgical Tool Segmentation using Kinematic Information; 2019.","DOI":"10.1109\/ICRA.2019.8794334"},{"key":"509_CR14","doi-asserted-by":"crossref","unstructured":"Choi B, Jo K, Choi S, Choi J. Surgical-tools detection based on Convolutional NeuralNetwork in laparoscopic robot-assisted surgery. 2017 39th Annual International Conference of theIEEE Engineering in Medicine and Biology Society (EMBC); 2017.","DOI":"10.1109\/EMBC.2017.8037183"},{"key":"509_CR15","doi-asserted-by":"crossref","unstructured":"Wang S, Raju A, Huang J. Deep learning based multi-label classification for surgical tool presence detection in laparoscopic videos; 2017.","DOI":"10.1109\/ISBI.2017.7950597"},{"key":"509_CR16","doi-asserted-by":"crossref","unstructured":"Attia M, Hossny M, Nahavandi S, Asadi H. Surgical tool segmentation using a hybrid deep CNN-RNN auto encoder-decoder. 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC); 2017.","DOI":"10.1109\/SMC.2017.8123151"},{"key":"509_CR17","unstructured":"Automatic instrument segmentation in robot-assisted surgery using deep learning,\u201cA. A. Shvets. 2018 17th IEEE International Conference on machine learning and applications."},{"key":"509_CR18","unstructured":"Robotic instrument segmentation sub-challenge part of the endoscopic vision challenge. https:\/\/endovissub2017-roboticinstrumentsegmentation.grand-challenge.org\/."},{"key":"509_CR19","doi-asserted-by":"publisher","unstructured":"Islam M, Atputharuban DA, Ramesh R, Ren H. Real-time instrument segmentation in robotic surgery using auxiliary supervised deep adversarial learning. IEEE Robotics and Automation Letters. 2019; pp. 1\u20131. https:\/\/doi.org\/10.1109\/lra.2019.2900854.","DOI":"10.1109\/lra.2019.2900854"},{"key":"509_CR20","doi-asserted-by":"crossref","unstructured":"Shvets AA, Rakhlin A, Kalinin AA, Iglovikov VI. Automatic instrument segmentation in robot-assisted surgery using deep learning. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA); 2018.","DOI":"10.1109\/ICMLA.2018.00100"},{"key":"509_CR21","doi-asserted-by":"crossref","unstructured":"Colleoni E, Moccia S, Du X, De Momi E, Stoyanov D. Deep learning based robotic tool detection and articulation estimation with spatio-temporal layers; 2019.","DOI":"10.1109\/LRA.2019.2917163"},{"key":"509_CR22","doi-asserted-by":"crossref","unstructured":"Sarikaya D, Corso JJ, Guru KA. Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection; 2017.","DOI":"10.1109\/TMI.2017.2665671"},{"key":"509_CR23","doi-asserted-by":"publisher","unstructured":"Chittajallu DR, Dong B, Tunison P, Collins R, Wells K, Fleshman J, Enquobahrie A. XAI-CBIR: explainable AI system for content based retrieval of video frames from minimally invasive surgery videos. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019); 2019 https:\/\/doi.org\/10.1109\/isbi.2019.8759428.","DOI":"10.1109\/isbi.2019.8759428"},{"key":"509_CR24","first-page":"349","volume":"15","author":"E Travis","year":"2014","unstructured":"Travis E, Woodhouse S, Tan R, Patel S, Donovan J, Brogan K. Operating theatre time, where does it all go? a prospective observational study. BMJ. 2014;15:349.","journal-title":"BMJ."},{"key":"509_CR25","doi-asserted-by":"publisher","unstructured":"Twinanda AP, Yengera G, Mutter D, Marescaux J, Padoy N. RSDNet: Learning to predict remaining surgery duration from laparoscopic videos without manual annotations. IEEE Transactions on Medical Imaging. 2018; pp.1\u20131. https:\/\/doi.org\/10.1109\/tmi.2018.2878055.","DOI":"10.1109\/tmi.2018.2878055"},{"key":"509_CR26","doi-asserted-by":"publisher","unstructured":"Ucuzal H, Arslan AK, Colak C. Deep learning based-classification of dementia inmagneticresonance imaging scans. 2019 International Artificial Intelligence and Data Processing Symposium(IDAP). 2019. https:\/\/doi.org\/10.1109\/idap.2019.887596.","DOI":"10.1109\/idap.2019.887596"},{"key":"509_CR27","doi-asserted-by":"publisher","unstructured":"Zhao Y, Zhao J, Zhao C, Xiong W, Li Q, Yang J. Robust Real-Time Object Detection Based on Deep Learning for Very High Resolution Remote Sensing Images. IGARSS 2019\u20132019. IEEE International Geoscience and Remote Sensing Symposium; 2019. https:\/\/doi.org\/10.1109\/igarss.2019.8897976.","DOI":"10.1109\/igarss.2019.8897976"},{"key":"509_CR28","doi-asserted-by":"publisher","unstructured":"Qu X, Wei T, Peng C, Du P. A fast face recognition system based on deepLearning. 2018 11th International Symposium on Computational Intelligence and Design (ISCID). 2018; https:\/\/doi.org\/10.1109\/iscid.2018.00072.","DOI":"10.1109\/iscid.2018.00072"},{"key":"509_CR29","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436\u201344.","journal-title":"Nature"},{"issue":"9","key":"509_CR30","doi-asserted-by":"publisher","first-page":"2352","DOI":"10.1162\/neco_a_00990","volume":"29","author":"W Rawat","year":"2017","unstructured":"Rawat W, Wang Z. Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput. 2017;29(9):2352\u2013449.","journal-title":"Neural Comput."},{"issue":"1","key":"509_CR31","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1109\/tmi.2016.2593957","volume":"36","author":"AP Twinanda","year":"2017","unstructured":"Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N. EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imag. 2017;36(1):86\u201397. https:\/\/doi.org\/10.1109\/tmi.2016.2593957.","journal-title":"IEEE Trans Med Imag"},{"key":"509_CR32","unstructured":"Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processingsystems; 2012. pp. 1097\u20131105."},{"key":"509_CR33","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. https:\/\/doi.org\/10.1109\/cvpr.2016.90.","DOI":"10.1109\/cvpr.2016.90"},{"key":"509_CR34","doi-asserted-by":"publisher","unstructured":"Deng J, Dong W, Socher R, Li L-J, Li Kai, Fei-Fei Li. ImageNet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition; 2009. https:\/\/doi.org\/10.1109\/cvpr.2009.5206848.","DOI":"10.1109\/cvpr.2009.5206848"},{"key":"509_CR35","unstructured":"Sahu M, Mukhopadhyay A, Szengel A, Zachow S. Tool and phase recognition using contextual cnn features. 2016."},{"key":"509_CR36","doi-asserted-by":"publisher","first-page":"2865","DOI":"10.3390\/app9142865","volume":"9","author":"K Jo","year":"2019","unstructured":"Jo K, Choi Y, Choi J, Chung JW. Robust real-time detection of laparoscopic instruments in robot surgery using convolutional neural networks with motion vector prediction. Appl Sci. 2019;9:2865.","journal-title":"Appl Sci"},{"key":"509_CR37","doi-asserted-by":"publisher","unstructured":"Kanakatte A, Ramaswamy A, Gubbi J, Ghose A, Purushothaman B. \u201cSurgical tool segmentation and localization using spatio-temporal deep network,\u201d 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2020; pp. 1658\u20131661. https:\/\/doi.org\/10.1109\/EMBC44109.2020.9176676.","DOI":"10.1109\/EMBC44109.2020.9176676"}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-021-00509-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-021-00509-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-021-00509-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,30]],"date-time":"2021-08-30T12:10:26Z","timestamp":1630325426000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-021-00509-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,30]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["509"],"URL":"https:\/\/doi.org\/10.1186\/s40537-021-00509-8","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,30]]},"assertion":[{"value":"20 May 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 August 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 August 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The author confirms the sole responsibility for this manuscript. The author read and approved the final manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article\u2019s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article\u2019s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit .","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"115"}}