{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:10:32Z","timestamp":1740147032524,"version":"3.37.3"},"reference-count":34,"publisher":"Public Library of Science (PLoS)","issue":"12","license":[{"start":{"date-parts":[[2024,12,3]],"date-time":"2024-12-03T00:00:00Z","timestamp":1733184000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003125","name":"Suomen Kulttuurirahasto","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003125","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003125","name":"Suomen Kulttuurirahasto","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003125","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100015738","name":"Mary och Georg C. Ehrnrooths Stiftelse","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100015738","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008723","name":"Suomen L\u00e4\u00e4ketieteen S\u00e4\u00e4ti\u00f6","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008723","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006306","name":"Sigrid Jus\u00e9liuksen S\u00e4\u00e4ti\u00f6","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100006306","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006136","name":"Teknologiateollisuuden 100-Vuotisjuhlas\u00e4\u00e4ti\u00f6","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100006136","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006136","name":"Teknologiateollisuuden 100-Vuotisjuhlas\u00e4\u00e4ti\u00f6","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100006136","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004012","name":"Jane ja Aatos Erkon S\u00e4\u00e4ti\u00f6","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004012","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004012","name":"Jane ja Aatos Erkon S\u00e4\u00e4ti\u00f6","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004012","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>The utilization of artificial intelligence (AI) is expanding significantly within medical research and, to some extent, in clinical practice. Deep learning (DL) applications, which use large convolutional neural networks (CNN), hold considerable potential, especially in optimizing radiological evaluations. However, training DL algorithms to clinical standards requires extensive datasets, and their processing is labor-intensive. In this study, we developed an annotation tool named DLLabelsCT that utilizes CNN models to accelerate the image analysis process. To validate DLLabelsCT, we trained a CNN model with a ResNet34 encoder and a UNet decoder to segment the pancreas on an open-access dataset and used the DL model to assist in annotating a local dataset, which was further used to refine the model. DLLabelsCT was also tested on two external testing datasets. The tool accelerates annotation by 3.4 times compared to a completely manual annotation method. Out of 3,715 CT scan slices in the testing datasets, 50% did not require editing when reviewing the segmentations made by the ResNet34-UNet model, and the mean and standard deviation of the Dice similarity coefficient was 0.82\u00b10.24. DLLabelsCT is highly accurate and significantly saves time and resources. Furthermore, it can be easily modified to support other deep learning models for other organs, making it an efficient tool for future research involving larger datasets.<\/jats:p>","DOI":"10.1371\/journal.pone.0313126","type":"journal-article","created":{"date-parts":[[2024,12,3]],"date-time":"2024-12-03T18:33:23Z","timestamp":1733250803000},"page":"e0313126","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["DLLabelsCT: Annotation tool using deep transfer learning to assist in creating new datasets from abdominal computed tomography scans, case study: Pancreas"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-4437-8718","authenticated-orcid":true,"given":"Henrik","family":"Mustonen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Antti","family":"Isosalo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minna","family":"Nortunen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mika","family":"Nevalainen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miika T.","family":"Nieminen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Heikki","family":"Huhta","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2024,12,3]]},"reference":[{"issue":"22","key":"pone.0313126.ref001","article-title":"Deep learning workflow in radiology: A primer.","volume":"11","author":"E Montagnon","year":"2020","journal-title":"Insights Imaging"},{"issue":"4","key":"pone.0313126.ref002","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1002\/jmri.26534","article-title":"Deep learning in radiology: An overview of the concepts and a survey of the state of the art with focus on MRI.","volume":"49","author":"MA Mazurowski","year":"2019","journal-title":"J. Magn. Reason. Imaging"},{"issue":"7553","key":"pone.0313126.ref003","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning.","volume":"521","author":"Y LeCun","year":"2015","journal-title":"Nature"},{"issue":"18","key":"pone.0313126.ref004","doi-asserted-by":"crossref","first-page":"25877","DOI":"10.1007\/s11042-022-12100-1","article-title":"Towards a better understanding of annotation tools for medical imaging: A survey.","volume":"81","author":"M Aljabri","year":"2022","journal-title":"Multimed. Tools Appl."},{"issue":"8","key":"pone.0313126.ref005","doi-asserted-by":"crossref","first-page":"1822","DOI":"10.1109\/TMI.2018.2806309","article-title":"Automatic multi-organ segmentation on abdominal CT with dense V-networks","volume":"37","author":"E Gibson","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"issue":"4","key":"pone.0313126.ref006","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1007\/s10278-019-00232-0","article-title":"RILContour: A medical imaging dataset annotation tool for and with deep learning.","volume":"32","author":"KA Philbrick","year":"2019","journal-title":"J. Digit. Imaging."},{"author":"H Roth","key":"pone.0313126.ref007","article-title":"Deep convolutional networks for pancreas segmentation in CT imagingProc. SPIE 9413, Medical Imaging 2015: Image Processing, 94131G,"},{"key":"pone.0313126.ref008","article-title":"Data from Pancreas-CT (Version 2) [Data set].","author":"H Roth","year":"2016","journal-title":"The Cancer Imaging Archive"},{"key":"pone.0313126.ref009","first-page":"556","article-title":"DeepOrgan: Multi-level deep convolutional networks for automated pancreas segmentation.","volume":"9349","author":"H Roth","year":"2015","journal-title":"Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), Lecture Notes in Computer Science"},{"issue":"6","key":"pone.0313126.ref010","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1007\/s10278-013-9622-7","article-title":"The Cancer Imaging Archive (TCIA): Maintaining and operating a public information repository.","volume":"26","author":"K Clark","year":"2013","journal-title":"J. Digit. Imaging."},{"key":"pone.0313126.ref011","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770\u2013778. doi: 10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"issue":"3","key":"pone.0313126.ref012","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet large scale visual recognition challenge","volume":"115","author":"O Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis"},{"key":"pone.0313126.ref013","first-page":"234","article-title":"U-net: Convolutional networks for biomedical image segmentation.","volume":"9351","author":"O Ronneberger","year":"2015","journal-title":"Medical Image Computing and Computer-Assisted Intervention (MICCAI 2015), Lecture Notes in Computer Science"},{"key":"pone.0313126.ref014","doi-asserted-by":"crossref","unstructured":"Isosalo A, Mustonen H, Turunen T, Ipatti PS, Reponen J, Nieminen MT, et al. Evaluation of different convolutional neural network encoder-decoder architectures for breast mass segmentation. Proc. SPIE 12037, Medical Imaging 2022: Imaging Informatics for Healthcare, Research, and Applications, 120370W. doi: 10.1117\/12.2628190","DOI":"10.1117\/12.2628190"},{"key":"pone.0313126.ref015","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python.","volume":"12","author":"F Pedregosa","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"pone.0313126.ref016","doi-asserted-by":"crossref","unstructured":"Abraham, N. & Khan, N. M. A novel focal Tversky loss function with improved attention U-net for lesion segmentation. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI), 683\u2013687. doi: 10.1109\/ISBI.2019.8759329","DOI":"10.1109\/ISBI.2019.8759329"},{"issue":"4","key":"pone.0313126.ref017","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1037\/0033-295X.84.4.327","article-title":"Features of similarity.","volume":"84","author":"A. Tversky","year":"1977","journal-title":"Psychological review."},{"key":"pone.0313126.ref018","article-title":"Solt: Streaming over lightweight transformations","author":"A. Tiulpin","year":"2019","journal-title":"Computer software"},{"key":"pone.0313126.ref019","unstructured":"Kingma DP, Ba J. Adam: A method for stochastic optimization. International Conference on Learning Representations (ICLR), 2014. URL: https:\/\/arxiv.org\/abs\/1412.6980v5"},{"key":"pone.0313126.ref020","first-page":"8024","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume":"32","author":"A Paszke","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pone.0313126.ref021","doi-asserted-by":"crossref","first-page":"100599","DOI":"10.1016\/j.simpa.2023.100599","article-title":"MammogramAnnotationTool: Markup tool for breast tissue abnormality annotation.","volume":"19","author":"A Isosalo","year":"2024","journal-title":"Softw. Impacts"},{"key":"pone.0313126.ref022","doi-asserted-by":"crossref","unstructured":"Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017, 2117\u20132125. doi: 10.1109\/CVPR.2017.106","DOI":"10.1109\/CVPR.2017.106"},{"key":"pone.0313126.ref023","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.neucom.2022.10.060","article-title":"TD-Net: Trans-deformer network for automatic pancreas segmentation.","volume":"517","author":"S Dai","year":"2023","journal-title":"Neurocomputing"},{"issue":"5","key":"pone.0313126.ref024","doi-asserted-by":"crossref","first-page":"e0252287","DOI":"10.1371\/journal.pone.0252287","article-title":"Multi-scale U-like network with attention mechanism for automatic pancreas segmentation","volume":"16","author":"Y Yan","year":"2021","journal-title":"PLOS ONE"},{"issue":"11","key":"pone.0313126.ref025","doi-asserted-by":"crossref","first-page":"115010","DOI":"10.1088\/1361-6560\/abfce3","article-title":"Pancreas segmentation with probabilistic map guided bi-directional recurrent UNet","volume":"66","author":"J Li","year":"2021","journal-title":"Phys Med Biol"},{"issue":"4075","key":"pone.0313126.ref026","article-title":"Automated pancreas segmentation and volumetry using deep neural network on computed tomography","volume":"12","author":"S Lim","year":"2022","journal-title":"Sci. Rep"},{"issue":"24","key":"pone.0313126.ref027","doi-asserted-by":"crossref","first-page":"7864","DOI":"10.3748\/wjg.v20.i24.7864","article-title":"Imaging diagnosis of pancreatic cancer: A state-of-the-art review","volume":"20","author":"ES Lee","year":"2014","journal-title":"WJG"},{"issue":"2","key":"pone.0313126.ref028","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1007\/s11548-021-02530-x","article-title":"A cascaded fully convolutional network framework for dilated pancreatic duct segmentation.","volume":"17","author":"C Shen","year":"2022","journal-title":"Int. J. CARS"},{"issue":"4","key":"pone.0313126.ref029","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1158\/2159-8290.CD-21-0090","article-title":"Artificial intelligence in cancer research and precision medicine","volume":"11","author":"B Bhinder","year":"2021","journal-title":"Cancer Discov"},{"issue":"3","key":"pone.0313126.ref030","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1097\/SLA.0000000000005349","article-title":"Imaging-based machine-learning models to predict clinical outcomes and identify biomarkers in pancreatic cancer","volume":"275","author":"BV Janssen","year":"2022","journal-title":"Ann. Surg"},{"issue":"1","key":"pone.0313126.ref031","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1148\/radiol.220152","article-title":"Pancreatic cancer detection on CT scans with deep learning: A nationwide population-based study.","volume":"306","author":"P Chen","year":"2023","journal-title":"Radiology"},{"issue":"1","key":"pone.0313126.ref032","doi-asserted-by":"crossref","first-page":"11","DOI":"10.5152\/dir.2019.19025","article-title":"Comparison of semi-automatic and deep learning-based automatic methods for liver segmentation in living liver transplant donors","volume":"26","author":"AE Kavur","year":"2020","journal-title":"Diagn. Interv. Radiol"},{"issue":"21","key":"pone.0313126.ref033","doi-asserted-by":"crossref","first-page":"5382","DOI":"10.3390\/cancers14215382","article-title":"Artificial intelligence-driven diagnosis of pancreatic cancer","volume":"14","author":"BS Hameed","year":"2022","journal-title":"Cancers"},{"issue":"7","key":"pone.0313126.ref034","doi-asserted-by":"crossref","first-page":"8459","DOI":"10.1007\/s12652-021-03612-z","article-title":"Artificial intelligence in disease diagnosis: A systematic literature review, synthesizing framework and future research agenda.","volume":"14","author":"Y Kumar","year":"2023","journal-title":"J. Ambient Intell. Human Comput"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0313126","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,3]],"date-time":"2024-12-03T18:33:39Z","timestamp":1733250819000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0313126"}},"subtitle":[],"editor":[{"given":"Hadeel K.","family":"Aljobouri","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2024,12,3]]},"references-count":34,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12,3]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0313126","relation":{},"ISSN":["1932-6203"],"issn-type":[{"type":"electronic","value":"1932-6203"}],"subject":[],"published":{"date-parts":[[2024,12,3]]}}}