{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:03:36Z","timestamp":1787317416258,"version":"build-2736575974"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2022,6,2]],"date-time":"2022-06-02T00:00:00Z","timestamp":1654128000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,6,2]],"date-time":"2022-06-02T00:00:00Z","timestamp":1654128000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging"],"published-print":{"date-parts":[[2022,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>A correct protocol assignment is critical to high-quality imaging examinations, and its automation can be amenable to natural language processing (NLP). Assigning protocols for abdominal imaging CT scans is particularly challenging given the multiple organ specific indications and parameters. We compared conventional machine learning, deep learning, and automated machine learning builder workflows for this multiclass text classification task. A total of 94,501 CT studies performed over 4\u00a0years and their assigned protocols were obtained. Text data associated with each study including the ordering provider generated free text study indication and ICD codes were used for NLP analysis and protocol class prediction. The data was classified into one of 11 abdominal CT protocol classes before and after augmentations used to account for imbalances in the class sample sizes. Four machine learning (ML) algorithms, one deep learning algorithm, and an automated machine learning (AutoML) builder were used for the multilabel classification task: Random Forest (RF), Tree Ensemble (TE), Gradient Boosted Tree (GBT), multi-layer perceptron (MLP), Universal Language Model Fine-tuning\u00a0(ULMFiT), and Google\u2019s AutoML builder (Alphabet, Inc., Mountain View, CA), respectively. On the unbalanced dataset, the manually coded algorithms all performed similarly with F1 scores of 0.811 for RF, 0.813 for TE, 0.813 for GBT, 0.828 for MLP, and 0.847 for ULMFiT. The AutoML builder performed better with a F1 score of 0.854. On the balanced dataset, the tree ensemble machine learning algorithm performed the best with an F1 score of 0.803 and a Cohen\u2019s kappa of 0.612. AutoML methods took a longer time for completion of NLP model training and evaluation, 4\u00a0h and 45\u00a0min compared to an average of 51\u00a0min for manual methods. Machine learning and natural language processing can be used for the complex multiclass classification task of abdominal imaging CT scan protocol assignment.<\/jats:p>","DOI":"10.1007\/s10278-022-00633-8","type":"journal-article","created":{"date-parts":[[2022,6,2]],"date-time":"2022-06-02T18:02:25Z","timestamp":1654192945000},"page":"1120-1130","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Natural Language Processing for Imaging Protocol Assignment: Machine Learning for Multiclass Classification of Abdominal CT Protocols Using Indication Text Data"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8147-7123","authenticated-orcid":false,"given":"Brian Arun","family":"Xavier","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Po-Hao","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,6,2]]},"reference":[{"key":"633_CR1","doi-asserted-by":"publisher","unstructured":"Selvarajan SK, Levin DC, Parker L: The Increasing Use of Emergency Department Imaging in the United States: Is It Appropriate? https:\/\/doi.org\/10.2214\/AJR.19.21386. American Roentgen Ray Society. 213(4):W180\u2013W184, 2019. https:\/\/doi.org\/10.2214\/AJR.19.21386.","DOI":"10.2214\/AJR.19.21386 10.2214\/AJR.19.21386"},{"key":"633_CR2","doi-asserted-by":"publisher","unstructured":"McDonald RJ, Schwartz KM, Eckel LJ, et al: The Effects of Changes in Utilization and Technological Advancements of Cross-Sectional Imaging on Radiologist Workload. Academic Radiology. Elsevier\u00a022(9):1191\u20131198,2015. https:\/\/doi.org\/10.1016\/J.ACRA.2015.05.007.","DOI":"10.1016\/J.ACRA.2015.05.007"},{"key":"633_CR3","doi-asserted-by":"publisher","unstructured":"Bell LTO, James R, Rosa JA, Pollentine A, Pettet G, McCoubrie P: Reducing interruptions during duty radiology shifts, assessment of its benefits and review of factors affecting the radiology working environment. Clinical Radiology. W.B. Saunders\u00a073(8):759.e19\u2013759.e25,2018. https:\/\/doi.org\/10.1016\/J.CRAD.2018.04.007.","DOI":"10.1016\/J.CRAD.2018.04.007"},{"key":"633_CR4","doi-asserted-by":"publisher","unstructured":"Kansagra AP, Liu K, Yu JPJ: Disruption of radiologist workflow. Current problems in diagnostic radiology. Mosby\u00a045(2):101\u2013106,2016. https:\/\/doi.org\/10.1067\/J.CPRADIOL.2015.05.006.","DOI":"10.1067\/J.CPRADIOL.2015.05.006"},{"key":"633_CR5","doi-asserted-by":"publisher","unstructured":"Lee MH, Schemmel AJ, Pooler BD, et al: Workflow dynamics and the imaging value chain: Quantifying the effect of designating a nonimage-interpretive task workflow. current problems in diagnostic radiology. Mosby\u00a046(4):275\u2013281,2017. https:\/\/doi.org\/10.1067\/J.CPRADIOL.2016.11.010.","DOI":"10.1067\/J.CPRADIOL.2016.11.010"},{"key":"633_CR6","doi-asserted-by":"publisher","unstructured":"Chen PH: Essential elements of natural language processing: What the radiologist should know. Academic Radiology. Elsevier\u00a027(1):6\u201312,2020. https:\/\/doi.org\/10.1016\/J.ACRA.2019.08.010.","DOI":"10.1016\/J.ACRA.2019.08.010"},{"key":"633_CR7","doi-asserted-by":"publisher","unstructured":"Lee YH: Efficiency improvement in a busy radiology practice: Determination of musculoskeletal magnetic resonance imaging protocol using deep-learning convolutional neural networks. J Digit Imaging. Springer New York LLC. 31(5):604\u2013610,2018. https:\/\/doi.org\/10.1007\/S10278-018-0066-Y\/TABLES\/5.","DOI":"10.1007\/S10278-018-0066-Y\/TABLES\/5"},{"key":"633_CR8","doi-asserted-by":"publisher","unstructured":"Trivedi H, Mesterhazy J, Laguna B, Vu T, Sohn JH: Automatic Determination of the Need for Intravenous Contrast in Musculoskeletal MRI Examinations Using IBM Watson\u2019s Natural Language Processing Algorithm. J Digit Imaging. Springer New York LLC 31(2):245\u2013251,2018. https:\/\/doi.org\/10.1007\/S10278-017-0021-3\/TABLES\/4.","DOI":"10.1007\/S10278-017-0021-3\/TABLES\/4"},{"key":"633_CR9","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/J.ARTINT.2012.06.009","volume":"199\u2013200","author":"D Ferrucci","year":"2013","unstructured":"Ferrucci D, Levas A, Bagchi S, Gondek D, Mueller ET. Watson: Beyond Jeopardy! Artificial Intelligence. Elsevier\u00a0199\u2013200:93\u2013105,2013. https:\/\/doi.org\/10.1016\/J.ARTINT.2012.06.009.","journal-title":"Elsevier"},{"key":"633_CR10","doi-asserted-by":"publisher","unstructured":"Chatley R, Donaldson A, Mycroft A: The Next 7000 Programming Languages. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, Cham\u00a010000:250\u2013282,2019. https:\/\/doi.org\/10.1007\/978-3-319-91908-9_15.","DOI":"10.1007\/978-3-319-91908-9_15"},{"key":"633_CR11","doi-asserted-by":"publisher","unstructured":"Berthold MR, Cebron N, Dill F, et al: KNIME - the Konstanz information miner. ACM SIGKDD Explorations Newsletter. ACM PUB27 New York, NY, USA; 58\u201361, 2009. https:\/\/doi.org\/10.1145\/1656274.1656280.","DOI":"10.1145\/1656274.1656280"},{"key":"633_CR12","doi-asserted-by":"publisher","unstructured":"Olthof AW, van Ooijen PMA, Cornelissen LJ: Deep learning-based natural language processing in radiology: The impact of report complexity, disease prevalence, dataset size, and algorithm type on model performance. J Med Syst. Springer; 45(10):1\u201316,2021. https:\/\/doi.org\/10.1007\/S10916-021-01761-4\/TABLES\/6.","DOI":"10.1007\/S10916-021-01761-4\/TABLES\/6"},{"key":"633_CR13","doi-asserted-by":"publisher","unstructured":"Ayoub J, Yang XJ, Zhou F: Combat COVID-19 infodemic using explainable natural language processing models. Information Processing & Management. Elsevier\u00a058(4):102569,2021. https:\/\/doi.org\/10.1016\/J.IPM.2021.102569.","DOI":"10.1016\/J.IPM.2021.102569"},{"key":"633_CR14","doi-asserted-by":"crossref","unstructured":"Friedman JH: Greedy function approximation: A gradient boosting machine. The Annals of Statistics. Institute of Mathematical Statistics 29(5):1189\u20131232,2001. http:\/\/www.jstor.org\/stable\/2699986.","DOI":"10.1214\/aos\/1013203451"},{"key":"633_CR15","doi-asserted-by":"publisher","unstructured":"Wang Y, Sohn S, Liu S, et al: A clinical text classification paradigm using weak supervision and deep representation. BMC Med Inform Decis Mak. BioMed Central Ltd 19(1):1\u201313, 2019. https:\/\/doi.org\/10.1186\/S12911-018-0723-6\/FIGURES\/4.","DOI":"10.1186\/S12911-018-0723-6\/FIGURES\/4"},{"key":"633_CR16","unstructured":"Merity S, Xiong C, Bradbury J, Socher R: Pointer Sentinel Mixture Models. 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings. International Conference on Learning Representations, ICLR,\u00a02016.\u00a0https:\/\/arxiv.org\/abs\/1609.07843v1. Accessed February 10, 2022."},{"key":"633_CR17","unstructured":"Merity S, Keskar NS, Socher R: Regularizing and Optimizing LSTM Language Models. 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings. International Conference on Learning Representations, ICLR, 2017. https:\/\/arxiv.org\/abs\/1708.02182v1. Accessed February 10, 2022."},{"key":"633_CR18","doi-asserted-by":"publisher","unstructured":"Howard J, Ruder S: Universal Language Model Fine-tuning for Text Classification. ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). Association for Computational Linguistics (ACL)\u00a01:328\u2013339,2018. https:\/\/doi.org\/10.18653\/v1\/p18-1031.","DOI":"10.18653\/v1\/p18-1031"},{"key":"633_CR19","unstructured":"Protocol Design and Optimization | American College of Radiology. . https:\/\/www.acr.org\/Practice-Management-Quality-Informatics\/Imaging-3\/Imaging-Value-Chain\/Protocol-Optimization. Accessed February 10, 2022."},{"key":"633_CR20","doi-asserted-by":"publisher","unstructured":"Dong Q, Gong S, Zhu X: Imbalanced deep learning by minority class incremental rectification. IEEE Transactions on Pattern Analysis and Machine Intelligence. IEEE Computer Society 41(6):1367\u20131381,2019. https:\/\/doi.org\/10.1109\/TPAMI.2018.2832629.","DOI":"10.1109\/TPAMI.2018.2832629"}],"container-title":["Journal of Digital Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-022-00633-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-022-00633-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-022-00633-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,19]],"date-time":"2022-10-19T14:16:52Z","timestamp":1666189012000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-022-00633-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,2]]},"references-count":20,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,10]]}},"alternative-id":["633"],"URL":"https:\/\/doi.org\/10.1007\/s10278-022-00633-8","relation":{},"ISSN":["0897-1889","1618-727X"],"issn-type":[{"value":"0897-1889","type":"print"},{"value":"1618-727X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,2]]},"assertion":[{"value":"24 June 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 April 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 June 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"N\/A","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and Consent to Participate"}},{"value":"N\/A","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}