{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T06:17:17Z","timestamp":1778048237523,"version":"3.51.4"},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,7,12]],"date-time":"2021-07-12T00:00:00Z","timestamp":1626048000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,7,12]],"date-time":"2021-07-12T00:00:00Z","timestamp":1626048000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>A systematic approach to MRI protocol assignment is essential for the efficient delivery of safe patient care. Advances in natural language processing (NLP) allow for the development of accurate automated protocol assignment. We aim to develop, evaluate, and deploy an NLP model that automates protocol assignment, given the clinician indication text.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We collected 7139 spine MRI protocols (routine or contrast) and 990 head MRI protocols (routine brain, contrast brain, or other) from a single institution. Protocols were split into training (n\u2009=\u20094997 for spine MRI; n\u2009=\u2009839 for head MRI), validation (n\u2009=\u20091071 for spine MRI, fivefold cross-validation used for head MRI), and test (n\u2009=\u20091071 for spine MRI; n\u2009=\u2009151 for head MRI) sets. fastText and XGBoost were used to develop 2 NLP models to classify spine and head MRI protocols, respectively. A Flask-based web app was developed to be deployed via Heroku.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The spine MRI model had an accuracy of 83.38% and a receiver operator characteristic area under the curve (ROC-AUC) of 0.8873. The head MRI model had an accuracy of 85.43% with a routine brain protocol ROC-AUC of 0.9463 and contrast brain protocol ROC-AUC of 0.9284. Cancer, infectious, and inflammatory related keywords were associated with contrast administration. Structural anatomic abnormalities and stroke\/altered mental status were indicative of routine spine and brain MRI, respectively. Error analysis revealed increasing the sample size may improve performance for head MRI protocols. A web version of the model is provided for demonstration and deployment.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>We developed and web-deployed two NLP models that accurately predict spine and head MRI protocol assignment, which could improve radiology workflow efficiency.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-021-01574-y","type":"journal-article","created":{"date-parts":[[2021,7,12]],"date-time":"2021-07-12T15:03:18Z","timestamp":1626102198000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Development and web deployment of an automated neuroradiology MRI protocoling tool with natural language processing"],"prefix":"10.1186","volume":"21","author":[{"given":"Yeshwant Reddy","family":"Chillakuru","sequence":"first","affiliation":[]},{"given":"Shourya","family":"Munjal","sequence":"additional","affiliation":[]},{"given":"Benjamin","family":"Laguna","sequence":"additional","affiliation":[]},{"given":"Timothy L.","family":"Chen","sequence":"additional","affiliation":[]},{"given":"Gunvant R.","family":"Chaudhari","sequence":"additional","affiliation":[]},{"given":"Thienkhai","family":"Vu","sequence":"additional","affiliation":[]},{"given":"Youngho","family":"Seo","sequence":"additional","affiliation":[]},{"given":"Jared","family":"Narvid","sequence":"additional","affiliation":[]},{"given":"Jae Ho","family":"Sohn","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,7,12]]},"reference":[{"issue":"5","key":"1574_CR1","doi-asserted-by":"publisher","first-page":"440","DOI":"10.1016\/j.jacr.2014.01.021","volume":"11","author":"GW Boland","year":"2014","unstructured":"Boland GW, Duszak R, Kalra M. Protocol design and optimization. J Am Coll Radiol. 2014;11(5):440\u20131.","journal-title":"J Am Coll Radiol"},{"issue":"22","key":"1574_CR2","doi-asserted-by":"publisher","first-page":"2400","DOI":"10.1001\/jama.2012.5960","volume":"307","author":"R Smith-Bindman","year":"2012","unstructured":"Smith-Bindman R, Miglioretti DL, Johnson E, Lee C, Feigelson HS, Flynn M, et al. Use of diagnostic imaging studies and associated radiation exposure for patients enrolled in large integrated health care systems, 1996\u20132010. JAMA. 2012;307(22):2400\u20139.","journal-title":"JAMA"},{"issue":"12","key":"1574_CR3","doi-asserted-by":"publisher","first-page":"1385","DOI":"10.1111\/acem.12816","volume":"22","author":"CC Blackmore","year":"2015","unstructured":"Blackmore CC, Castro A. Improving the quality of imaging in the emergency department. Acad Emerg Med. 2015;22(12):1385\u201392.","journal-title":"Acad Emerg Med"},{"issue":"10","key":"1574_CR4","doi-asserted-by":"publisher","first-page":"1210","DOI":"10.1016\/j.jacr.2016.04.009","volume":"13","author":"A Schemmel","year":"2016","unstructured":"Schemmel A, Lee M, Hanley T, Pooler BD, Kennedy T, Field A, et al. Radiology workflow disruptors: a detailed analysis. J Am Coll Radiol. 2016;13(10):1210\u20134.","journal-title":"J Am Coll Radiol"},{"issue":"5","key":"1574_CR5","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1016\/j.jacr.2015.08.027","volume":"13","author":"DT Ginat","year":"2016","unstructured":"Ginat DT, Uppuluri P, Christoforidis G, Katzman G, Lee S-K. Identification of neuroradiology MRI protocol errors via a quality-driven categorization approach. J Am Coll Radiol. 2016;13(5):545\u20138.","journal-title":"J Am Coll Radiol"},{"issue":"1","key":"1574_CR6","doi-asserted-by":"publisher","first-page":"29","DOI":"10.7326\/0003-4819-157-1-201207030-00450","volume":"157","author":"TJ Bright","year":"2012","unstructured":"Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157(1):29\u201343.","journal-title":"Ann Intern Med"},{"issue":"10","key":"1574_CR7","doi-asserted-by":"publisher","first-page":"1223","DOI":"10.1001\/jama.293.10.1223","volume":"293","author":"AX Garg","year":"2005","unstructured":"Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223\u201338.","journal-title":"JAMA"},{"key":"1574_CR8","doi-asserted-by":"crossref","unstructured":"Kim Y. Convolutional Neural Networks for Sentence Classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). Doha, Qatar: Association for Computational Linguistics; 2014. p. 1746\u201351. Available from: https:\/\/www.aclweb.org\/anthology\/D14-1181","DOI":"10.3115\/v1\/D14-1181"},{"issue":"2","key":"1574_CR9","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1148\/radiol.16142770","volume":"279","author":"E Pons","year":"2016","unstructured":"Pons E, Braun LMM, Hunink MGM, Kors JA. Natural language processing in radiology: a systematic review. Radiology. 2016;279(2):329\u201343.","journal-title":"Radiology"},{"issue":"5","key":"1574_CR10","doi-asserted-by":"publisher","first-page":"568","DOI":"10.1093\/jamia\/ocx125","volume":"25","author":"AD Brown","year":"2018","unstructured":"Brown AD, Marotta TR. Using machine learning for sequence-level automated MRI protocol selection in neuroradiology. J Am Med Inform Assoc. 2018;25(5):568\u201371.","journal-title":"J Am Med Inform Assoc"},{"key":"1574_CR11","doi-asserted-by":"publisher","first-page":"103301","DOI":"10.1016\/j.jbi.2019.103301","volume":"100","author":"S Datta","year":"2019","unstructured":"Datta S, Bernstam EV, Roberts K. A frame semantic overview of NLP-based information extraction for cancer-related EHR notes. J Biomed Inform. 2019;100:103301.","journal-title":"J Biomed Inform"},{"issue":"8","key":"1574_CR12","doi-asserted-by":"publisher","first-page":"764","DOI":"10.1007\/s00117-018-0426-0","volume":"58","author":"F Jungmann","year":"2018","unstructured":"Jungmann F, Kuhn S, K\u00e4mpgen B. Basics and applications of Natural Language Processing (NLP) in radiology. Radiologe. 2018;58(8):764\u20138.","journal-title":"Radiologe"},{"key":"1574_CR13","doi-asserted-by":"publisher","first-page":"1149","DOI":"10.1016\/j.jacr.2020.03.012","volume":"17","author":"A Kalra","year":"2020","unstructured":"Kalra A, Chakraborty A, Fine B, Reicher J. Machine learning for automation of radiology protocols for quality and efficiency improvement. J Am Coll Radiol. 2020;17:1149\u201358.","journal-title":"J Am Coll Radiol"},{"key":"1574_CR14","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/j.ijmedinf.2019.05.021","volume":"129","author":"G Trivedi","year":"2019","unstructured":"Trivedi G, Hong C, Dadashzadeh ER, Handzel RM, Hochheiser H, Visweswaran S. Identifying incidental findings from radiology reports of trauma patients: an evaluation of automated feature representation methods. Int J Med Inform. 2019;129:81\u20137.","journal-title":"Int J Med Inform"},{"key":"1574_CR15","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1007\/s10278-017-0021-3","volume":"31","author":"H Trivedi","year":"2017","unstructured":"Trivedi H, Mesterhazy J, Laguna B, Vu T, Sohn JH. Automatic determination of the need for intravenous contrast in musculoskeletal MRI examinations using IBM Watson\u2019s natural language processing algorithm. J Digital Imaging. 2017;31:245\u201351.","journal-title":"J Digital Imaging"},{"key":"1574_CR16","first-page":"411","volume":"2017","author":"I Banerjee","year":"2018","unstructured":"Banerjee I, Madhavan S, Goldman RE, Rubin DL. Intelligent word embeddings of free-text radiology reports. AMIA Annu Symp Proc. 2018;2017:411\u201320.","journal-title":"AMIA Annu Symp Proc"},{"issue":"6","key":"1574_CR17","doi-asserted-by":"publisher","first-page":"1595","DOI":"10.1148\/rg.266065168","volume":"26","author":"CP Langlotz","year":"2006","unstructured":"Langlotz CP. RadLex: a new method for indexing online educational materials. Radiographics. 2006;26(6):1595\u20137.","journal-title":"Radiographics"},{"key":"1574_CR18","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825\u201330.","journal-title":"J Mach Learn Res"},{"key":"1574_CR19","unstructured":"Joulin A, Grave E, Bojanowski P, Mikolov T. Bag of Tricks for Efficient Text Classification. http:\/\/arxiv.org\/abs\/1607.01759 [cs] [Internet]. 2016 Jul 6 [cited 2019 Aug 6]; Available from: http:\/\/arxiv.org\/abs\/1607.01759"},{"key":"1574_CR20","doi-asserted-by":"crossref","unstructured":"Chen T, Guestrin C. XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining\u2014KDD \u201916. 2016;785\u201394.","DOI":"10.1145\/2939672.2939785"},{"key":"1574_CR21","unstructured":"Appropriateness Criteria [Internet]. American College of Radiology. 2020 [cited 2020 May 22]. Available from: https:\/\/acsearch.acr.org\/list"},{"issue":"1","key":"1574_CR22","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1055\/s-0038-1641221","volume":"27","author":"I Cho","year":"2018","unstructured":"Cho I, Bates DW. Behavioral economics interventions in clinical decision support systems. Yearb Med Inform. 2018;27(1):114\u201321.","journal-title":"Yearb Med Inform"},{"issue":"8","key":"1574_CR23","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1016\/j.healthpol.2018.05.014","volume":"122","author":"LA Baumann","year":"2018","unstructured":"Baumann LA, Baker J, Elshaug AG. The impact of electronic health record systems on clinical documentation times: a systematic review. Health Policy. 2018;122(8):827\u201336.","journal-title":"Health Policy"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01574-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01574-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01574-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,12]],"date-time":"2021-07-12T15:03:23Z","timestamp":1626102203000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01574-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,12]]},"references-count":23,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1574"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01574-y","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,12]]},"assertion":[{"value":"23 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 July 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 July 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This University of California, San Francisco Institutional Review Board approved, written informed consent-waived, and HIPAA compliant study collected de-identified protocol assignments and clinical indications from a single academic institution.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"We have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"213"}}