{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T19:36:49Z","timestamp":1771702609532,"version":"3.50.1"},"reference-count":100,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,2,14]],"date-time":"2023-02-14T00:00:00Z","timestamp":1676332800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,2,14]],"date-time":"2023-02-14T00:00:00Z","timestamp":1676332800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003246","name":"Dutch Research Council","doi-asserted-by":"crossref","award":["629.002.205"],"award-info":[{"award-number":["629.002.205"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Ministry of Electronics and Information Technology (MeITY), India","award":["13(2)-2015-CC-BT"],"award-info":[{"award-number":["13(2)-2015-CC-BT"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Rising incidence and mortality of cancer have led to an incremental amount of research in the field. To learn from preexisting data, it has become important to capture maximum information related to disease type, stage, treatment, and outcomes. Medical imaging reports are rich in this kind of information but are only present as free text. The extraction of information from such unstructured text reports is labor-intensive. The use of Natural Language Processing (NLP) tools to extract information from radiology reports can make it less time-consuming as well as more effective. In this study, we have developed and compared different models for the classification of lung carcinoma reports using clinical concepts. This study was approved by the institutional ethics committee as a retrospective study with a waiver of informed consent. A clinical concept-based classification pipeline for lung carcinoma radiology reports was developed using rule-based as well as machine learning models and compared. The machine learning models used were XGBoost and two more deep learning model architectures with bidirectional long short-term neural networks. A corpus consisting of 1700 radiology reports including computed tomography (CT) and positron emission tomography\/computed tomography (PET\/CT) reports were used for development and testing. Five hundred one radiology reports from MIMIC-III Clinical Database version 1.4 was used for external validation. The pipeline achieved an overall F1 score of 0.94 on the internal set and 0.74 on external validation with the rule-based algorithm using expert input giving the best performance. Among the machine learning models, the Bi-LSTM_dropout model performed better than the ML model using XGBoost and the Bi-LSTM_simple model on internal set, whereas on external validation, the Bi-LSTM_simple model performed relatively better than other 2. This pipeline can be used for clinical concept-based classification of radiology reports related to lung carcinoma from a huge corpus and also for automated annotation of these reports.\n<\/jats:p>","DOI":"10.1007\/s10278-023-00787-z","type":"journal-article","created":{"date-parts":[[2023,2,17]],"date-time":"2023-02-17T06:45:32Z","timestamp":1676616332000},"page":"812-826","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Clinical Concept-Based Radiology Reports Classification Pipeline for Lung Carcinoma"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9424-2726","authenticated-orcid":false,"given":"Sneha","family":"Mithun","sequence":"first","affiliation":[]},{"given":"Ashish Kumar","family":"Jha","sequence":"additional","affiliation":[]},{"given":"Umesh B.","family":"Sherkhane","sequence":"additional","affiliation":[]},{"given":"Vinay","family":"Jaiswar","sequence":"additional","affiliation":[]},{"given":"Nilendu C.","family":"Purandare","sequence":"additional","affiliation":[]},{"given":"Andre","family":"Dekker","sequence":"additional","affiliation":[]},{"given":"Sander","family":"Puts","sequence":"additional","affiliation":[]},{"given":"Inigo","family":"Bermejo","sequence":"additional","affiliation":[]},{"given":"V.","family":"Rangarajan","sequence":"additional","affiliation":[]},{"given":"Catharina M. L.","family":"Zegers","sequence":"additional","affiliation":[]},{"given":"Leonard","family":"Wee","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,2,14]]},"reference":[{"key":"787_CR1","doi-asserted-by":"publisher","unstructured":"Bray, Freddie, et al. \u201cGlobal Cancer Statistics 2018: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries.\u201d\u00a0CA: A Cancer Journal for Clinicians, vol. 68, no. 6, Nov. 2018, pp. 394\u2013424.\u00a0https:\/\/doi.org\/10.3322\/caac.21492.","DOI":"10.3322\/caac.21492"},{"key":"787_CR2","unstructured":"National Guideline Clearinghouse.\u00a0ACR practice guideline for communication of diagnostic imaging findings. Rockville, Md: Agency for Healthcare Research and Quality (AHRQ).\u00a0Revised 2020 (Resolution 37). [Online]. Available:\u00a0https:\/\/www.acr.org\/-\/media\/acr\/files\/practice-parameters\/communicationdiag.pdf. Accessed on: December 05, 2020."},{"key":"787_CR3","doi-asserted-by":"crossref","unstructured":"Jha AK, DesRoches CM, Campbell EG, et al.Use of electronic health records in U.S. hospitals.\u00a0N Engl J Med\u00a02009;360(16):1628\u20131638.","DOI":"10.1056\/NEJMsa0900592"},{"key":"787_CR4","doi-asserted-by":"publisher","unstructured":"Khorasani, Ramin, et al. \u201cIs Terminology Used Effectively to Convey Diagnostic Certainty in Radiology Reports?\u201d\u00a0Academic Radiology, vol. 10, no. 6, June 2003, pp. 685\u201388.\u00a0https:\/\/doi.org\/10.1016\/S1076-6332(03)80089-2.","DOI":"10.1016\/S1076-6332(03)80089-2"},{"key":"787_CR5","doi-asserted-by":"publisher","unstructured":"Hobby, J. L., et al. \u201cCommunication of Doubt and Certainty in Radiological Reports.\u201d\u00a0The British Journal of Radiology, vol. 73, no. 873, Sept. 2000, pp. 999\u20131001.\u00a0https:\/\/doi.org\/10.1259\/bjr.73.873.11064655.","DOI":"10.1259\/bjr.73.873.11064655"},{"key":"787_CR6","doi-asserted-by":"publisher","unstructured":"Wibmer, Andreas, et al. \u201cValue of a Standardized Lexicon for Reporting Levels of Diagnostic Certainty in Prostate MRI.\u201d\u00a0American Journal of Roentgenology, vol. 203, no. 6, Dec. 2014, pp. W651\u201357.\u00a0https:\/\/doi.org\/10.2214\/AJR.14.12654.","DOI":"10.2214\/AJR.14.12654"},{"key":"787_CR7","doi-asserted-by":"publisher","unstructured":"Panicek, David M., and HedvigHricak. \u201cHow Sure Are You, Doctor? A Standardized Lexicon to Describe the Radiologist\u2019s Level of Certainty.\u201d\u00a0American Journal of Roentgenology, vol. 207, no. 1, July 2016, pp. 2\u20133.\u00a0https:\/\/doi.org\/10.2214\/AJR.15.15895.","DOI":"10.2214\/AJR.15.15895"},{"key":"787_CR8","doi-asserted-by":"publisher","unstructured":"Yim, Wen-wai, et al. \u201cNatural Language Processing in Oncology: A Review.\u201d\u00a0JAMA Oncology, vol. 2, no. 6, June 2016, p. 797.\u00a0https:\/\/doi.org\/10.1001\/jamaoncol.2016.0213.","DOI":"10.1001\/jamaoncol.2016.0213"},{"key":"787_CR9","doi-asserted-by":"publisher","unstructured":"Travis, Adam R., et al. \u201cPreferences for Structured Reporting of Measurement Data.\u201d\u00a0Academic Radiology, vol. 21, no. 6, June 2014, pp. 785\u201396.\u00a0https:\/\/doi.org\/10.1016\/j.acra.2014.02.008.","DOI":"10.1016\/j.acra.2014.02.008"},{"key":"787_CR10","doi-asserted-by":"publisher","unstructured":"Larson, David B., et al. \u201cImproving Consistency in Radiology Reporting through the Use of Department-Wide Standardized Structured Reporting.\u201d\u00a0Radiology, vol. 267, no. 1, Apr. 2013, pp. 240\u201350.\u00a0https:\/\/doi.org\/10.1148\/radiol.12121502.","DOI":"10.1148\/radiol.12121502"},{"key":"787_CR11","unstructured":"Garbade, Dr. Michael J. \u201cA Simple Introduction to Natural Language Processing.\u201d Medium, Becoming Human: Artificial Intelligence Magazine, 15 Oct. 2018, https:\/\/becominghuman.ai\/a-simple-introduction-to-natural-language-processing-ea66a1747b32"},{"key":"787_CR12","unstructured":"Nigam, Vibhor. \u201cNatural Language Processing: From Basics, to Using RNN and LSTM.\u201d\u00a0Medium, 12 June 2020.\u00a0https:\/\/towardsdatascience.com\/natural-language-processing-from-basics-to-using-rnn-and-lstm-ef6779e4ae66."},{"key":"787_CR13","doi-asserted-by":"publisher","unstructured":"Martin, M. \u201cSemantic Web May Be Cancer Information\u2019s Next Step Forward.\u201d\u00a0JNCI Journal of the National Cancer Institute, vol. 103, no. 16, Aug. 2011, pp. 1215\u201318.\u00a0https:\/\/doi.org\/10.1093\/jnci\/djr321.","DOI":"10.1093\/jnci\/djr321"},{"key":"787_CR14","doi-asserted-by":"publisher","unstructured":"Zhu, Fei, et al. \u201cBiomedical Text Mining and Its Applications in Cancer Research.\u201d\u00a0Journal of Biomedical Informatics, vol. 46, no. 2, Apr. 2013, pp. 200\u201311.\u00a0https:\/\/doi.org\/10.1016\/j.jbi.2012.10.007.","DOI":"10.1016\/j.jbi.2012.10.007"},{"key":"787_CR15","doi-asserted-by":"publisher","unstructured":"Pons, Ewoud, et al. \u201cNatural Language Processing in Radiology: A Systematic Review.\u201d\u00a0Radiology, vol. 279, no. 2, May 2016, pp. 329\u201343.\u00a0https:\/\/doi.org\/10.1148\/radiol.16142770.","DOI":"10.1148\/radiol.16142770"},{"key":"787_CR16","doi-asserted-by":"publisher","unstructured":"Stanfill, Mary H., et al. \u201cA Systematic Literature Review of Automated Clinical Coding and Classification Systems.\u201d\u00a0Journal of the American Medical Informatics Association, vol. 17, no. 6, Nov. 2010, pp. 646\u201351.\u00a0https:\/\/doi.org\/10.1136\/jamia.2009.001024.","DOI":"10.1136\/jamia.2009.001024"},{"issue":"2","key":"787_CR17","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1148\/radiol.11110951","volume":"262","author":"AS Raja","year":"2012","unstructured":"Raja AS, Ip IK, Prevedello LM, Sodickson AD, Farkas C, Zane RD, Hanson R, Goldhaber SZ, Gill RR, Khorasani R. Effect of computerized clinical decision support on the use and yield of CT pulmonary angiography in the emergency department. Radiology. 2012 Feb;262(2):468-74.\u00a0https:\/\/doi.org\/10.1148\/radiol.11110951.","journal-title":"Radiology."},{"key":"787_CR18","unstructured":"Hogan, Aidan, et al. \u201cKnowledge Graphs.\u201d [Cs], Apr. 2020. arXiv.org,\u00a0arXiv:2003.02320.\u00a0"},{"key":"787_CR19","doi-asserted-by":"publisher","unstructured":"Asim, Muhammad Nabeel, et al. \u201cA Survey of Ontology Learning Techniques and Applications.\u201d\u00a0Database, vol. 2018, Jan. 2018. https:\/\/doi.org\/10.1093\/database\/bay101.","DOI":"10.1093\/database\/bay101"},{"key":"787_CR20","doi-asserted-by":"publisher","unstructured":"Bodenreider, O. \u201cThe Unified Medical Language System (UMLS): Integrating Biomedical Terminology.\u201d\u00a0Nucleic Acids Research, vol. 32, no. 90001, Jan. 2004, pp. 267D \u2013 270.\u00a0https:\/\/doi.org\/10.1093\/nar\/gkh061.","DOI":"10.1093\/nar\/gkh061"},{"key":"787_CR21","unstructured":"US National Institutes of Health. NationalCancer Institute: NCI Thesaurus. https:\/\/ncit.nci.nih.gov\/ncitbrowser\/. Accessed 5 Dec. 2020."},{"key":"787_CR22","unstructured":"National Cancer Institute Thesaurus | NCBO BioPortal. https:\/\/bioportal.bioontology.org\/ontologies\/NCIT. Accessed 5 Dec. 2020."},{"key":"787_CR23","unstructured":"Radiation Oncology Ontology | NCBO BioPortal. https:\/\/bioportal.bioontology.org\/ontologies\/ROO. Accessed 5 Dec. 2020."},{"key":"787_CR24","doi-asserted-by":"publisher","unstructured":"Traverso, Alberto, et al. \u201cThe Radiation Oncology Ontology (ROO): Publishing Linked Data in Radiation Oncology Using Semantic Web and Ontology Techniques.\u201d\u00a0Medical Physics, vol. 45, no. 10, Oct. 2018, pp. e854\u201362.\u00a0https:\/\/doi.org\/10.1002\/mp.12879.","DOI":"10.1002\/mp.12879"},{"key":"787_CR25","doi-asserted-by":"publisher","unstructured":"Leijnen, Stefan, and Fjodor van Veen. \u201cThe Neural Network Zoo.\u201d Proceedings, vol. 47, no. 1, May 2020, p. 9.\u00a0https:\/\/doi.org\/10.3390\/proceedings2020047009.","DOI":"10.3390\/proceedings2020047009"},{"key":"787_CR26","doi-asserted-by":"publisher","unstructured":"Ristevski, Blagoj, and Ming Chen. \u201cBig Data Analytics in Medicine and Healthcare.\u201d\u00a0Journal of Integrative Bioinformatics, vol. 15, no. 3, Sept. 2018.\u00a0https:\/\/doi.org\/10.1515\/jib-2017-0030.","DOI":"10.1515\/jib-2017-0030"},{"key":"787_CR27","doi-asserted-by":"publisher","unstructured":"Kankanhalli, Atreyi, et al. \u201cBig Data and Analytics in Healthcare: Introduction to the Special Section.\u201d\u00a0Information Systems Frontiers, vol. 18, no. 2, Apr. 2016, pp. 233\u201335.\u00a0https:\/\/doi.org\/10.1007\/s10796-016-9641-2.","DOI":"10.1007\/s10796-016-9641-2"},{"key":"787_CR28","doi-asserted-by":"publisher","unstructured":"Wu PY, Cheng CW, Kaddi CD, Venugopalan J, Hoffman R, Wang MD. -Omic and Electronic Health Record Big Data Analytics for Precision Medicine. IEEE Trans Biomed Eng. vol. 64, no. 2, Feb. 2017, pp. 263\u201373.\u00a0https:\/\/doi.org\/10.1109\/TBME.2016.2573285.","DOI":"10.1109\/TBME.2016.2573285"},{"key":"787_CR29","doi-asserted-by":"publisher","unstructured":"Sarmiento, Raymond Francis, and Franck Dernoncourt. \u201cImproving Patient Cohort Identification Using Natural Language Processing.\u201d Secondary Analysis of Electronic Health Records, edited by MIT Critical Data, Springer International Publishing, 2016, pp. 405\u201317. Springer Link, https:\/\/doi.org\/10.1007\/978-3-319-43742-2_28.","DOI":"10.1007\/978-3-319-43742-2_28"},{"issue":"8","key":"787_CR30","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1002\/pds.3418","volume":"22","author":"S Dublin","year":"2013","unstructured":"Dublin S, Baldwin E, Walker RL et al. Natural Language Processing to identify pneumonia from radiology reports. Pharmacoepidemiol Drug Saf 2013;22(8):834\u2013841.","journal-title":"Pharmacoepidemiol Drug Saf"},{"issue":"8","key":"787_CR31","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1097\/JTO.0b013e31825bd9f5","volume":"7","author":"KN Danforth","year":"2012","unstructured":"Danforth KN, Early MI, Ngan S, Kosco AE, Zheng C, Gould MK. Automated identification of patients with pulmonary nodules in an integrated health system using administrative health plan data, radiology reports, and natural language processing. J Thorac Oncol2012;7(8):1257\u20131262.","journal-title":"J Thorac Oncol"},{"issue":"5","key":"787_CR32","doi-asserted-by":"publisher","first-page":"913","DOI":"10.1136\/amiajnl-2011-000607","volume":"19","author":"B Percha","year":"2012","unstructured":"Percha B, Nassif H, Lipson J, Burnside E, Rubin D. Automatic classification of mammography reports by BI-RADS breast tissue composition class. J Am Med Inform Assoc 2012;19(5):913\u2013916.","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"787_CR33","doi-asserted-by":"publisher","first-page":"730","DOI":"10.1007\/s10278-014-9708-x","volume":"27","author":"Y Zhou","year":"2014","unstructured":"Zhou Y, Amundson PK, Yu F, Kessler MM,Benzinger TL, Wippold FJ. Automated classification of radiology reports to facilitate retrospective study in radiology. J Digit Imaging 2014;27(6):730\u2013736.","journal-title":"J Digit Imaging"},{"key":"787_CR34","doi-asserted-by":"publisher","unstructured":"Carrodeguas, Emmanuel, et al. \u201cUse of Machine Learning to Identify Follow-Up Recommendations in Radiology Reports.\u201d Journal of the American College of Radiology, vol. 16, no. 3, Mar. 2019, pp. 336\u201343. https:\/\/doi.org\/10.1016\/j.jacr.2018.10.020.","DOI":"10.1016\/j.jacr.2018.10.020"},{"issue":"7","key":"787_CR35","first-page":"2039","volume":"30","author":"BH Do","year":"2010","unstructured":"Do BH, Wu A, Biswal S, Kamaya A, Rubin DL. Informatics in radiology: RADTF\u2014a semantic search-enabled, natural language processor-generated radiology teaching file. Radio Graphics 2010;30(7):2039\u20132048","journal-title":"Radio Graphics"},{"issue":"12","key":"787_CR36","doi-asserted-by":"publisher","first-page":"1370","DOI":"10.1177\/1535370213508172","volume":"238","author":"VI Petkov","year":"2013","unstructured":"Petkov VI, Penberthy LT, Dahman BA, Poklepovic A, Gillam CW, McDermott JH. Automated determination of metastases in unstructured radiology reports for eligibility screening in oncology clinical trials. Exp Biol Med (Maywood) 2013;238(12):1370\u20131378.","journal-title":"Exp Biol Med (Maywood)"},{"key":"787_CR37","doi-asserted-by":"publisher","unstructured":"Sarker, Iqbal H. \u201cDeep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions.\u201d SN Computer Science, vol. 2, no. 6, Nov. 2021, p. 420. https:\/\/doi.org\/10.1007\/s42979-021-00815-1.","DOI":"10.1007\/s42979-021-00815-1"},{"key":"787_CR38","doi-asserted-by":"publisher","unstructured":"S. Mithun, A. K. Jha, U. K. Sherkhane, V. Jaiswar, R. V. Prasad, C. M. Ortiz, S. Puts, V. Rangarajan, A. Dekker, L. Wee. Validation of an open source Natural Language Processing (NLP) and an in-house developed python script for named entity recognition from radiology reports of lung carcinoma cases. Presented at: Annual Congress of the European Association of Nuclear Medicine October 12 \u2013 16, 2019 Barcelona, Spain. Eur J Nucl Med Mol Imaging 46, 1\u2013952 (2019). vol. 46, no. S1, Oct. 2019, pp. 1\u2013952. [Online] https:\/\/doi.org\/10.1007\/s00259-019-04486-2. Available at: https:\/\/posterng.netkey.at\/eanm\/viewing\/index.php?module=viewing_poster&task=viewsection&pi=4393&ti=32007&si=43&searchkey=#poster","DOI":"10.1007\/s00259-019-04486-2"},{"key":"787_CR39","doi-asserted-by":"publisher","unstructured":"Chen, Tianqi, and Carlos Guestrin. \u201cXGBoost: A Scalable Tree Boosting System.\u201d Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2016, pp. 785\u201394.\u00a0https:\/\/doi.org\/10.1145\/2939672.2939785.","DOI":"10.1145\/2939672.2939785"},{"key":"787_CR40","unstructured":"\u201cC-Martinez\/SARRLEK.\u201d GitHub, https:\/\/github.com\/c-martinez\/SARRLEK. Accessed 14 May 2021."},{"key":"787_CR41","unstructured":"Team, Keras. Keras Documentation: Bidirectional LSTM on IMDB. https:\/\/keras.io\/examples\/nlp\/bidirectional_lstm_imdb\/. Accessed 14 May 2021."},{"key":"787_CR42","unstructured":"\u201cExplain Neural Networks with Keras and Eli5.\u201d Depends on the Definition, 2 June 2018, https:\/\/www.depends-on-the-definition.com\/keras-and-eli5\/."},{"key":"787_CR43","doi-asserted-by":"publisher","unstructured":"Johnson, A., Pollard, T., & Mark, R. (2016). MIMIC-III Clinical Database (version 1.4). PhysioNet. https:\/\/doi.org\/10.13026\/C2XW26.","DOI":"10.13026\/C2XW26"},{"key":"787_CR44","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AEW Johnson","year":"2016","unstructured":"Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Celi, L. A., & Mark, R. G. (2016). MIMIC-III, a freely accessible critical care database. Scientific Data, 3, 160035.","journal-title":"Scientific Data"},{"key":"787_CR45","doi-asserted-by":"crossref","unstructured":"Goldberger, A., L. Amaral, L. Glass, J. Hausdorff, P. C. Ivanov, R. Mark, J. E. Mietus, G. B. Moody, C. K. Peng, and H. E. Stanley. \"PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215\u2013e220.\" (2000).","DOI":"10.1161\/01.CIR.101.23.e215"},{"key":"787_CR46","unstructured":"NCI Thesaurus. https:\/\/ncit.nci.nih.gov\/ncitbrowser\/. [Online] http:\/\/ncicb.nci.nih.gov\/xml\/owl\/EVS\/Thesaurus.owl#C4878. Accessed 15 Dec. 2020."},{"key":"787_CR47","unstructured":"NCI Thesaurus. https:\/\/ncit.nci.nih.gov\/ncitbrowser\/. [Online] http:\/\/ncicb.nci.nih.gov\/xml\/owl\/EVS\/Thesaurus.owl#C2926. Accessed 15 Dec. 2020."},{"key":"787_CR48","unstructured":"NCI Thesaurus. https:\/\/ncit.nci.nih.gov\/ncitbrowser\/. [Online] http:\/\/ncicb.nci.nih.gov\/xml\/owl\/EVS\/Thesaurus.owl#C4917. Accessed 15 Dec. 2020."},{"key":"787_CR49","doi-asserted-by":"publisher","unstructured":"Gupta, Anupama, et al. \u201cAutomatic Information Extraction from Unstructured Mammography Reports Using Distributed Semantics.\u201d\u00a0Journal of Biomedical Informatics, vol. 78, Feb. 2018, pp. 78\u201386.\u00a0https:\/\/doi.org\/10.1016\/j.jbi.2017.12.016.","DOI":"10.1016\/j.jbi.2017.12.016"},{"key":"787_CR50","doi-asserted-by":"publisher","unstructured":"H. Liu and A. Gegov, \"Rule based systems and networks: Deterministic and fuzzy approaches,\" 2016 IEEE 8th International Conference on Intelligent Systems (IS), 2016, pp. 316\u2013321. https:\/\/doi.org\/10.1109\/IS.2016.7737440.","DOI":"10.1109\/IS.2016.7737440"},{"key":"787_CR51","unstructured":"Devroye, Luc, et al. A Probabilistic Theory of Pattern Recognition. Repr, Springer, 2014."},{"key":"787_CR52","unstructured":"Patel, Ankit B., et al. \u201cA Probabilistic Theory of Deep Learning.\u201d [Cs, Stat], Apr. 2015. \narXiv.org, ArXiv:1504.00641.\u00a0"},{"key":"787_CR53","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1038\/nature14541","volume":"521","author":"Z Ghahramani","year":"2015","unstructured":"Ghahramani, Z. Probabilistic machine learning and artificial intelligence. Nature 521, 452\u2013459 (2015). https:\/\/doi.org\/10.1038\/nature14541","journal-title":"Nature"},{"key":"787_CR54","doi-asserted-by":"publisher","unstructured":"Remedios, Samuel W., et al. \u201cDistributed Deep Learning across Multisite Datasets for Generalized CT Hemorrhage Segmentation.\u201d Medical Physics, vol. 47, no. 1, Jan. 2020, pp. 89\u201398. https:\/\/doi.org\/10.1002\/mp.13880.","DOI":"10.1002\/mp.13880"},{"key":"787_CR55","unstructured":"Mikolov, Tomas, et al. \u201cDistributed Representations of Words and Phrases and Their Compositionality.\u201d Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, Curran Associates Inc., 2013, pp. 3111\u201319."},{"key":"787_CR56","first-page":"2764","volume":"11","author":"J Weston","year":"2011","unstructured":"J. Weston, S. Bengio, and N. Usunier, \u201cWsabie: Scaling up to large vocabulary image annotation,\u201d Proc. Int. Joint Conf. Artificial Intelligence., 2011, vol. 11, pp. 2764\u2013 2770.","journal-title":"Proc. Int. Joint Conf. Artificial Intelligence."},{"key":"787_CR57","unstructured":"R. Socher, C. C. Lin, C. Manning, and A. Y. Ng, \u201cParsing natural scenes and natural language with recursive neural networks,\u201d in Proc. 28th Int. Joint Conf. Machine Learning, 2011, pp. 129\u2013136."},{"key":"787_CR58","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1613\/jair.2934","volume":"37","author":"PD Turney","year":"2010","unstructured":"P. D. Turney and P. Pantel, \u201cFrom frequency to meaning: Vector space models of semantics,\u201d J. Artif. Intell. Res., vol. 37, pp. 141\u2013188, Nov. 2010.","journal-title":"J. Artif. Intell. Res."},{"issue":"6","key":"787_CR59","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1109\/MIS.2017.4531228","volume":"32","author":"E Cambria","year":"2017","unstructured":"E. Cambria, S. Poria, A. Gelbukh, and M. Thelwall, \u201cSentiment analysis is a big suitcase,\u201d IEEE Intell. Syst., vol. 32, no. 6, pp. 74\u201380, Nov. 2017.","journal-title":"IEEE Intell. Syst."},{"key":"787_CR60","unstructured":"X. Glorot, A. Bordes, and Y. Bengio, \u201cDomain adaptation for large-scale sentiment classification: A deep learning approach,\u201d in Proc. 28th Int. Conf. Machine Learning, 2011, pp. 513\u2013520."},{"key":"787_CR61","unstructured":"Hermann, Karl Moritz, and Phil Blunsom. \u201cThe Role of Syntax in Vector Space Models of Compositional Semantics.\u201d Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, 2013, pp. 894\u2013904. ACLWeb, https:\/\/aclanthology.org\/P13-1088."},{"issue":"2\u20133","key":"787_CR62","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1007\/BF00114844","volume":"7","author":"JL Elman","year":"1991","unstructured":"J. L. Elman, \u201cDistributed representations, simple recurrent networks, and grammatical structure,\u201d Mach. Learn., vol. 7, no. 2\u20133, pp. 195\u2013225, 1991.","journal-title":"Mach. Learn."},{"key":"787_CR63","doi-asserted-by":"crossref","unstructured":"J. Pennington, R. Socher, and C. D. Manning, \u201cGlove: Global vectors for word representation,\u201d in Proc. Conf. Empirical Methods Natural Language Processing, 2014, vol. 14, pp. 1532\u20131543.","DOI":"10.3115\/v1\/D14-1162"},{"key":"787_CR64","unstructured":"T. Mikolov, K. Chen, G. Corrado, and J. Dean, \u201cEfficient estimation of word representations in vector space,\u201d arXiv Preprint, arXiv:1301.3781, 2013."},{"key":"787_CR65","first-page":"2493","volume":"999888","author":"R Collobert","year":"2011","unstructured":"Collobert, R., Weston, J., Bottou, L\u00e9., Karlen, M., Kavukcuoglu, K. & Kuksa, P. (2011). Natural Language Processing (Almost) from Scratch. J. Mach. Learn. Res., 999888, 2493--2537.","journal-title":"J. Mach. Learn. Res."},{"key":"787_CR66","doi-asserted-by":"crossref","unstructured":"R. Collobert and J. Weston, \u201cA unified architecture for natural language processing: Deep neural networks with multitask learning,\u201d in Proc. 25th Int. Conf. Machine Learning, 2008, pp. 160\u2013167.","DOI":"10.1145\/1390156.1390177"},{"key":"787_CR67","doi-asserted-by":"publisher","unstructured":"Kalchbrenner, Nal, et al. \u201cA Convolutional Neural Network for Modelling Sentences.\u201d Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, 2014, pp. 655\u201365. ACLWeb, https:\/\/doi.org\/10.3115\/v1\/P14-1062.","DOI":"10.3115\/v1\/P14-1062"},{"key":"787_CR68","doi-asserted-by":"crossref","unstructured":"Y. Kim, \u201cConvolutional neural networks for sentence classification,\u201d arXiv Preprint, arXiv:1408.5882, 2014.","DOI":"10.3115\/v1\/D14-1181"},{"key":"787_CR69","doi-asserted-by":"crossref","unstructured":"S. Ruder, P. Ghaffari, and J. G. Breslin, \u201cInsight-1 at semeval-2016 task 5: Deep learning for multilingual aspect-based sentiment analysis,\u201d arXiv Preprint, arXiv:1609.02748, 2016.","DOI":"10.18653\/v1\/S16-1053"},{"key":"787_CR70","doi-asserted-by":"crossref","unstructured":"Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil, \u201cA latent semantic model with convolutional-pooling structure for information retrieval,\u201d in Proc. 23rd ACM Int. Conf. Information and Knowledge Management, 2014, pp. 101\u2013110.","DOI":"10.1145\/2661829.2661935"},{"issue":"3","key":"787_CR71","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1109\/MCI.2018.2840738","volume":"13","author":"T Young","year":"2018","unstructured":"T. Young, D. Hazarika, S. Poria and E. Cambria, \"Recent Trends in Deep Learning Based Natural Language Processing [Review Article],\" in\u00a0IEEE Computational Intelligence Magazine, vol. 13, no. 3, pp. 55-75, Aug. 2018. https:\/\/doi.org\/10.1109\/MCI.2018.2840738.","journal-title":"IEEE Computational Intelligence Magazine"},{"issue":"2","key":"787_CR72","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1207\/s15516709cog1402_1","volume":"14","author":"JL Elman","year":"1990","unstructured":"J. L. Elman, \u201cFinding structure in time,\u201d Cogn. Sci., vol. 14, no. 2, pp. 179\u2013211, 1990.","journal-title":"Cogn. Sci."},{"issue":"8","key":"787_CR73","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"S. Hochreiter and J. Schmidhuber, \u201cLong short-term memory,\u201d Neural Comput., vol. 9, no. 8, pp. 1735\u20131780,1997.","journal-title":"Neural Comput."},{"key":"787_CR74","doi-asserted-by":"crossref","unstructured":"F. A. Gers, J. Schmidhuber, and F. Cummins, \u201cLearning to forget: Continual prediction with LSTM,\u201d in Proc. 9th Int. Conf. Artificial Neural Networks, pp. 850\u2013855, 1999.","DOI":"10.1049\/cp:19991218"},{"key":"787_CR75","doi-asserted-by":"crossref","unstructured":"K. Cho, B. Van Merri\u00ebnboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, \u201cLearning phrase representations using RNN encoder-decoder for statistical machine translation,\u201d arXiv Preprint, arXiv:1406.1078, 2014.","DOI":"10.3115\/v1\/D14-1179"},{"key":"787_CR76","doi-asserted-by":"crossref","unstructured":"G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer, \u201cNeural architectures for named entity recognition,\u201d arXiv Preprint, arXiv:1603.01360, 2016.","DOI":"10.18653\/v1\/N16-1030"},{"key":"787_CR77","unstructured":"Sutskever, Ilya, et al. \u201cSequence to Sequence Learning with Neural Networks.\u201d Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, MIT Press, 2014, pp. 3104\u201312."},{"key":"787_CR78","unstructured":"D. Bahdanau, K. Cho, and Y. Bengio, \u201cNeural machine translation by jointly learning to align and translate,\u201d arXiv Preprint, arXiv:1409.0473, 2014."},{"key":"787_CR79","unstructured":"Vaswani, Ashish, et al. \u201cAttention Is All You Need.\u201d Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., 2017, pp. 6000\u201310."},{"key":"787_CR80","unstructured":"Devlin, Jacob, et al. \u201cBERT: Pre-Training of Deep Bidirectional Transformers for Language \nUnderstanding.\u201d [Cs], May 2019. arXiv.org, ArXiv:1810.04805."},{"key":"787_CR81","doi-asserted-by":"publisher","unstructured":"Yang, Xi, et al. \u201cClinical Concept Extraction Using Transformers.\u201d\u00a0Journal of the American Medical Informatics Association, vol. 27, no. 12, Dec. 2020, pp. 1935\u201342.\u00a0https:\/\/doi.org\/10.1093\/jamia\/ocaa189.","DOI":"10.1093\/jamia\/ocaa189"},{"issue":"11","key":"787_CR82","doi-asserted-by":"publisher","first-page":"1297","DOI":"10.1093\/jamia\/ocz096","volume":"26","author":"Yuqi Si","year":"2019","unstructured":"Yuqi Si, Jingqi Wang, Hua Xu, Kirk Roberts, Enhancing clinical concept extraction with contextual embeddings, Journal of the American Medical Informatics Association, Volume 26, Issue 11, November 2019, Pages 1297\u20131304. https:\/\/doi.org\/10.1093\/jamia\/ocz096.","journal-title":"Journal of the American Medical Informatics Association"},{"issue":"12","key":"787_CR83","doi-asserted-by":"publisher","DOI":"10.2196\/22982","volume":"8","author":"X Yang","year":"2020","unstructured":"Yang X, Zhang H, He X, Bian J, Wu Y, Extracting Family History of Patients From Clinical Narratives: Exploring an End-to-End Solution With Deep Learning Models, JMIR Med Inform 2020;8(12):e22982.","journal-title":"JMIR Med Inform"},{"key":"787_CR84","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun, Y., Bengio, Y. & Hinton, G. Deep learning.\u00a0Nature\u00a0521,\u00a0436\u2013444 (2015). https:\/\/doi.org\/10.1038\/nature14539","journal-title":"Nature"},{"key":"787_CR85","unstructured":"\u201cGoogle BERT Update and What You Should Know.\u201d MarketMuse, 7 Nov. 2019, https:\/\/blog.marketmuse.com\/google-bert-update\/."},{"key":"787_CR86","doi-asserted-by":"publisher","unstructured":"Ettinger, Allyson. \u201cWhat BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models.\u201d Transactions of the Association for Computational Linguistics, vol. 8, Dec. 2020, pp. 34\u201348.\u00a0https:\/\/doi.org\/10.1162\/tacl_a_00298.","DOI":"10.1162\/tacl_a_00298"},{"key":"787_CR87","unstructured":"\u201cNamed Entity Recognition.\u201d NLP-Progress, http:\/\/nlpprogress.com\/english\/named_entity_recognition.html. Accessed 13 Dec. 2022."},{"key":"787_CR88","unstructured":"\u201cIs BERT Better than LSTM?\u201d Quora, https:\/\/www.quora.com\/Is-BERT-better-than-LSTM. Accessed 10 Jan. 2023."},{"key":"787_CR89","doi-asserted-by":"publisher","unstructured":"Ezen-Can, Aysu. A Comparison of LSTM and BERT for Small Corpus. 2020. https:\/\/doi.org\/10.48550\/ARXIV.2009.05451.","DOI":"10.48550\/ARXIV.2009.05451"},{"key":"787_CR90","unstructured":"\u201cText Classification.\u201d NLP-Progress, http:\/\/nlpprogress.com\/english\/text_classification.html. Accessed 13 Dec. 2022."},{"key":"787_CR91","unstructured":"The Limitations of Deep Learning. https:\/\/blog.keras.io\/the-limitations-of-deep-learning.html. Accessed 29 June 2021."},{"key":"787_CR92","unstructured":"Overview of SNOMED CT. https:\/\/www.nlm.nih.gov\/healthit\/snomedct\/snomed_overview.html. Accessed 5 Dec 2020."},{"key":"787_CR93","unstructured":"RxNorm. https:\/\/www.nlm.nih.gov\/research\/umls\/rxnorm\/index.html. Accessed 9 Dec. 2020."},{"key":"787_CR94","doi-asserted-by":"publisher","unstructured":"Bodenreider, Olivier, and Alexa T. McCray. \u201cExploring Semantic Groups through Visual Approaches.\u201d\u00a0Journal of Biomedical Informatics, vol. 36, no. 6, Dec. 2003, pp. 414\u201332.\u00a0https:\/\/doi.org\/10.1016\/j.jbi.2003.11.002.","DOI":"10.1016\/j.jbi.2003.11.002"},{"key":"787_CR95","doi-asserted-by":"publisher","unstructured":"Savova, Guergana K., et al. \u201cMayo Clinical Text Analysis and Knowledge Extraction System (CTAKES): Architecture, Component Evaluation and Applications.\u201d\u00a0Journal of the American Medical Informatics Association, vol. 17, no. 5, Sept. 2010, pp. 507\u201313.\u00a0https:\/\/doi.org\/10.1136\/jamia.2009.001560.","DOI":"10.1136\/jamia.2009.001560"},{"key":"787_CR96","doi-asserted-by":"publisher","unstructured":"Goff, Daniel J., and Thomas W. Loehfelm. \u201cAutomated Radiology Report Summarization Using an Open-Source Natural Language Processing Pipeline.\u201d\u00a0Journal of Digital Imaging, vol. 31, no. 2, Apr. 2018, pp. 185\u201392.\u00a0https:\/\/doi.org\/10.1007\/s10278-017-0030-2.","DOI":"10.1007\/s10278-017-0030-2"},{"key":"787_CR97","doi-asserted-by":"publisher","unstructured":"Hassanpour, Saeed, and Curtis P. Langlotz. \u201cInformation Extraction from Multi-Institutional Radiology Reports.\u201d\u00a0Artificial Intelligence in Medicine, vol. 66, Jan. 2016, pp. 29\u201339.\u00a0https:\/\/doi.org\/10.1016\/j.artmed.2015.09.007.","DOI":"10.1016\/j.artmed.2015.09.007"},{"key":"787_CR98","doi-asserted-by":"publisher","unstructured":"Nobel, J. Martijn, et al. \u201cNatural Language Processing in Dutch Free Text Radiology Reports: Challenges in a Small Language Area Staging Pulmonary Oncology.\u201d\u00a0Journal of Digital Imaging, vol. 33, no. 4, Aug. 2020, pp. 1002\u201308.\u00a0https:\/\/doi.org\/10.1007\/s10278-020-00327-z.","DOI":"10.1007\/s10278-020-00327-z"},{"key":"787_CR99","unstructured":"Maxime. \u201cWhat Is a Transformer?\u201d\u00a0Medium, 5 Mar. 2020, https:\/\/medium.com\/inside-machine-learning\/what-is-a-transformer-d07dd1fbec04."},{"key":"787_CR100","doi-asserted-by":"publisher","unstructured":"Johnson, Alistair E. W., et al. \u201cMIMIC-III, a Freely Accessible Critical Care Database.\u201d\u00a0Scientific Data, vol. 3, no. 1, Dec. 2016, p. 160035.\u00a0https:\/\/doi.org\/10.1038\/sdata.2016.35.","DOI":"10.1038\/sdata.2016.35"}],"container-title":["Journal of Digital Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-023-00787-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-023-00787-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-023-00787-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,23]],"date-time":"2023-06-23T10:07:53Z","timestamp":1687514873000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-023-00787-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,14]]},"references-count":100,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["787"],"URL":"https:\/\/doi.org\/10.1007\/s10278-023-00787-z","relation":{},"ISSN":["1618-727X"],"issn-type":[{"value":"1618-727X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,14]]},"assertion":[{"value":"31 August 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 January 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 January 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 February 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This research study was conducted retrospectively from data obtained for clinical purposes. This study was approved by the institutional ethics committee with waiver of informed consent (IEC no. 1905).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"This study was approved by the institutional ethics committee as a retrospective study with waiver of informed consent.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed Consent"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}