{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T15:44:40Z","timestamp":1778600680710,"version":"3.51.4"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2021,9,4]],"date-time":"2021-09-04T00:00:00Z","timestamp":1630713600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,9,4]],"date-time":"2021-09-04T00:00:00Z","timestamp":1630713600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"published-print":{"date-parts":[[2021,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In radiology, natural language processing (NLP) allows the extraction of valuable information from radiology reports. It can be used for various downstream tasks such as quality improvement, epidemiological research, and monitoring guideline adherence. Class imbalance, variation in dataset size, variation in report complexity, and algorithm type all influence NLP performance but have not yet been systematically and interrelatedly evaluated. In this study, we investigate these factors on the performance of four types [a fully connected neural network (Dense), a long short-term memory recurrent neural network (LSTM), a convolutional neural network (CNN), and a Bidirectional Encoder Representations from Transformers (BERT)] of deep learning-based NLP.\u00a0Two datasets consisting of radiologist-annotated reports of both trauma radiographs (n\u2009=\u20092469) and chest radiographs and computer tomography (CT) studies (n\u2009=\u20092255) were split into training sets (80%) and testing sets (20%). The training data was used as a source to train all four model types in 84 experiments (Fracture-data) and 45 experiments (Chest-data) with variation in size and prevalence. The performance was evaluated on sensitivity, specificity, positive predictive value, negative predictive value, area under the curve, and F score.\u00a0After the NLP of radiology reports, all four model-architectures demonstrated high performance with metrics up to\u2009&gt;\u20090.90.\u00a0CNN, LSTM, and Dense were outperformed by the BERT algorithm because of its stable results despite variation in training size and prevalence. Awareness of variation in prevalence is warranted because it impacts sensitivity and specificity in opposite directions.<\/jats:p>","DOI":"10.1007\/s10916-021-01761-4","type":"journal-article","created":{"date-parts":[[2021,9,4]],"date-time":"2021-09-04T02:13:28Z","timestamp":1630721608000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance"],"prefix":"10.1007","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0809-9722","authenticated-orcid":false,"given":"A. W.","family":"Olthof","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"P. M. A.","family":"van Ooijen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"L. J.","family":"Cornelissen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,9,4]]},"reference":[{"issue":"3","key":"1761_CR1","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1067\/j.cpradiol.2016.11.005","volume":"46","author":"B Lee","year":"2017","unstructured":"Lee B, Whitehead MT. Radiology Reports: What YOU Think You\u2019re Saying and What THEY Think You\u2019re Saying. Curr Probl Diagn Radiol. 2017;46(3):186\u201395. https:\/\/doi.org\/10.1067\/j.cpradiol.2016.11.005","journal-title":"Curr Probl Diagn Radiol."},{"issue":"985","key":"1761_CR2","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1259\/bjr\/16360063","volume":"83","author":"FM Grieve","year":"2010","unstructured":"Grieve FM, Plumb AA, Khan SH. Radiology reporting: A general practitioner\u2019s perspective. Br J Radiol. 2010 Jan;83(985):17\u201322. https:\/\/doi.org\/10.1259\/bjr\/16360063","journal-title":"Br J Radiol."},{"issue":"3","key":"1761_CR3","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1007\/s00261-016-0679-x","volume":"41","author":"VA Sahni","year":"2016","unstructured":"Sahni VA, Khorasani R. The actionable imaging report. Abdom Radiol. 2016 Mar 10;41(3):429\u201343. https:\/\/doi.org\/10.1007\/s00261-016-0679-x","journal-title":"Abdom Radiol."},{"issue":"2","key":"1761_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10916-018-1150-z","volume":"43","author":"SJ Baccei","year":"2019","unstructured":"Baccei SJ, DiRoberto C, Greene J, Rosen MP. Improving Communication of Actionable Findings in Radiology Imaging Studies and Procedures Using an EMR-Independent System. J Med Syst 2019;43(2):1\u20136. https:\/\/doi.org\/10.1007\/s10916-018-1150-z","journal-title":"J Med Syst"},{"issue":"4","key":"1761_CR5","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1016\/j.jacr.2016.08.033","volume":"14","author":"S Jay Kabadi","year":"2017","unstructured":"Jay Kabadi S, Krishnaraj A. Strategies for improving the value of the radiology report: a retrospective analysis of errors in formally over-read studies. J Am Coll Radiol. 2017;14(4):459\u201366. https:\/\/doi.org\/10.1016\/j.jacr.2016.08.033","journal-title":"J Am Coll Radiol."},{"issue":"3","key":"1761_CR6","doi-asserted-by":"publisher","first-page":"866","DOI":"10.1148\/rg.2015140221","volume":"35","author":"A Sarwar","year":"2015","unstructured":"Sarwar A, Boland G, Monks A, Kruskal JB. Metrics for Radiologists in the Era of Value-based Health Care Delivery. Radiographics. 2015 Jan 3;35(3):866\u201376. https:\/\/doi.org\/10.1148\/rg.2015140221","journal-title":"Radiographics."},{"issue":"6","key":"1761_CR7","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1007\/s10278-019-00214-2","volume":"32","author":"AK Goel","year":"2019","unstructured":"Goel AK, DiLella D, Dotsikas G, Hilts M, Kwan D, Paxton L. Unlocking Radiology Reporting Data: an Implementation of Synoptic Radiology Reporting in Low-Dose CT Cancer Screening. J Digit Imaging. 2019 Dec 1;32(6):1044\u201351. https:\/\/doi.org\/10.1007\/s10278-019-00214-2","journal-title":"J Digit Imaging."},{"issue":"2","key":"1761_CR8","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1111\/acem.12859","volume":"23","author":"K Yadav","year":"2016","unstructured":"Yadav K, Sarioglu E, Choi HA, Cartwright WB 4th, Hinds PS, Chamberlain JM. Automated Outcome Classification of Computed Tomography Imaging Reports for Pediatric Traumatic Brain Injury. Acad Emerg Med. 2016 Feb;23(2):171\u20138. https:\/\/doi.org\/10.1111\/acem.12859","journal-title":"Acad Emerg Med."},{"issue":"5","key":"1761_CR9","doi-asserted-by":"publisher","first-page":"2632","DOI":"10.1007\/s00330-018-5954-5","volume":"29","author":"E Pons","year":"2019","unstructured":"Pons E, Foks KA, Dippel DWJ, Hunink MGM. Impact of guidelines for the management of minor head injury on the utilization and diagnostic yield of CT over two decades, using natural language processing in a large dataset. Eur Radiol. 2019;29(5):2632\u201340. https:\/\/doi.org\/10.1007\/s00330-018-5954-5","journal-title":"Eur Radiol."},{"issue":"5","key":"1761_CR10","doi-asserted-by":"publisher","first-page":"598","DOI":"10.1177\/0284185114532922","volume":"56","author":"G Issa","year":"2015","unstructured":"Issa G, Taslakian B, Itani M, Hitti E, Batley N, Saliba M, et al. The discrepancy rate between preliminary and official reports of emergency radiology studies: a performance indicator and quality improvement method. Acta Radiol. 2015 May 1;56(5):598\u2013604. https:\/\/doi.org\/10.1177\/0284185114532922","journal-title":"Acta Radiol."},{"key":"1761_CR11","doi-asserted-by":"publisher","unstructured":"Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. In: Ivers N, editor. Cochrane Database of Systematic Reviews. Chichester, UK: John Wiley & Sons, Ltd; 2012. https:\/\/doi.org\/10.1002\/14651858.CD000259.pub3","DOI":"10.1002\/14651858.CD000259.pub3"},{"key":"1761_CR12","doi-asserted-by":"publisher","unstructured":"Spasic I, Nenadic G, Goran Nenadic. Clinical Text Data in Machine Learning: Systematic Review. JMIR Med Informatics. 2020;8(3). https:\/\/doi.org\/10.2196\/17984","DOI":"10.2196\/17984"},{"key":"1761_CR13","doi-asserted-by":"publisher","unstructured":"Wang J, Deng H, Liu B, Hu A, Liang J, Fan L, et al. Systematic evaluation of research progress on natural language processing in medicine over the past 20 years: Bibliometric study on pubmed. J Med Internet Res. 2020;22(1). https:\/\/doi.org\/10.2196\/16816","DOI":"10.2196\/16816"},{"issue":"4","key":"1761_CR14","doi-asserted-by":"publisher","first-page":"554","DOI":"10.1007\/s10278-019-00234-y","volume":"32","author":"JM Steinkamp","year":"2019","unstructured":"Steinkamp JM, Chambers C, Lalevic D, Zafar HM, Cook TS. Toward Complete Structured Information Extraction from Radiology Reports Using Machine Learning. J Digit Imaging. 2019;32(4):554\u201364. https:\/\/doi.org\/10.1007\/s10278-019-00234-y","journal-title":"J Digit Imaging."},{"issue":"2","key":"1761_CR15","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1148\/radiol.2018171093","volume":"287","author":"J Zech","year":"2018","unstructured":"Zech J, Pain M, Titano J, Badgeley M, Schefflein J, Su A, et al. Natural Language\u2013based Machine Learning Models for the Annotation of Clinical Radiology Reports. Radiology. 2018;287(2):570\u201380. https:\/\/doi.org\/10.1148\/radiol.2018171093","journal-title":"Radiology."},{"key":"1761_CR16","doi-asserted-by":"publisher","unstructured":"Jungmann F, K\u00e4mpgen B, Mildenberger P, Tsaur I, Jorg T, D\u00fcber C, et al. Towards data-driven medical imaging using natural language processing in patients with suspected urolithiasis. Int J Med Inform. 2020;137. https:\/\/doi.org\/10.1016\/j.ijmedinf.2020.104106","DOI":"10.1016\/j.ijmedinf.2020.104106"},{"issue":"1","key":"1761_CR17","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1016\/j.acra.2019.08.010","volume":"27","author":"PH Chen","year":"2020","unstructured":"Chen PH. Essential Elements of Natural Language Processing: What the Radiologist Should Know. Acad Radiol. 2020 Sep 16;27(1):6\u201312. https:\/\/doi.org\/10.1016\/j.acra.2019.08.010","journal-title":"Acad Radiol."},{"issue":"2","key":"1761_CR18","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1007\/s10278-017-0027-x","volume":"31","author":"P-H Chen","year":"2018","unstructured":"Chen P-H, Zafar H, Galperin-Aizenberg M, Cook T. Integrating Natural Language Processing and Machine Learning Algorithms to Categorize Oncologic Response in Radiology Reports. J Digit Imaging. 2018 Apr;31(2):178\u201384. https:\/\/doi.org\/10.1007\/s10278-017-0027-x","journal-title":"J Digit Imaging."},{"issue":"4","key":"1761_CR19","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1016\/j.nic.2020.08.001","volume":"30","author":"JW Luo","year":"2020","unstructured":"Luo JW, Chong JJR. Review of Natural Language Processing in Radiology. Neuroimaging Clin N Am. 2020 Nov 1;30(4):447\u201358. https:\/\/doi.org\/10.1016\/j.nic.2020.08.001","journal-title":"Neuroimaging Clin N Am."},{"issue":"1","key":"1761_CR20","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1186\/s12911-021-01451-8","volume":"21","author":"FA Dahl","year":"2021","unstructured":"Dahl FA, Rama T, Hurlen P, Brekke PH, Husby H, Gundersen T, et al. Neural classification of Norwegian radiology reports: using NLP to detect findings in CT-scans of children. BMC Med Inform Decis Mak. 2021;21(1):84. https:\/\/doi.org\/10.1186\/s12911-021-01451-8","journal-title":"BMC Med Inform Decis Mak."},{"issue":"1","key":"1761_CR21","doi-asserted-by":"publisher","DOI":"10.2196\/19689","volume":"23","author":"H Chen","year":"2020","unstructured":"Chen H, Liu H, Wang N, Huang Y, Zhang Z, Xu Y, et al. Use of BERT (Bidirectional Encoder Representations from Transformers)-Based Deep Learning Method for Extracting Evidences in Chinese Radiology Reports: Development of a Computer-Aided Liver Cancer Diagnosis Framework. J Med Internet Res. 2020;23(1):e19689. https:\/\/doi.org\/10.2196\/19689","journal-title":"J Med Internet Res."},{"key":"1761_CR22","doi-asserted-by":"publisher","unstructured":"Bressem KK, Adams LC, Gaudin RA, Tr\u00f6ltzsch D, Hamm B, Makowski MR, et al. Highly accurate classification of chest radiographic reports using a deep learning natural language model pre-trained on 3.8 million text reports. Bioinformatics. 2021;36(21):5255\u201361. https:\/\/doi.org\/10.1093\/bioinformatics\/btaa668","DOI":"10.1093\/bioinformatics\/btaa668"},{"key":"1761_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1200\/CCI.19.00034","volume":"3","author":"I Banerjee","year":"2019","unstructured":"Banerjee I, Bozkurt S, Caswell-Jin JL, Kurian AW, Rubin DL. Natural Language Processing Approaches to Detect the Timeline of Metastatic Recurrence of Breast Cancer. JCO Clin cancer informatics. 2019 Oct;3:1\u201312. https:\/\/doi.org\/10.1200\/CCI.19.00034","journal-title":"JCO Clin cancer informatics."},{"issue":"10","key":"1761_CR24","doi-asserted-by":"publisher","first-page":"1247","DOI":"10.1007\/s00234-020-02420-0","volume":"62","author":"Y Barash","year":"2020","unstructured":"Barash Y, Guralnik G, Tau N, Soffer S, Levy T, Shimon O, et al. Comparison of deep learning models for natural language processing-based classification of non-English head CT reports. Neuroradiology. 2020 Oct 1;62(10):1247\u201356. https:\/\/doi.org\/10.1007\/s00234-020-02420-0","journal-title":"Neuroradiology."},{"issue":"1","key":"1761_CR25","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","volume":"25","author":"A Esteva","year":"2019","unstructured":"Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M, Chou K, et al. A guide to deep learning in healthcare. Nat Med. 2019 Jan 7;25(1):24\u20139. https:\/\/doi.org\/10.1038\/s41591-018-0316-z","journal-title":"Nat Med."},{"key":"1761_CR26","doi-asserted-by":"publisher","unstructured":"Chartrand G, Cheng PM, Eugene Vorontsov M, Eng Sci Michal Drozdzal Bas, Turcotte S, Pal CJ, et al. Deep Learning: A Primer for Radiologists 1 From the Departments of Radiology (G. RadioGraphics). 2017;37:2113\u201331. https:\/\/doi.org\/10.1148\/rg.2017170077","DOI":"10.1148\/rg.2017170077"},{"issue":"1","key":"1761_CR27","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1186\/s40537-019-0192-5","volume":"6","author":"JM Johnson","year":"2019","unstructured":"Johnson JM, Khoshgoftaar TM. Survey on deep learning with class imbalance. J Big Data. 2019;6(1):27. https:\/\/doi.org\/10.1186\/s40537-019-0192-5","journal-title":"J Big Data."},{"key":"1761_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11548-020-02260-6","volume":"23","author":"W Qu","year":"2020","unstructured":"Qu W, Balki I, Mendez M, Valen J, Levman J, Tyrrell PN. Assessing and mitigating the effects of class imbalance in machine learning with application to X-ray imaging. Int J Comput Assist Radiol Surg. 2020 Sep 23;1\u20138. https:\/\/doi.org\/10.1007\/s11548-020-02260-6","journal-title":"Int J Comput Assist Radiol Surg."},{"issue":"4","key":"1761_CR29","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1016\/j.carj.2019.06.002","volume":"70","author":"I Balki","year":"2019","unstructured":"Balki I, Amirabadi A, Levman J, Martel AL, Emersic Z, Meden B, et al. Sample-Size Determination Methodologies for Machine Learning in Medical Imaging Research: A Systematic Review. Can Assoc Radiol J. 2019 Nov 1;70(4):344\u201353. https:\/\/doi.org\/10.1016\/j.carj.2019.06.002","journal-title":"Can Assoc Radiol J."},{"issue":"9","key":"1761_CR30","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1007\/s10916-020-01604-8","volume":"44","author":"HB Fevrier","year":"2020","unstructured":"Fevrier HB, Liu L, Herrinton LJ, Li D. A Transparent and Adaptable Method to Extract Colonoscopy and Pathology Data Using Natural Language Processing. J Med Syst. 2020;44(9):151. https:\/\/doi.org\/10.1007\/s10916-020-01604-8","journal-title":"J Med Syst."},{"issue":"2","key":"1761_CR31","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.2020200029","volume":"2","author":"J Mongan","year":"2020","unstructured":"Mongan J, Moy L, Kahn CE. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. Radiol Artif Intell. 2020;2(2):e200029. https:\/\/doi.org\/10.1148\/ryai.2020200029","journal-title":"Radiol Artif Intell."},{"key":"1761_CR32","doi-asserted-by":"publisher","unstructured":"Li F, Jin Y, Liu W, Rawat BPS, Cai P, Yu H. Fine-tuning bidirectional encoder representations from transformers (BERT)\u2013based models on large-scale electronic health record notes: An empirical study. J Med Internet Res. 2019;21(9). https:\/\/doi.org\/10.2196\/14830","DOI":"10.2196\/14830"},{"key":"1761_CR33","unstructured":"de Vries W, van Cranenburgh A, Bisazza A, Caselli T, van Noord G, Nissim M. BERTje: A Dutch BERT Model. arXiv. 2019;\u00a01912.09582."},{"key":"1761_CR34","doi-asserted-by":"publisher","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, et al. Transformers: State-of-the-Art Natural Language Processing. arXiv. 2020;1910.03771v5. https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-demos.6","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"1761_CR35","doi-asserted-by":"publisher","unstructured":"Weikert T, Nesic I, Cyriac J, Bremerich J, Sauter AW, Sommer G, et al. Towards automated generation of curated datasets in radiology: Application of natural language processing to unstructured reports exemplified on CT for pulmonary embolism. Eur J Radiol. 2020;125. https:\/\/doi.org\/10.1016\/j.ejrad.2020.108862","DOI":"10.1016\/j.ejrad.2020.108862"},{"key":"1761_CR36","doi-asserted-by":"publisher","unstructured":"Krsnik I, Glava\u0161 G, Krsnik M, Miletic D, \u0160tajduhar I. Automatic annotation of narrative radiology reports. Diagnostics. 2020;10(4). https:\/\/doi.org\/10.3390\/diagnostics10040196","DOI":"10.3390\/diagnostics10040196"},{"key":"1761_CR37","doi-asserted-by":"publisher","unstructured":"Datta S, Si Y, Rodriguez L, Shooshan SE, Demner-Fushman D, Roberts K. Understanding spatial language in radiology: Representation framework, annotation, and spatial relation extraction from chest X-ray reports using deep learning. J Biomed Inform. 2020;108. https:\/\/doi.org\/10.1016\/j.jbi.2020.103473","DOI":"10.1016\/j.jbi.2020.103473"},{"key":"1761_CR38","doi-asserted-by":"publisher","unstructured":"Bressem KK, Adams LC, Gaudin RA, Tr\u00f6ltzsch D, Hamm B, Makowski MR, et al. Highly accurate classification of chest radiographic reports using a deep learning natural language model pretrained on 3.8 million text reports. Bioinformatics. 2020.\u00a0https:\/\/doi.org\/10.1093\/bioinformatics\/btaa668\/5875602","DOI":"10.1093\/bioinformatics\/btaa668\/5875602"},{"issue":"4","key":"1761_CR39","doi-asserted-by":"publisher","first-page":"606","DOI":"10.1055\/s-0040-1715892","volume":"11","author":"W Bala","year":"2020","unstructured":"Bala W, Steinkamp J, Feeney T, Gupta A, Sharma A, Kantrowitz J, et al. A Web application for adrenal incidentaloma identification, tracking, and management using machine learning. Appl Clin Inform. 2020 Aug 1;11(4):606\u201316. https:\/\/doi.org\/10.1055\/s-0040-1715892","journal-title":"Appl Clin Inform."},{"key":"1761_CR40","doi-asserted-by":"publisher","DOI":"10.1007\/s10278-019-00271-7","author":"R Lou","year":"2019","unstructured":"Lou R, Lalevic D, Chambers C, Zafar HM, Cook TS. Automated Detection of Radiology Reports that Require Follow-up Imaging Using Natural Language Processing Feature Engineering and Machine Learning Classification. J Digit Imaging. 2019 Sep; https:\/\/doi.org\/10.1007\/s10278-019-00271-7","journal-title":"J Digit Imaging."},{"issue":"2","key":"1761_CR41","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1016\/j.jacr.2019.07.018","volume":"17","author":"VI Valtchinov","year":"2020","unstructured":"Valtchinov VI, Lacson R, Wang A, Khorasani R. Comparing Artificial Intelligence Approaches to Retrieve Clinical Reports Documenting Implantable Devices Posing MRI Safety Risks. J Am Coll Radiol. 2020 Feb 1;17(2):272\u20139. https:\/\/doi.org\/10.1016\/j.jacr.2019.07.018","journal-title":"J Am Coll Radiol."},{"key":"1761_CR42","doi-asserted-by":"publisher","unstructured":"Ong CJ, Orfanoudaki A, Zhang R, Caprasse FPM, Hutch M, Ma L, et al. Machine learning and natural language processing methods to identify ischemic stroke, acuity and location from radiology reports. PLoS One. 2020;15(6). https:\/\/doi.org\/10.1371\/journal.pone.0234908","DOI":"10.1371\/journal.pone.0234908"},{"key":"1761_CR43","doi-asserted-by":"publisher","unstructured":"Gao S, Alawad M, Young MT, Gounley J, Schaefferkoetter N, Yoon HJ, et al. Limitations of Transformers on Clinical Text Classification. IEEE J Biomed Heal Informatics. 2021;PP. https:\/\/doi.org\/10.1109\/JBHI.2021.3062322","DOI":"10.1109\/JBHI.2021.3062322"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-021-01761-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-021-01761-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-021-01761-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T04:13:48Z","timestamp":1632284028000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-021-01761-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,4]]},"references-count":43,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2021,10]]}},"alternative-id":["1761"],"URL":"https:\/\/doi.org\/10.1007\/s10916-021-01761-4","relation":{},"ISSN":["0148-5598","1573-689X"],"issn-type":[{"value":"0148-5598","type":"print"},{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,4]]},"assertion":[{"value":"30 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 August 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 September 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Institutional Review Board approval was not required because of the retrospective nature of the study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Written informed consent was not required for this study because patient data were anonymously used for the study and no interventions took place for the study.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest\/Competing interests"}}],"article-number":"91"}}