{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T08:24:51Z","timestamp":1778055891434,"version":"3.51.4"},"reference-count":32,"publisher":"Georg Thieme Verlag KG","issue":"04","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2016,10]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>Background Important information to support healthcare quality improvement is often recorded in free text documents such as radiology reports. Natural language processing (NLP) methods may help extract this information, but these methods have rarely been applied outside the research laboratories where they were developed.<\/jats:p><jats:p>Objective To implement and validate NLP tools to identify long bone fractures for pediatric emergency medicine quality improvement.<\/jats:p><jats:p>Methods Using freely available statistical software packages, we implemented NLP methods to identify long bone fractures from radiology reports. A sample of 1,000 radiology reports was used to construct three candidate classification models. A test set of 500 reports was used to validate the model performance. Blinded manual review of radiology reports by two independent physicians provided the reference standard. Each radiology report was segmented and word stem and bigram features were constructed. Common English \u201cstop words\u201d and rare features were excluded. We used 10-fold cross-validation to select optimal configuration parameters for each model. Accuracy, recall, precision and the F1 score were calculated. The final model was compared to the use of diagnosis codes for the identification of patients with long bone fractures.<\/jats:p><jats:p>Results There were 329 unique word stems and 344 bigrams in the training documents. A support vector machine classifier with Gaussian kernel performed best on the test set with accuracy=0.958, recall=0.969, precision=0.940, and F1 score=0.954. Optimal parameters for this model were cost=4 and gamma=0.005. The three classification models that we tested all performed better than diagnosis codes in terms of accuracy, precision, and F1 score (diagnosis code accuracy=0.932, recall=0.960, precision=0.896, and F1 score=0.927).<\/jats:p><jats:p>Conclusions NLP methods using a corpus of 1,000 training documents accurately identified acute long bone fractures from radiology reports. Strategic use of straightforward NLP methods, implemented with freely available software, offers quality improvement teams new opportunities to extract information from narrative documents.<\/jats:p><jats:p>Citation: Grundmeier RW, Masino AJ, Casper TC, Dean JM, Bell J, Enriquez R, Deakyne S, Chamberlain JM, Alpern ER. Identification of long bone fractures in radiology reports using natural language processing to support healthcare quality improvement.<\/jats:p>","DOI":"10.4338\/aci-2016-08-ra-0129","type":"journal-article","created":{"date-parts":[[2016,11,9]],"date-time":"2016-11-09T03:11:20Z","timestamp":1478661080000},"page":"1051-1068","source":"Crossref","is-referenced-by-count":33,"title":["Identification of Long Bone Fractures in Radiology Reports Using Natural Language Processing to support Healthcare Quality Improvement"],"prefix":"10.4338","volume":"07","author":[{"given":"Aaron","family":"Masino","sequence":"first","affiliation":[]},{"given":"T.","family":"Casper","sequence":"first","affiliation":[]},{"given":"Jonathan","family":"Dean","sequence":"first","affiliation":[]},{"given":"Jamie","family":"Bell","sequence":"first","affiliation":[]},{"given":"Rene","family":"Enriquez","sequence":"first","affiliation":[]},{"given":"Sara","family":"Deakyne","sequence":"first","affiliation":[]},{"given":"James","family":"Chamberlain","sequence":"first","affiliation":[]},{"given":"Elizabeth","family":"Alpern","sequence":"first","affiliation":[]},{"given":"Robert","family":"Grundmeier","sequence":"additional","affiliation":[]},{"name":"The Pediatric Emergency Care Applied Research Network","sequence":"additional","affiliation":[]}],"member":"194","published-online":{"date-parts":[[2017,12,18]]},"reference":[{"key":"10.4338\/ACI-2016-08-RA-0129-1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1000387"},{"key":"10.4338\/ACI-2016-08-RA-0129-2","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-015-0274-7"},{"key":"10.4338\/ACI-2016-08-RA-0129-3","doi-asserted-by":"publisher","DOI":"10.1097\/MLR.0b013e31829b1dbd"},{"issue":"1","key":"10.4338\/ACI-2016-08-RA-0129-4","first-page":"1025","volume":"1","author":"Devine","year":"2013","journal-title":"EGEMS (Washington, DC)"},{"key":"10.4338\/ACI-2016-08-RA-0129-5","doi-asserted-by":"publisher","DOI":"10.18553\/jmcp.2014.20.2.165"},{"key":"10.4338\/ACI-2016-08-RA-0129-6","doi-asserted-by":"publisher","DOI":"10.1097\/01.mlr.0000132391.59713.0d"},{"issue":"4","key":"10.4338\/ACI-2016-08-RA-0129-7","first-page":"720","volume":"21","author":"Heintzman","year":"2014","journal-title":"The Oxford University Press"},{"key":"10.4338\/ACI-2016-08-RA-0129-8","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajem.2015.03.001"},{"key":"10.4338\/ACI-2016-08-RA-0129-9","doi-asserted-by":"publisher","DOI":"10.1111\/acem.12174"},{"key":"10.4338\/ACI-2016-08-RA-0129-10","doi-asserted-by":"crossref","unstructured":"Friedlin J, Mahoui M, Jones J, Jamieson P. Knowledge Discovery and Data Mining of Free Text Radiology Reports. IEEE 2011; 89-96","DOI":"10.1109\/HISB.2011.31"},{"key":"10.4338\/ACI-2016-08-RA-0129-11","first-page":"1a","volume":"7","author":"Womack","year":"2010","journal-title":"Perspectives in health information management \/ AHIMA, American Health Information Management Association"},{"key":"10.4338\/ACI-2016-08-RA-0129-12","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/6.4.344"},{"key":"10.4338\/ACI-2016-08-RA-0129-13","unstructured":"Alpern ER, Alessandrini EA, Casper TC, Bajaj L, Gorelick MH, Gerber JS, Funai T, Grundmeier RW, Enriquez R, Dean JM, Campos DA, Deakyne SJ, Bell J, Hayes KL, Kittick M, Chamberlain JM. Benchmarks in Pediatric Emergency Medicine Performance Measures Derived from an Multicenter Electronic Health Record Registry. Platform Presentation at the Pediatric Academic Societies. San Diego, CA 2015"},{"key":"10.4338\/ACI-2016-08-RA-0129-14","unstructured":"Deakyne SJ, Grundmeier RW, Campos DA, Hayes KL, Cao J, Enriquez R, Bell J, Fahim C, Casper TC, Funai T, Scheid B, Kittick M, Dean JM, Alessandrini EA, Bajaj L, Gorelick MH, Chamberlain JM, Alpern ER. Building a Pediatric Emergency Care Electronic Medical Registry. Poster Session at the Pediatric Academic Societies. San Diego, CA 2015"},{"issue":"23","key":"10.4338\/ACI-2016-08-RA-0129-15","first-page":"E215","volume":"101","author":"Goldberger","year":"2000","journal-title":"Lippincott Williams & Wilkins"},{"key":"10.4338\/ACI-2016-08-RA-0129-16","doi-asserted-by":"publisher","DOI":"10.1186\/1471-244X-8-32"},{"key":"10.4338\/ACI-2016-08-RA-0129-17","unstructured":"R Core Team. R: A Language and Environment for Statistical Computing [Internet]. 3rd ed. Vienna, Austria: R Foundation for Statistical Computing. Available from: http:\/\/www.R-project.org\/"},{"issue":"5","key":"10.4338\/ACI-2016-08-RA-0129-18","first-page":"544","volume":"18","author":"Nadkarni","year":"2011","journal-title":"The Oxford University Press"},{"key":"10.4338\/ACI-2016-08-RA-0129-19","unstructured":"Feinerer I, Hornik K. Package \u201ctm\u201d [Internet]. cran.r-project.org. 2015 [cited 2015 Sep 10]. pp. 34-5. Available from: https:\/\/cran.r-project.org\/web\/packages\/tm\/tm.pdf"},{"key":"10.4338\/ACI-2016-08-RA-0129-20","doi-asserted-by":"publisher","DOI":"10.1108\/eb046814"},{"key":"10.4338\/ACI-2016-08-RA-0129-21","unstructured":"Bouchet-Valat M. Package \u201cSnowballC\u201d [Internet]. cran.r-project.org. 2014 [cited 2015 Sep 10]. Available from: https:\/\/cran.r-project.org\/web\/packages\/SnowballC\/SnowballC.pdf"},{"issue":"3","key":"10.4338\/ACI-2016-08-RA-0129-22","first-page":"273","volume":"20","author":"Cortes","year":"1995","journal-title":"Kluwer Academic Publishers"},{"key":"10.4338\/ACI-2016-08-RA-0129-23","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v033.i01"},{"key":"10.4338\/ACI-2016-08-RA-0129-24","doi-asserted-by":"publisher","DOI":"10.4338\/ACI-2014-11-RA-0110"},{"key":"10.4338\/ACI-2016-08-RA-0129-25","doi-asserted-by":"publisher","DOI":"10.1007\/s10278-012-9531-1"},{"issue":"e2","key":"10.4338\/ACI-2016-08-RA-0129-26","first-page":"e341","volume":"20","author":"Pathak","year":"2013","journal-title":"The Oxford University Press"},{"issue":"6","key":"10.4338\/ACI-2016-08-RA-0129-27","first-page":"e66341","volume":"8","author":"Lasko","year":"2013","journal-title":"Public Library of Science"},{"key":"10.4338\/ACI-2016-08-RA-0129-28","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2015.09.007"},{"issue":"1","key":"10.4338\/ACI-2016-08-RA-0129-29","first-page":"69","volume":"23","author":"Widmer","year":"1996","journal-title":"Kluwer Academic Publishers-Plenum Publishers"},{"key":"10.4338\/ACI-2016-08-RA-0129-30","unstructured":"Learning under Concept Drift: an Overview. Vol. cs.AI, arXiv.org 2010"},{"key":"10.4338\/ACI-2016-08-RA-0129-31","doi-asserted-by":"crossref","unstructured":"Holzinger A. Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inf. Berlin: Springer 2016; 1-13","DOI":"10.1007\/978-3-319-50478-0_1"},{"issue":"5","key":"10.4338\/ACI-2016-08-RA-0129-32","first-page":"839","volume":"42","author":"Harkema","year":"2009","journal-title":"JBI"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.4338\/ACI-2016-08-RA-0129.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,9,15]],"date-time":"2019-09-15T07:40:41Z","timestamp":1568533241000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.4338\/ACI-2016-08-RA-0129"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,10]]},"references-count":32,"journal-issue":{"issue":"04","published-online":{"date-parts":[[2017,12,18]]},"published-print":{"date-parts":[[2016]]}},"URL":"https:\/\/doi.org\/10.4338\/aci-2016-08-ra-0129","relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,10]]}}}