{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,25]],"date-time":"2026-10-25T09:13:07Z","timestamp":1792919587155,"version":"4.5.0"},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,4,16]],"date-time":"2022-04-16T00:00:00Z","timestamp":1650067200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,4,16]],"date-time":"2022-04-16T00:00:00Z","timestamp":1650067200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The BoneXpert method for automated determination of bone age from hand X-rays was introduced in 2009 and is currently running in over 200 hospitals. The aim of this work\u00a0is to present version 3 of the method and validate its accuracy and self-validation mechanism that automatically rejects an image if it is at risk of being analysed incorrectly. The training set included 14,036 images from the 2017 Radiological Society of North America (RSNA) Bone Age Challenge, 1642 images of normal Dutch and Californian children, and 8250 images from T\u00fcbingen from patients with Short Stature,\u00a0Congenital Adrenal Hyperplasia and\u00a0Precocious Puberty. The\u00a0study resulted in a cross-validated root mean square (RMS) error in the T\u00fcbingen images of 0.62 y, compared to 0.72 y in the previous version. The RMS error on the RSNA test set of 200 images was 0.45 y relative to the average of six manual ratings. The self-validation mechanism rejected 0.4% of the RSNA images. 121 outliers among the self-validated images of the T\u00fcbingen study were rerated, resulting in 6 cases where BoneXpert deviated more than 1.5\u00a0years from the average of the three re-ratings, compared to 72 such cases for the original manual ratings. The accuracy of BoneXpert is clearly better than the accuracy of a single manual rating. The self-validation mechanism rejected very few images, typically with abnormal anatomy, and among the accepted images, there were 12 times fewer severe bone age errors than in manual ratings, suggesting that BoneXpert could be safer than manual rating.<\/jats:p>","DOI":"10.1038\/s41598-022-10292-y","type":"journal-article","created":{"date-parts":[[2022,4,16]],"date-time":"2022-04-16T13:03:43Z","timestamp":1650114223000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":56,"title":["Accuracy and self-validation of automated bone age determination"],"prefix":"10.1038","volume":"12","author":[{"given":"D. D.","family":"Martin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A. D.","family":"Calder","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"M. B.","family":"Ranke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"G.","family":"Binder","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9607-882X","authenticated-orcid":false,"given":"H. H.","family":"Thodberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,4,16]]},"reference":[{"key":"10292_CR1","volume-title":"Radiographic atlas of skeletal development of the hand and wrist","author":"WW Greulich","year":"1959","unstructured":"Greulich, W. W. & Pyle, S. I. Radiographic atlas of skeletal development of the hand and wrist 2nd edn. (Stanford University Press, California, 1959).","edition":"2"},{"key":"10292_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1159\/000329372","volume":"76","author":"DD Martin","year":"2011","unstructured":"Martin, D. D. et al. The use of bone age in clinical practice - part 1. Horm. Res. Paediatr. 76, 1\u20139 (2011).","journal-title":"Horm. Res. Paediatr."},{"key":"10292_CR3","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1159\/000329374","volume":"76","author":"DD Martin","year":"2011","unstructured":"Martin, D. D. et al. The use of bone age in clinical practice - part 2. Horm. Res. Paediatr. 76, 10\u201316 (2011).","journal-title":"Horm. Res. Paediatr."},{"key":"10292_CR4","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1109\/TMI.2008.926067","volume":"28","author":"HH Thodberg","year":"2009","unstructured":"Thodberg, H. H., Kreiborg, S., Juul, A. & Pedersen, K. D. The BoneXpert method for automated determination of skeletal maturity. IEEE Trans. Med. Imaging 28, 52\u201366 (2009).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"10292_CR5","doi-asserted-by":"publisher","first-page":"690","DOI":"10.1007\/s00247-010-1915-0","volume":"41","author":"P Kaplowitz","year":"2010","unstructured":"Kaplowitz, P. et al. Comparison of bone age readings by pediatric endocrinologists and pediatric radiologists using two bone age atlases. Pediatr. Radiol. 41, 690\u2013693 (2010).","journal-title":"Pediatr. Radiol."},{"key":"10292_CR6","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1148\/radiol.2018180736","volume":"290","author":"SS Halabi","year":"2019","unstructured":"Halabi, S. S. et al. The RSNA pediatric bone age machine learning challenge. Radiology 290, 498\u2013503 (2019).","journal-title":"Radiology"},{"key":"10292_CR7","doi-asserted-by":"publisher","first-page":"e190053","DOI":"10.1148\/ryai.2019190053","volume":"1","author":"I Pan","year":"2019","unstructured":"Pan, I., Thodberg, H. H., Halabi, S. S., Kalpathy-Cramer, J. & Larson, D. B. Improving automated pediatric bone age estimation using ensembles of models from the 2017 RSNA machine learning challenge. Radiol. Artif. Intell. 1, e190053 (2019).","journal-title":"Radiol. Artif. Intell."},{"issue":"1","key":"10292_CR8","first-page":"1","volume":"2022","author":"HH Thodberg","year":"2022","unstructured":"Thodberg, H. H., Thodberg, B., Ahlkvist, J. & Offiah, A. C. Autonomous artificial intelligence in pediatric radiology: the use and perception of BoneXpert for bone age assessment. Pediatr. Radiol. 2022(1), 1\u20139 (2022).","journal-title":"Pediatr. Radiol."},{"key":"10292_CR9","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1007\/s00247-015-3468-8","volume":"46","author":"DD Martin","year":"2016","unstructured":"Martin, D. D., Schittenhelm, J. & Thodberg, H. H. Validation of adult height prediction based on automated bone age determination in the Paris Longitudinal Study of healthy children. Pediatr. Radiol. 46, 263\u2013269 (2016).","journal-title":"Pediatr. Radiol."},{"key":"10292_CR10","doi-asserted-by":"publisher","first-page":"598","DOI":"10.1007\/s00247-008-1114-4","volume":"39","author":"DD Martin","year":"2009","unstructured":"Martin, D. D. et al. Clinical application of automated Greulich-Pyle bone age determination in children with short stature. Pediatr. Radiol. 39, 598\u2013607 (2009).","journal-title":"Pediatr. Radiol."},{"key":"10292_CR11","doi-asserted-by":"publisher","first-page":"1615","DOI":"10.1007\/s00247-013-2744-8","volume":"43","author":"DD Martin","year":"2013","unstructured":"Martin, D. D. et al. Validation of automatic bone age determination in children with congenital adrenal hyperplasia. Pediatr. Radiol. 43, 1615\u20131621 (2013).","journal-title":"Pediatr. Radiol."},{"key":"10292_CR12","doi-asserted-by":"publisher","first-page":"1009","DOI":"10.1515\/JPEM.2011.420","volume":"24","author":"DD Martin","year":"2011","unstructured":"Martin, D. D. et al. Validation of automatic bone age rating in children with precocious and early puberty. J. Pediatr. Endocrinol. Metab. 24, 1009\u20131014 (2011).","journal-title":"J. Pediatr. Endocrinol. Metab."},{"key":"10292_CR13","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1007\/s00223001119","volume":"67","author":"MH Lequin","year":"2000","unstructured":"Lequin, M. H., van Rijn, R. R., Robben, S. G., Hop, W. C. & van Kuijk, C. Normal values for tibial quantitative ultrasonometry in caucasian children and adolescents (aged 6 to 19 years). Calcif. Tissue Int. 67, 101\u2013105 (2000).","journal-title":"Calcif. Tissue Int."},{"key":"10292_CR14","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1007\/s00247-008-1090-8","volume":"39","author":"RR van Rijn","year":"2009","unstructured":"van Rijn, R. R., Lequin, M. H. & Thodberg, H. H. Automatic determination of Greulich and Pyle bone age in healthy Dutch children. Pediatr. Radiol. 39, 591\u2013597 (2009).","journal-title":"Pediatr. Radiol."},{"key":"10292_CR15","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1148\/radiol.2493080468","volume":"250","author":"A Zhang","year":"2009","unstructured":"Zhang, A., Sayre, J. W., Vachon, L., Liu, B. J. & Huang, H. K. Racial differences in growth patterns of children assessed on the basis of bone age. Radiology 250, 228\u2013235 (2009).","journal-title":"Radiology"},{"key":"10292_CR16","doi-asserted-by":"publisher","first-page":"1425","DOI":"10.1016\/j.acra.2010.06.007","volume":"17","author":"HH Thodberg","year":"2010","unstructured":"Thodberg, H. H. & S\u00e4vendahl, L. Validation and reference values of automated bone age determination for four ethnicities. Acad. Radiol. 17, 1425\u20131432 (2010).","journal-title":"Acad. Radiol."},{"key":"10292_CR17","doi-asserted-by":"publisher","first-page":"771","DOI":"10.1007\/s00414-016-1471-8","volume":"131","author":"HH Thodberg","year":"2017","unstructured":"Thodberg, H. H., van Rijn, R. R., Jenni, O. G. & Martin, D. D. Automated determination of bone age from hand X-rays at the end of puberty and its applicability for age estimation. Int. J. Legal Med. 131, 771\u2013780 (2017).","journal-title":"Int. J. Legal Med."},{"key":"10292_CR18","unstructured":"Roy-Perrot, M. -P., Sempe, M. & Fillozat, A. -M. Rapport d\u2019activite terminal de l\u2019equipe francaise. Supplement to: Compte rendu de la 13\u2019ieme Reunion des Equipes Charges des Etudes sur la Croissance et le d\u00e9veloppement de l\u2019enfant normal. (Centre International de l\u2019Enfance, Paris, 1976)."},{"key":"10292_CR19","doi-asserted-by":"publisher","first-page":"692","DOI":"10.1148\/radiol.2021204021","volume":"301","author":"DK Eng","year":"2021","unstructured":"Eng, D. K. et al. artificial intelligence algorithm improves radiologist performance in skeletal age assessment: a prospective multicenter randomized controlled trial. Radiology 301, 692\u2013699 (2021).","journal-title":"Radiology"},{"key":"10292_CR20","unstructured":"Regulation (EU) 2017\/745 of the European Parliament and of the council on medical devices. https:\/\/eur-lex.europa.eu\/eli\/reg\/2017\/745\/oj."},{"key":"10292_CR21","doi-asserted-by":"publisher","DOI":"10.1007\/S00256-021-03880-Y","author":"PH Yi","year":"2021","unstructured":"Yi, P. H. et al. Can AI distinguish a bone radiograph from photos of flowers or cars? Evaluation of bone age deep learning model on inappropriate data inputs. Skeletal Radiol. https:\/\/doi.org\/10.1007\/S00256-021-03880-Y (2021).","journal-title":"Skeletal Radiol."},{"key":"10292_CR22","doi-asserted-by":"publisher","first-page":"1374","DOI":"10.2214\/AJR.17.18224","volume":"209","author":"JR Kim","year":"2017","unstructured":"Kim, J. R. et al. Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. Am. J. Roentgenol. 209, 1374\u20131380 (2017).","journal-title":"Am. J. Roentgenol."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-10292-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-10292-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-10292-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T02:19:17Z","timestamp":1669342757000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-10292-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,16]]},"references-count":22,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["10292"],"URL":"https:\/\/doi.org\/10.1038\/s41598-022-10292-y","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,16]]},"assertion":[{"value":"16 January 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 March 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"H.H.T. is the owner of Visiana, which develops and markets the BoneXpert medical device for bone age assessment. The other authors declare no relationships with any companies, whose products or services may be related to the subject matter of the article.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"6388"}}