{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T14:21:45Z","timestamp":1788272505802,"version":"build-2803163510"},"reference-count":22,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T00:00:00Z","timestamp":1788220800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T00:00:00Z","timestamp":1788220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Pediatr Radiol"],"DOI":"10.1007\/s00247-026-06758-0","type":"journal-article","created":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T13:59:43Z","timestamp":1788271183000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Quantifying human variability in pediatric bone age assessment: a multi-institutional multi-reader study and benchmark for AI evaluation"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6225-727X","authenticated-orcid":false,"given":"Jin","family":"Long","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David B.","family":"Larson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hans H.","family":"Thodberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter Bavngaard","family":"Thrane","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michelle","family":"Phung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cicero T.","family":"Silva","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Neil U.","family":"Lall","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1729-5071","authenticated-orcid":false,"given":"Alex","family":"Towbin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sanjay P.","family":"Prabhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8972-8051","authenticated-orcid":false,"given":"Curtis P.","family":"Langlotz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergios","family":"Gatidis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,1]]},"reference":[{"key":"6758_CR1","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1148\/radiol.2018180736","volume":"290","author":"SS Halabi","year":"2019","unstructured":"Halabi SS, Prevedello LM, Kalpathy-Cramer J et al (2019) The RSNA pediatric bone age machine learning challenge. Radiology 290:498\u2013503","journal-title":"Radiology"},{"key":"6758_CR2","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2017-1486","volume":"140","author":"AL Creo","year":"2017","unstructured":"Creo AL, Schwenk WF 2nd (2017) Bone age: a handy tool for pediatric providers. Pediatrics 140:e20171486","journal-title":"Pediatrics"},{"key":"6758_CR3","doi-asserted-by":"publisher","DOI":"10.3389\/fped.2021.580314","volume":"9","author":"F Cavallo","year":"2021","unstructured":"Cavallo F, Mohn A, Chiarelli F, Giannini C (2021) Evaluation of bone age in children: a mini-review. Front Pediatr 9:580314","journal-title":"Front Pediatr"},{"key":"6758_CR4","doi-asserted-by":"publisher","DOI":"10.1097\/00000441-195909000-00030","volume-title":"Radiographic atlas of skeletal development of the hand and wrist","author":"WW Greulich","year":"1959","unstructured":"Greulich WW, Pyle SI (1959) Radiographic atlas of skeletal development of the hand and wrist. Stanford University Press, Stanford"},{"key":"6758_CR5","doi-asserted-by":"publisher","first-page":"624","DOI":"10.1203\/00006450-200111000-00015","volume":"50","author":"S Mora","year":"2001","unstructured":"Mora S, Boechat MI, Pietka E et al (2001) Skeletal age determinations in children of European and African descent: applicability of the Greulich and Pyle standards. Pediatr Res 50:624\u2013628","journal-title":"Pediatr Res"},{"key":"6758_CR6","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1109\/TMI.2008.926067","volume":"28","author":"HH Thodberg","year":"2009","unstructured":"Thodberg HH, Kreiborg S, Juul A, Pedersen KD (2009) The BoneXpert method for automated determination of skeletal maturity. IEEE Trans Med Imaging 28:52\u201366","journal-title":"IEEE Trans Med Imaging"},{"key":"6758_CR7","doi-asserted-by":"publisher","first-page":"1395","DOI":"10.2214\/ajr.167.6.8956565","volume":"167","author":"FK Ontell","year":"1996","unstructured":"Ontell FK, Ivanovic M, Ablin DS, Barlow TW (1996) Bone age in children of diverse ethnicity. AJR Am J Roentgenol 167:1395\u20131398","journal-title":"AJR Am J Roentgenol"},{"key":"6758_CR8","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1016\/j.ajodo.2012.08.027","volume":"143","author":"S Suri","year":"2013","unstructured":"Suri S, Prasad C, Tompson B, Lou W (2013) Longitudinal comparison of skeletal age determined by the Greulich and Pyle method and chronologic age in normally growing children, and clinical interpretations for orthodontics. Am J Orthod Dentofacial Orthop 143:50\u201360","journal-title":"Am J Orthod Dentofacial Orthop"},{"key":"6758_CR9","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.4158\/EP-2020-0047","volume":"26","author":"M Kl\u00fcnder-Kl\u00fcnder","year":"2020","unstructured":"Kl\u00fcnder-Kl\u00fcnder M, Espinosa-Espindola M, Lopez-Gonzalez D et al (2020) Skeletal maturation in the current pediatric Mexican population. Endocr Pract 26:1053\u20131061","journal-title":"Endocr Pract"},{"key":"6758_CR10","first-page":"630","volume":"31","author":"E \u00d6zmen","year":"2025","unstructured":"\u00d6zmen E, \u00d6zen Atalay H, Uzer E, Veznikli M (2025) A comparison of two artificial intelligence-based methods for assessing bone age in Turkish children: BoneXpert and VUNO Med-Bone Age. Diagn Interv Radiol 31:630\u2013635","journal-title":"Diagn Interv Radiol"},{"key":"6758_CR11","doi-asserted-by":"publisher","first-page":"692","DOI":"10.1148\/radiol.2021204021","volume":"301","author":"DK Eng","year":"2021","unstructured":"Eng DK, Khandwala NB, Long J et al (2021) Artificial intelligence algorithm improves radiologist performance in skeletal age assessment: a prospective multicenter randomized controlled trial. Radiology 301:692\u2013699","journal-title":"Radiology"},{"key":"6758_CR12","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1136\/adc.81.2.172","volume":"81","author":"RK Bull","year":"1999","unstructured":"Bull RK, Edwards PD, Kemp PM et al (1999) Bone age assessment: a large scale comparison of the Greulich and Pyle, and Tanner and Whitehouse (TW2) methods. Arch Dis Child 81:172\u2013173","journal-title":"Arch Dis Child"},{"key":"6758_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23762-1","volume-title":"Hand bone age: a digital atlas of skeletal maturity","author":"V Gilsanz","year":"2012","unstructured":"Gilsanz V, Ratib O (2012) Hand bone age: a digital atlas of skeletal maturity. Springer, Berlin Heidelberg"},{"key":"6758_CR14","doi-asserted-by":"publisher","first-page":"1374","DOI":"10.2214\/AJR.17.18224","volume":"209","author":"JR Kim","year":"2017","unstructured":"Kim JR, Shim WH, Yoon HM et al (2017) Computerized bone age estimation using deep learning based program: evaluation of the accuracy and efficiency. AJR Am J Roentgenol 209:1374\u20131380","journal-title":"AJR Am J Roentgenol"},{"key":"6758_CR15","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-10292-y","volume":"12","author":"DD Martin","year":"2022","unstructured":"Martin DD, Calder AD, Ranke MB et al (2022) Accuracy and self-validation of automated bone age determination. Sci Rep 12:6388","journal-title":"Sci Rep"},{"key":"6758_CR16","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1148\/radiol.2017170236","volume":"287","author":"DB Larson","year":"2018","unstructured":"Larson DB, Chen MC, Lungren MP et al (2018) Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology 287:313\u2013322","journal-title":"Radiology"},{"key":"6758_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-05282-z","volume":"12","author":"WWI Lea","year":"2022","unstructured":"Lea WWI, Hong SJ, Nam HK et al (2022) External validation of deep learning-based bone-age software: a preliminary study with real world data. Sci Rep 12:1232","journal-title":"Sci Rep"},{"key":"6758_CR18","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1007\/s00247-023-05789-1","volume":"54","author":"S Rassmann","year":"2024","unstructured":"Rassmann S, Keller A, Skaf K et al (2024) Deeplasia: deep learning for bone age assessment validated on skeletal dysplasias. Pediatr Radiol 54:82\u201395","journal-title":"Pediatr Radiol"},{"key":"6758_CR19","doi-asserted-by":"publisher","first-page":"242.e1","DOI":"10.1016\/j.forsciint.2008.05.005","volume":"179","author":"N Lynnerup","year":"2008","unstructured":"Lynnerup N, Belard E, Buch-Olsen K et al (2008) Intra- and interobserver error of the Greulich-Pyle method as used on a Danish forensic sample. Forensic Sci Int 179:242.e1-242.e6","journal-title":"Forensic Sci Int"},{"key":"6758_CR20","doi-asserted-by":"publisher","first-page":"222","DOI":"10.2340\/17453674.2021.1133","volume":"93","author":"AB Breen","year":"2022","unstructured":"Breen AB, Steen H, Pripp A et al (2022) A comparison of 3 different methods for assessment of skeletal age when treating leg-length discrepancies: an inter- and intra-observer study. Acta Orthop 93:222\u2013228","journal-title":"Acta Orthop"},{"key":"6758_CR21","doi-asserted-by":"publisher","first-page":"2188","DOI":"10.1007\/s00247-022-05516-2","volume":"52","author":"C Oza","year":"2022","unstructured":"Oza C, Khadilkar AV, Mondkar S et al (2022) A comparison of bone age assessments using automated and manual methods in children of Indian ethnicity. Pediatr Radiol 52:2188\u20132196","journal-title":"Pediatr Radiol"},{"key":"6758_CR22","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0271386","volume":"17","author":"P Sousa-E-Silva","year":"2022","unstructured":"Sousa-E-Silva P, Coelho-E-Silva MJ, Seabra A et al (2022) Skeletal age assessed by TW2 using 20-bone, carpal and RUS score systems: intra-observer and inter-observer agreement among male pubertal soccer players. PLoS ONE 17:e0271386","journal-title":"PLoS ONE"}],"container-title":["Pediatric Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00247-026-06758-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00247-026-06758-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00247-026-06758-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T13:59:44Z","timestamp":1788271184000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00247-026-06758-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,1]]},"references-count":22,"alternative-id":["6758"],"URL":"https:\/\/doi.org\/10.1007\/s00247-026-06758-0","relation":{},"ISSN":["1432-1998"],"issn-type":[{"value":"1432-1998","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,1]]},"assertion":[{"value":"14 March 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 August 2026","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 August 2026","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 September 2026","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"This study is a retrospective secondary analysis of de-identified clinical and imaging data collected as part of the prospective multicenter randomized controlled trial, Artificial Intelligence Algorithm Improves Radiologist Performance in Skeletal Age Assessment (ClinicalTrials.gov identifier NCT03530098). The original trial was approved by the Institutional Review Boards of all participating institutions, including Stanford University School of Medicine [Stanford University Institutional Review Board], Harvard Medical School\/Boston Children\u2019s Hospital [Boston Children\u2019s Hospital Institutional Review Board], Cincinnati Children\u2019s Hospital Medical Center [Cincinnati Children\u2019s Hospital Medical Center Institutional Review Board], Emory University School of Medicine [Children\u2019s Healthcare of Atlanta Institutional Review Board], and Yale University School of Medicine [Yale University Institutional Review Board]. The present secondary analysis used data collected under these approvals. The study was conducted in accordance with the ethical principles of the Declaration of Helsinki and applicable institutional regulations. Verbal informed consent was obtained from participating radiologists. The requirement for informed consent for pediatric participants was waived by the Institutional Review Boards at all participating institutions because the study used clinical radiographs collected as part of routine care, in accordance with the approved study protocol.","order":1,"name":"Ethics","label":"Ethics approval and consent to participate","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"J.L., S.G., D.B.L., C.P.L., M.P., C.T.S., N.U.L., A.T., and S.P. declare that they have no competing financial interests related to this work. H.H.T. is the Chief Executive Officer of Visiana, the developer of the BoneXpert software evaluated in this study. P.B.T. is an employee of Visiana.\u00a0This study received funding from Visiana. The company provided access to the BoneXpert software and technical support but did not have access to the underlying imaging data or patient-level information and did not participate in the statistical analysis. Data analysis and interpretation were performed independently by the academic investigators.","order":2,"name":"Ethics","label":"Conflicts of interest","group":{"name":"EthicsHeading","label":"Declarations"}}]}}