{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T20:56:05Z","timestamp":1790974565460,"version":"4.1.0"},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,11,15]],"date-time":"2022-11-15T00:00:00Z","timestamp":1668470400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,15]],"date-time":"2022-11-15T00:00:00Z","timestamp":1668470400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Objectives<\/jats:title>\n                <jats:p>To assess how an artificial intelligence (AI) algorithm performs against five experienced musculoskeletal radiologists in diagnosing scaphoid fractures and whether it aids their diagnosis on conventional multi-view radiographs.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>Four datasets of conventional hand, wrist, and scaphoid radiographs were retrospectively acquired at two hospitals (hospitals A and B). Dataset 1 (12,990 radiographs from 3353 patients, hospital A) and dataset 2 (1117 radiographs from 394 patients, hospital B) were used for training and testing a scaphoid localization and laterality classification component. Dataset 3 (4316 radiographs from 840 patients, hospital A) and dataset 4 (688 radiographs from 209 patients, hospital B) were used for training and testing the fracture detector. The algorithm was compared with the radiologists in an observer study. Evaluation metrics included sensitivity, specificity, positive predictive value (PPV), area under the characteristic operating curve (AUC), Cohen\u2019s kappa coefficient (\u03ba), fracture localization precision, and reading time.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The algorithm detected scaphoid fractures with a sensitivity of 72%, specificity of 93%, PPV of 81%, and AUC of 0.88. The AUC of the algorithm did not differ from each radiologist (0.87 [radiologists\u2019 mean], <jats:italic>p<\/jats:italic> \u2265 .05). AI assistance improved five out of ten pairs of inter-observer Cohen\u2019s \u03ba agreements (<jats:italic>p<\/jats:italic> &lt; .05) and reduced reading time in four radiologists (<jats:italic>p<\/jats:italic> &lt; .001), but did not improve other metrics in the majority of radiologists (<jats:italic>p<\/jats:italic> \u2265 .05).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The AI algorithm detects scaphoid fractures on conventional multi-view radiographs at the level of five experienced musculoskeletal radiologists and could significantly shorten their reading time.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Key Points<\/jats:title>\n                <jats:p><jats:italic>\u2022 An artificial intelligence algorithm automatically detects scaphoid fractures on conventional multi-view radiographs at the same level of five experienced musculoskeletal radiologists.<\/jats:italic><\/jats:p>\n                <jats:p><jats:italic>\u2022 There is preliminary evidence that automated scaphoid fracture detection can significantly shorten the reading time of musculoskeletal radiologists.<\/jats:italic><\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s00330-022-09205-4","type":"journal-article","created":{"date-parts":[[2022,11,15]],"date-time":"2022-11-15T18:14:41Z","timestamp":1668536081000},"page":"1575-1588","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":58,"title":["Musculoskeletal radiologist-level performance by using deep learning for detection of scaphoid fractures on conventional multi-view radiographs of hand and wrist"],"prefix":"10.1007","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3332-6245","authenticated-orcid":false,"given":"Nils","family":"Hendrix","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ward","family":"Hendrix","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kees","family":"van Dijke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bas","family":"Maresch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mario","family":"Maas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stijn","family":"Bollen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Scholtens","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Milko","family":"de Jonge","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lee-Ling Sharon","family":"Ong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bram","family":"van Ginneken","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthieu","family":"Rutten","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,11,15]]},"reference":[{"key":"9205_CR1","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/1865-1380-4-4","volume":"4","author":"SJ Rhemrev","year":"2011","unstructured":"Rhemrev SJ, Ootes D, Beeres FJP, Meylaerts SAG, Schipper IB (2011) Current methods of diagnosis and treatment of scaphoid fractures. Int J Emerg Med 4:4","journal-title":"Int J Emerg Med"},{"issue":"6","key":"9205_CR2","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1007\/s00068-015-0594-9","volume":"42","author":"AD de Zwart","year":"2016","unstructured":"de Zwart AD, Beeres FJP, Rhemrev SJ, Bartlema K, Schipper IB (2016) Comparison of MRI, CT and bone scintigraphy for suspected scaphoid fractures. Eur J Trauma Emerg Surg 42(6):725\u2013731","journal-title":"Eur J Trauma Emerg Surg"},{"issue":"1","key":"9205_CR3","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1302\/0301-620X.75B1.8421037","volume":"75","author":"MM Tiel-van Buul","year":"1993","unstructured":"Tiel-van Buul MM, van Beek EJ, Broekhuizen AH, Bakker AJ, Bos KE, van Royen EA (1993) Radiography and scintigraphy of suspected scaphoid fracture. A long-term study in 160 patients. J Bone Joint Surg Br 75(1):61\u201365","journal-title":"J Bone Joint Surg Br"},{"issue":"5","key":"9205_CR4","doi-asserted-by":"publisher","first-page":"1117","DOI":"10.2214\/AJR.19.21478","volume":"213","author":"B Gibney","year":"2019","unstructured":"Gibney B, Smith M, Moughty A, Kavanagh EC, Hynes D, MacMahon PJ (2019) Incorporating cone-beam CT into the diagnostic algorithm for suspected radiocarpal fractures: a new standard of care? AJR Am J Roentgenol 213(5):1117\u20131123","journal-title":"AJR Am J Roentgenol"},{"issue":"1","key":"9205_CR5","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1016\/j.jhsa.2015.10.013","volume":"41","author":"R Grewal","year":"2016","unstructured":"Grewal R, Lutz K, MacDermid JC, Suh N (2016) Proximal pole scaphoid fractures: a computed tomographic assessment of outcomes. J Hand Surg Am 41(1):54\u201358","journal-title":"J Hand Surg Am"},{"issue":"2","key":"9205_CR6","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1302\/2058-5241.5.190025","volume":"5","author":"M Clementson","year":"2020","unstructured":"Clementson M, Bj\u00f6rkman A, Thomsen NOB (2020) Acute scaphoid fractures: guidelines for diagnosis and treatment. EFORT Open Rev 5(2):96\u2013103","journal-title":"EFORT Open Rev"},{"issue":"3","key":"9205_CR7","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1177\/0036933013496950","volume":"58","author":"MJ Burns","year":"2013","unstructured":"Burns MJ, Aitken SA, McRae D, Duckworth AD, Gray A (2013) The suspected scaphoid injury: resource implications in the absence of magnetic resonance imaging. Scott Med J 58(3):143\u2013148","journal-title":"Scott Med J"},{"key":"9205_CR8","doi-asserted-by":"publisher","first-page":"741","DOI":"10.1016\/S0221-0363(07)91342-6","volume":"88","author":"A Blum","year":"2007","unstructured":"Blum A, Sauer B, Detreille R et al (2007) The diagnosis of recent scaphoid fractures: review of the literature. J Radiol 88:741\u2013759","journal-title":"J Radiol"},{"issue":"22","key":"9205_CR9","doi-asserted-by":"publisher","first-page":"1860","DOI":"10.2106\/JBJS.O.00099","volume":"97","author":"JW Karl","year":"2015","unstructured":"Karl JW, Swart E, Strauch RJ (2015) Diagnosis of occult scaphoid fractures: a cost-effectiveness analysis. J Bone Joint Surg Am 97(22):1860\u20131868","journal-title":"J Bone Joint Surg Am"},{"issue":"8","key":"9205_CR10","doi-asserted-by":"publisher","first-page":"e245","DOI":"10.1097\/BOT.0000000000000316","volume":"29","author":"ZG Yin","year":"2015","unstructured":"Yin ZG, Zhang JB, Gong KT (2015) Cost-effectiveness of diagnostic strategies for suspected scaphoid fractures. J Orthop Trauma 29(8):e245\u2013e252","journal-title":"J Orthop Trauma"},{"key":"9205_CR11","doi-asserted-by":"publisher","unstructured":"Kuo RYL, Harrison C, Curran TA et al (2022) Artificial intelligence in fracture detection: a systematic review and meta-analysis. Radiology. https:\/\/doi.org\/10.1148\/radiol.211785","DOI":"10.1148\/radiol.211785"},{"issue":"4","key":"9205_CR12","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.2021200260","volume":"3","author":"N Hendrix","year":"2021","unstructured":"Hendrix N, Scholten E, Vernhout B et al (2021) Development and validation of a convolutional neural network for automated detection of scaphoid fractures on conventional radiographs. Radiol Artif Intell 3(4):e200260","journal-title":"Radiol Artif Intell"},{"issue":"5","key":"9205_CR13","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.6096","volume":"4","author":"AP Yoon","year":"2021","unstructured":"Yoon AP, Lee YL, Kane RL, Kuo CF, Lin C, Chung KC (2021) Development and validation of a deep learning model using convolutional neural networks to identify scaphoid fractures in radiographs. JAMA Netw Open 4(5):e216096","journal-title":"JAMA Netw Open"},{"issue":"1","key":"9205_CR14","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1148\/radiol.2021203886","volume":"300","author":"L Duron","year":"2021","unstructured":"Duron L, Ducarouge A, Gillibert A et al (2021) Assessment of an AI aid in detection of adult appendicular skeletal fractures by emergency physicians and radiologists: a multicenter cross-sectional diagnostic study. Radiology 300(1):120\u2013129","journal-title":"Radiology"},{"issue":"3","key":"9205_CR15","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1148\/radiol.210937","volume":"302","author":"A Guermazi","year":"2022","unstructured":"Guermazi A, Tannoury C, Kompel AJ et al (2022) Improving radiographic fracture recognition performance and efficiency using artificial intelligence. Radiology 302(3):627\u2013636","journal-title":"Radiology"},{"issue":"4","key":"9205_CR16","doi-asserted-by":"publisher","first-page":"895","DOI":"10.3390\/diagnostics12040895","volume":"12","author":"TH Yang","year":"2022","unstructured":"Yang TH, Horng MH, Li RS, Sun YN (2022) Scaphoid fracture detection by using convolutional neural network. Diagnostics (Basel) 12(4):895","journal-title":"Diagnostics (Basel)"},{"issue":"18","key":"9205_CR17","doi-asserted-by":"publisher","first-page":"8485","DOI":"10.3390\/app11188485","volume":"11","author":"YC Tung","year":"2021","unstructured":"Tung YC, Su JH, Liao YW et al (2021) High-performance scaphoid fracture recognition via effectiveness assessment of artificial neural networks. Appl Sci 11(18):8485","journal-title":"Appl Sci"},{"issue":"3","key":"9205_CR18","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/j.hcl.2011.06.003","volume":"27","author":"WYC Wong","year":"2011","unstructured":"Wong WYC, Ho PC (2011) Minimal invasive management of scaphoid fractures: from fresh to nonunion. Hand Clin 27(3):291\u2013307","journal-title":"Hand Clin"},{"key":"9205_CR19","first-page":"8026","volume":"32","author":"A Paszke","year":"2019","unstructured":"Paszke A, Gross S, Massa F et al (2019) PyTorch: an imperative style, high-performance deep learning library. Adv Neural Inf Proces Syst 32:8026\u20138037","journal-title":"Adv Neural Inf Proces Syst"},{"key":"9205_CR20","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine Learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"issue":"24","key":"9205_CR21","doi-asserted-by":"publisher","first-page":"638","DOI":"10.21105\/joss.00638","volume":"3","author":"S Raschka","year":"2018","unstructured":"Raschka S (2018) MLxtend: providing machine learning and data science utilities and extensions to Python\u2019s scientific computing stack. J Open Source Softw 3(24):638","journal-title":"J Open Source Softw"},{"issue":"1","key":"9205_CR22","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1016\/J.JHSB.2005.09.001","volume":"31","author":"GC Cheung","year":"2006","unstructured":"Cheung GC, Lever CJ, Morris AD (2006) X-ray diagnosis of acute scaphoid fractures. J Hand Surg Br 31(1):104\u2013109","journal-title":"J Hand Surg Br"},{"issue":"11","key":"9205_CR23","doi-asserted-by":"publisher","first-page":"2653","DOI":"10.1097\/CORR.0000000000001318","volume":"478","author":"DWG Langerhuizen","year":"2020","unstructured":"Langerhuizen DWG, Bulstra AEJ, Janssen SJ et al (2020) Is deep learning on par with human observers for detection of radiographically visible and occult fractures of the scaphoid? Clin Orthop Relat Res 478(11):2653\u20132659","journal-title":"Clin Orthop Relat Res"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-022-09205-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-022-09205-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-022-09205-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,16]],"date-time":"2023-02-16T18:10:39Z","timestamp":1676571039000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-022-09205-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,15]]},"references-count":23,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["9205"],"URL":"https:\/\/doi.org\/10.1007\/s00330-022-09205-4","relation":{},"ISSN":["1432-1084"],"issn-type":[{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,15]]},"assertion":[{"value":"25 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 September 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 September 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 November 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The scientific guarantor of this publication is Nils Hendrix.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"One of the authors has significant statistical expertise.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"Written informed consent was waived by the Institutional Review Board.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Institutional Review Board approval was obtained.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Some study subjects or cohorts have been previously reported in the following publication: <i>Hendrix N, Scholten E, Vernhout B, et al (2021) Development and Validation of a Convolutional Neural Network for Automated Detection of Scaphoid Fractures on Conventional Radiographs. Radiol Artif Intell 3(4):e200260.<\/i>","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Study subjects or cohorts overlap"}},{"value":"retrospective experimental multicenter study","order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}