{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T13:11:39Z","timestamp":1792588299575,"version":"4.5.0"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2021,3,19]],"date-time":"2021-03-19T00:00:00Z","timestamp":1616112000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,19]],"date-time":"2021-03-19T00:00:00Z","timestamp":1616112000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005688","name":"Gottfried und Julia Bangerter-Rhyner-Stiftung","doi-asserted-by":"publisher","award":["-"],"award-info":[{"award-number":["-"]}],"id":[{"id":"10.13039\/501100005688","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Objectives<\/jats:title>\n                <jats:p>To evaluate the performance of a deep convolutional neural network (DCNN) in detecting and classifying distal radius fractures, metal, and cast on radiographs using labels based on radiology reports. The secondary aim was to evaluate the effect of the training set size on the algorithm\u2019s performance.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>A total of 15,775 frontal and lateral radiographs, corresponding radiology reports, and a ResNet18 DCNN were used. Fracture detection and classification models were developed per view and merged. Incrementally sized subsets served to evaluate effects of the training set size. Two musculoskeletal radiologists set the standard of reference on radiographs (test set A). A subset (B) was rated by three radiology residents. For a per-study-based comparison with the radiology residents, the results of the best models were merged. Statistics used were ROC and AUC, Youden\u2019s J statistic (J), and Spearman\u2019s correlation coefficient (\u03c1).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The models\u2019 AUC\/J on (A) for metal and cast were 0.99\/0.98 and 1.0\/1.0. The models\u2019 and residents\u2019 AUC\/J on (B) were similar on fracture (0.98\/0.91; 0.98\/0.92) and multiple fragments (0.85\/0.58; 0.91\/0.70). Training set size and AUC correlated on metal (\u03c1 = 0.740), cast (\u03c1 = 0.722), fracture (frontal \u03c1 = 0.947, lateral \u03c1 = 0.946), multiple fragments (frontal \u03c1 = 0.856), and fragment displacement (frontal \u03c1 = 0.595).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The models trained on a DCNN with report-based labels to detect distal radius fractures on radiographs are suitable to aid as a secondary reading tool; models for fracture classification are not ready for clinical use. Bigger training sets lead to better models in all categories except joint affection.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Key Points<\/jats:title>\n                <jats:p><jats:italic>\u2022 Detection of metal and cast on radiographs is excellent using AI and labels extracted from radiology reports.<\/jats:italic><\/jats:p>\n                <jats:p><jats:italic>\u2022 Automatic detection of distal radius fractures on radiographs is feasible and the performance approximates radiology residents.<\/jats:italic><\/jats:p>\n                <jats:p><jats:italic>\u2022 Automatic classification of the type of distal radius fracture varies in accuracy and is inferior for joint involvement and fragment displacement.<\/jats:italic><\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s00330-021-07811-2","type":"journal-article","created":{"date-parts":[[2021,3,19]],"date-time":"2021-03-19T21:02:47Z","timestamp":1616187767000},"page":"6816-6824","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":43,"title":["AI-based detection and classification of distal radius fractures using low-effort data labeling: evaluation of applicability and effect of training set size"],"prefix":"10.1007","volume":"31","author":[{"given":"Patrick","family":"Tobler","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshy","family":"Cyriac","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Balazs K.","family":"Kovacs","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Verena","family":"Hofmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raphael","family":"Sexauer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabiano","family":"Paciolla","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bram","family":"Stieltjes","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Amsler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1560-0247","authenticated-orcid":false,"given":"Anna","family":"Hirschmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,3,19]]},"reference":[{"key":"7811_CR1","doi-asserted-by":"publisher","first-page":"691","DOI":"10.1016\/j.injury.2006.04.130","volume":"37","author":"CM Court-Brown","year":"2006","unstructured":"Court-Brown CM, Caesar B (2006) Epidemiology of adult fractures: a review. Injury 37:691\u2013697","journal-title":"Injury"},{"key":"7811_CR2","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1097\/01.rli.0000233326.94998.b0","volume":"41","author":"S Metz","year":"2006","unstructured":"Metz S, Kuhn V, Kettler M et al (2006) Comparison of different radiography systems in an experimental study for detection of forearm fractures and evaluation of the M\u00fcller-AO and Frykman Classification for distal radius fractures. Invest Radiol 41:681\u2013690","journal-title":"Invest Radiol"},{"key":"7811_CR3","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1055\/s-0039-1684024","volume":"23","author":"A Hirschmann","year":"2019","unstructured":"Hirschmann A, Cyriac J, Stieltjes B, Kober T, Richiardi J, Omoumi P (2019) Artificial intelligence in musculoskeletal imaging: review of current literature, challenges, and trends. Semin Musculoskelet Radiol 23:304\u2013311","journal-title":"Semin Musculoskelet Radiol"},{"key":"7811_CR4","doi-asserted-by":"publisher","first-page":"e180001","DOI":"10.1148\/ryai.2019180001","volume":"1","author":"YL Thian","year":"2019","unstructured":"Thian YL, Li Y, Jagmohan P, Sia D, Chan VEY, Tan RT (2019) Convolutional neural networks for automated fracture detection and localization on wrist radiographs. Radiol Artif Intell 1:e180001","journal-title":"Radiol Artif Intell"},{"key":"7811_CR5","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1080\/17453674.2019.1600125","volume":"90","author":"K Gan","year":"2019","unstructured":"Gan K, Xu D, Lin Y et al (2019) Artificial intelligence detection of distal radius fractures: a comparison between the convolutional neural network and professional assessments. Acta Orthop 90:394\u2013400","journal-title":"Acta Orthop"},{"key":"7811_CR6","doi-asserted-by":"publisher","first-page":"11591","DOI":"10.1073\/pnas.1806905115","volume":"115","author":"R Lindsey","year":"2018","unstructured":"Lindsey R, Daluiski A, Chopra S et al (2018) Deep neural network improves fracture detection by clinicians. Proc Natl Acad Sci U S A 115:11591\u201311596","journal-title":"Proc Natl Acad Sci U S A"},{"key":"7811_CR7","doi-asserted-by":"publisher","first-page":"237.e1","DOI":"10.1016\/j.crad.2019.10.022","volume":"75","author":"JS Yu","year":"2020","unstructured":"Yu JS, Yu SM, Erdal BS et al (2020) Detection and localisation of hip fractures on anteroposterior radiographs with artificial intelligence: proof of concept. Clin Radiol 75:237.e1\u2013237.e9. https:\/\/doi.org\/10.1016\/j.crad.2019.10.022","journal-title":"Clin Radiol"},{"key":"7811_CR8","doi-asserted-by":"publisher","first-page":"5469","DOI":"10.1007\/s00330-019-06167-y","volume":"29","author":"CT Cheng","year":"2019","unstructured":"Cheng CT, Ho TY, Lee TY et al (2019) Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs. Eur Radiol 29:5469\u20135477","journal-title":"Eur Radiol"},{"key":"7811_CR9","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1080\/17453674.2018.1453714","volume":"89","author":"SW Chung","year":"2018","unstructured":"Chung SW, Han SS, Lee JW et al (2018) Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 89:468\u2013473","journal-title":"Acta Orthop"},{"key":"7811_CR10","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1097\/RLI.0000000000000615","volume":"55","author":"JW Choi","year":"2020","unstructured":"Choi JW, Cho YJ, Lee S et al (2020) Using a dual-input convolutional neural network for automated detection of pediatric supracondylar fracture on conventional radiography. Invest Radiol 55:101\u2013110","journal-title":"Invest Radiol"},{"key":"7811_CR11","doi-asserted-by":"publisher","first-page":"672","DOI":"10.1007\/s10278-018-0167-7","volume":"32","author":"G Kitamura","year":"2019","unstructured":"Kitamura G, Chung CY, Moore BE (2019) Ankle fracture detection utilizing a convolutional neural network ensemble implemented with a small sample, de novo training, and multiview incorporation. J Digit Imaging 32:672\u2013677","journal-title":"J Digit Imaging"},{"key":"7811_CR12","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1080\/17453674.2017.1344459","volume":"88","author":"J Olczak","year":"2017","unstructured":"Olczak J, Fahlberg N, Maki A et al (2017) Artificial intelligence for analyzing orthopedic trauma radiographs: deep learning algorithms\u2014are they on par with humans for diagnosing fractures? Acta Orthop 88:581\u2013586","journal-title":"Acta Orthop"},{"key":"7811_CR13","doi-asserted-by":"publisher","first-page":"108925","DOI":"10.1016\/j.ejrad.2020.108925","volume":"126","author":"C Bl\u00fcthgen","year":"2020","unstructured":"Bl\u00fcthgen C, Becker AS, Vittoria de Martini I, Meier A, Martini K, Frauenfelder T (2020) Detection and localization of distal radius fractures: deep learning system versus radiologists. Eur J Radiol 126:108925. https:\/\/doi.org\/10.1016\/j.ejrad.2020.108925","journal-title":"Eur J Radiol"},{"key":"7811_CR14","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1016\/j.crad.2017.11.015","volume":"73","author":"DH Kim","year":"2018","unstructured":"Kim DH, MacKinnon T (2018) Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol 73:439\u2013445","journal-title":"Clin Radiol"},{"key":"7811_CR15","doi-asserted-by":"publisher","first-page":"e190023","DOI":"10.1148\/ryai.2020190023","volume":"2","author":"JD Krogue","year":"2020","unstructured":"Krogue JD, Cheng KV, Hwang KM et al (2020) Automatic hip fracture identification and functional subclassification with deep learning. Radiol Artif Intell 2:e190023. https:\/\/doi.org\/10.1148\/ryai.2020190023","journal-title":"Radiol Artif Intell"},{"key":"7811_CR16","doi-asserted-by":"crossref","unstructured":"Meinberg EG, Agel J, Roberts CS, Karam MD, Kellam JF. (2018) Fracture and Dislocation Classification Compendium-2018","DOI":"10.1097\/BOT.0000000000001063"},{"key":"7811_CR17","doi-asserted-by":"crossref","unstructured":"Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics.","DOI":"10.2307\/2529310"},{"key":"7811_CR18","doi-asserted-by":"publisher","first-page":"E68","DOI":"10.1055\/s-0039-1687898","volume":"23","author":"I Khodarahmi","year":"2019","unstructured":"Khodarahmi I, Isaac A, Fishman EK, Dalili D, Fritz J (2019) Metal about the hip and artifact reduction techniques: from basic concepts to advanced imaging. Semin Musculoskelet Radiol 23:E68\u2013E81","journal-title":"Semin Musculoskelet Radiol"},{"key":"7811_CR19","doi-asserted-by":"publisher","first-page":"670","DOI":"10.1302\/0301-620X.80B4.0800670","volume":"80","author":"T Flikkil\u00e4","year":"1998","unstructured":"Flikkil\u00e4 T, Nikkola-Sihto A, Kaarela O, P\u00e4\u00e4kk\u00f6 E, Raatikainen T (1998) Poor interobserver reliability of AO classification of fractures of the distal radius. Additional computed tomography is of minor value. J Bone Joint Surg Br 80:670\u2013672","journal-title":"J Bone Joint Surg Br"},{"key":"7811_CR20","doi-asserted-by":"publisher","first-page":"046","DOI":"10.1055\/s-0036-1587316","volume":"06","author":"P Jayakumar","year":"2016","unstructured":"Jayakumar P, Teunis T, Gim\u00e9nez B, Verstreken F, Di Mascio L, Jupiter JB (2016) AO distal radius fracture classification: global perspective on observer agreement. J Wrist Surg 06:046\u2013053","journal-title":"J Wrist Surg"},{"key":"7811_CR21","doi-asserted-by":"publisher","first-page":"e190053","DOI":"10.1148\/ryai.2019190053","volume":"1","author":"I Pan","year":"2019","unstructured":"Pan I, Thodberg HH, Halabi SS, Kalpathy-Cramer J, Larson DB (2019) Improving automated pediatric bone age estimation using ensembles of models from the 2017 RSNA Machine Learning Challenge. Radiol Artif Intell 1:e190053","journal-title":"Radiol Artif Intell"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-021-07811-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-021-07811-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-021-07811-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,20]],"date-time":"2021-08-20T14:29:11Z","timestamp":1629469751000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-021-07811-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,19]]},"references-count":21,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["7811"],"URL":"https:\/\/doi.org\/10.1007\/s00330-021-07811-2","relation":{},"ISSN":["0938-7994","1432-1084"],"issn-type":[{"value":"0938-7994","type":"print"},{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,19]]},"assertion":[{"value":"15 December 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 February 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The scientific guarantor of this publication is Anna Hirschmann.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"One of the authors (Felix Amsler) has significant statistical expertise.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"IRB not applicable due to the retrospective fully anonymized and de-identified data.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Institutional Review Board approval was not required because it is a fully anonymized, retrospective study. However, the local ethical committee (EKNZ, Req-2020-00149) has reviewed the submitted documents and can confirm that the research project fulfills the general ethical and scientific Standards for research with humans (see Arf. 51 Abs. 2 HRA).","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"\u2022 retrospective\u2022 diagnostic study\u2022 performed at one institution","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}