{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T09:58:55Z","timestamp":1790330335916,"version":"4.1.0"},"reference-count":18,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"3","license":[{"start":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T00:00:00Z","timestamp":1782864000000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background and Purpose:<\/jats:title>\n                    <jats:p>Foundation models including the Segment Anything Model 2 have generated considerable interest as annotation-efficient tools for musculoskeletal (MSK) radiographic image segmentation. Rigorous evaluation requires open-access datasets with well-characterized, high-quality segmentation annotations. No systematic analysis of whether currently available open-access MSK radiograph datasets meet this requirement has been published. This study evaluates the annotation completeness, image quality, and benchmark readiness of three widely cited open-access MSK imaging resources used for radiographic benchmarking: FracAtlas and Musculoskeletal Radiographs (MURA), two native projectional radiograph datasets, and computed tomography (CT)-derived digitally reconstructed radiographs (DRRs) generated from TotalSegmentator, an open-access CT segmentation dataset.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods:<\/jats:title>\n                    <jats:p>A structured, reproducible audit framework was applied to FracAtlas (4083 images), a 500-study stratified random sample from MURA (seven body regions), and 120 DRRs generated from TotalSegmentator CT volumes. Annotation completeness was assessed across four tiers. Mask quality in FracAtlas was graded independently by two raters: two orthopedic surgical trainees and a radiologist, across four levels, with inter-rater agreement quantified by Cohen\u2019s weighted kappa. Image quality was characterized using five no-reference metrics. Domain gap was quantified using structural similarity (SSIM) and Fr\u00e9chet Inception Distance (FID). A five-criterion, equally weighted Benchmark Readiness Score (BRS) framework was developed, applied to all three datasets, and evaluated through sensitivity analysis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>MURA contained no pixel-level segmentation annotations, rendering it unsuitable for direct segmentation benchmarking. FracAtlas provided whole-bone-quality (Grade A) masks for only 175 of 4,083 images (4.3%); 61.4% of masks were fracture-region-only. Inter-rater agreement for mask grading was substantial (Cohen\u2019s \u03baw = 0.76; 95% confidence interval: 0.71\u20130.81). TotalSegmentator-digitally reconstructed radiograph (DRRs) provided complete synthetic bone masks but demonstrated a substantial domain gap from real radiographs (mean SSIM 0.41 \u00b1 0.09; FID 87.3). BRS scores were: MURA 4\/25, FracAtlas 8\/25, FracAtlas-Complete subset 12\/25, and TotalSegmentator-DRR 16\/25. Dataset ranking was stable across sensitivity analysis thresholds of 18\/25, 20\/25, and 22\/25; no dataset reached publication-readiness under any threshold.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>\n                      No currently available open-access MSK radiograph dataset fully meets the requirements for rigorous foundation model segmentation benchmarking. The FracAtlas-Complete subset (\n                      <jats:italic toggle=\"yes\">n<\/jats:italic>\n                      = 175) represents the highest-quality immediately available resource for whole-bone segmentation. The BRS framework provides a reproducible, transparent instrument for dataset quality evaluation; the proposed readiness threshold should be regarded as a heuristic reference requiring future community validation. These findings establish the evidentiary basis for targeted dataset curation and define minimum quality requirements for MSK radiographic artificial intelligence benchmarking.\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.4103\/jmp.jmp_146_26","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T13:01:10Z","timestamp":1790168470000},"page":"662-671","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":0,"title":["Benchmark Readiness of Open-access Musculoskeletal Radiograph Datasets for Foundation Model Segmentation: A Systematic Audit"],"prefix":"10.4103","volume":"51","author":[{"given":"Sri Nikhil","family":"Zallipalli","sequence":"first","affiliation":[{"name":"Institute of Orthopaedics and Musculoskeletal Science, University College London, Royal National Orthopaedic Hospital NHS Trust, London, United Kingdom"},{"name":"Research and Development Division, Maram Biodesign Lab, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaushik Rao","family":"Juvvadi","sequence":"additional","affiliation":[{"name":"Department of Radiodiagnosis, Mediciti Institute of Medical Sciences, Hyderabad, Telangana, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gowthamini","family":"Asokan","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Badr Al Samaa Medical Centre, Dubai, United Arab Emirates"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saichand","family":"Linga","sequence":"additional","affiliation":[{"name":"Department of Trauma and Orthopaedics, South Tyneside District Hospital, South Tyneside and Sunderland NHS Foundation Trust, South Shields, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikitha Devi","family":"Thotapalli","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, SRM Institute of Science and Technology, Chennai, Tamil Nadu, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Preethii","family":"Asokan","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Mahatma Gandhi Medical College and Research Institute, Puducherry, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"key":"R1-20260925","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1016\/S0140-6736(20)30925-9","article-title":"Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: A systematic analysis for the Global Burden of Disease Study 2019","volume":"396","year":"2020","journal-title":"Lancet"},{"key":"R2-20260925","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/s41747-024-00422-8","article-title":"AI applications in musculoskeletal imaging: A narrative review","volume":"8","author":"Gitto","year":"2024","journal-title":"Eur Radiol Exp"},{"key":"R3-20260925","first-page":"4015","article-title":"Segment anything","author":"Kirillov","year":"2023","journal-title":"Proc IEEE\/CVF Int Conf Comput Vis (ICCV)"},{"key":"R5-20260925","doi-asserted-by":"crossref","first-page":"102918","DOI":"10.1016\/j.media.2023.102918","article-title":"Segment anything model for medical image analysis: An experimental study","volume":"89","author":"Mazurowski","year":"2023","journal-title":"Med Image Anal"},{"key":"R6-20260925","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1038\/s41597-023-02432-4","article-title":"FracAtlas: A dataset for fracture classification, localization and segmentation of musculoskeletal radiographs","volume":"10","author":"Abedeen","year":"2023","journal-title":"Sci Data"},{"key":"R8-20260925","doi-asserted-by":"crossref","first-page":"e230024","DOI":"10.1148\/ryai.230024","article-title":"TotalSegmentator: Robust segmentation of 104 anatomic structures in CT images","volume":"5","author":"Wasserthal","year":"2023","journal-title":"Radiol Artif Intell"},{"key":"R9-20260925","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1038\/s41746-022-00592-y","article-title":"Machine learning for medical imaging: Methodological failures and recommendations for the future","volume":"5","author":"Varoquaux","year":"2022","journal-title":"NPJ Digit Med"},{"key":"R10-20260925","doi-asserted-by":"crossref","first-page":"3032","DOI":"10.1002\/ksa.12702","article-title":"Artificial intelligence-assisted analysis of musculoskeletal imaging-a narrative review of the current state of machine learning models","volume":"33","author":"Oettl","year":"2025","journal-title":"Knee Surg Sports Traumatol Arthrosc"},{"key":"R12-20260925","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"R13-20260925","doi-asserted-by":"crossref","first-page":"103329","DOI":"10.1016\/j.cviu.2021.103329","article-title":"Pros and cons of GAN evaluation measures: New developments","volume":"215","author":"Borji","year":"2022","journal-title":"Comput Vis Image Underst"},{"key":"R14-20260925","doi-asserted-by":"crossref","first-page":"e200029","DOI":"10.1148\/ryai.2020200029","article-title":"Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A guide for authors and reviewers","volume":"2","author":"Mongan","year":"2020","journal-title":"Radiol Artif Intell"},{"key":"R15-20260925","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1038\/s41592-023-02151-z","article-title":"Metrics reloaded: Recommendations for image analysis validation","volume":"21","author":"Maier-Hein","year":"2024","journal-title":"Nat Methods"},{"key":"R16-20260925","doi-asserted-by":"crossref","first-page":"m3164","DOI":"10.1136\/bmj.m3164","article-title":"Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI extension","volume":"370","author":"Liu","year":"2020","journal-title":"BMJ"},{"key":"R17-20260925","doi-asserted-by":"crossref","first-page":"103061","DOI":"10.1016\/j.media.2023.103061","article-title":"Segment anything model for medical images?","volume":"92","author":"Huang","year":"2024","journal-title":"Med Image Anal"},{"key":"R18-20260925","doi-asserted-by":"crossref","first-page":"5915","DOI":"10.1038\/s41467-021-26216-9","article-title":"Annotation-efficient deep learning for automatic medical image segmentation","volume":"12","author":"Wang","year":"2021","journal-title":"Nat Commun"},{"key":"R19-20260925","doi-asserted-by":"crossref","first-page":"10998","DOI":"10.1109\/TPAMI.2024.3452629","article-title":"Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy","volume":"46","author":"Marinov","year":"2024","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"R20-20260925","doi-asserted-by":"crossref","first-page":"h5527","DOI":"10.1136\/bmj.h5527","article-title":"STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies","volume":"351","author":"Bossuyt","year":"2015","journal-title":"BMJ"},{"key":"R21-20260925","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.aca.2012.11.007","article-title":"Sample size planning for classification models","volume":"760","author":"Beleites","year":"2013","journal-title":"Anal Chim Acta"}],"container-title":["Journal of Medical Physics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.4103\/jmp.jmp_146_26","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T09:01:07Z","timestamp":1790326867000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.4103\/jmp.jmp_146_26"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7]]},"references-count":18,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026]]}},"URL":"https:\/\/doi.org\/10.4103\/jmp.jmp_146_26","relation":{},"ISSN":["0971-6203","1998-3913"],"issn-type":[{"value":"0971-6203","type":"print"},{"value":"1998-3913","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7]]},"assertion":[{"value":"2026-06-10","name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-08-20","name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}}]}}