{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,19]],"date-time":"2026-10-19T16:38:22Z","timestamp":1792427902495,"version":"4.5.0"},"reference-count":36,"publisher":"BMJ","issue":"4","license":[{"start":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T00:00:00Z","timestamp":1700697600000},"content-version":"unspecified","delay-in-days":22,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2021M3H9A2097957, 2021R1F1A1062148, and 2022R1F1A1"],"award-info":[{"award-number":["2021M3H9A2097957, 2021R1F1A1062148, and 2022R1F1A1"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["RMD Open"],"accepted":{"date-parts":[[2023,11,8]]},"published-print":{"date-parts":[[2023,11]]},"abstract":"<jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>Summarise the evidence of the performance of the machine learning algorithm in discriminating sacroiliitis features on MRI and compare it with the accuracy of human physicians.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>MEDLINE, EMBASE, CIHNAL, Web of Science, IEEE, American College of Rheumatology and European Alliance of Associations for Rheumatology abstract archives were searched for studies published between 2008 and 4 June 2023. Two authors independently screened and extracted the variables, and the results are presented using tables and forest plots.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Ten studies were selected from 2381. Over half of the studies used deep learning models, using Assessment of Spondyloarthritis International Society sacroiliitis criteria as the ground truth, and manually extracted the regions of interest. All studies reported the area under the curve as a performance index, ranging from 0.76 to 0.99. Sensitivity and specificity were the second-most commonly reported indices, with sensitivity ranging from 0.56 to 1.00 and specificity ranging from 0.67 to 1.00; these results are comparable to a radiologist\u2019s sensitivity of 0.67\u20131.00 and specificity of 0.78\u20131.00 in the same cohort. More than half of the studies showed a high risk of bias in the analysis domain of quality appraisal owing to the small sample size or overfitting issues.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>The performance of machine learning algorithms in discriminating sacroiliitis features on MRI varied owing to the high heterogeneity between studies and the small sample sizes, overfitting, and under-reporting issues of individual studies. Further well-designed and transparent studies are required.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1136\/rmdopen-2023-003783","type":"journal-article","created":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T12:40:28Z","timestamp":1700743228000},"page":"e003783","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":16,"title":["Performances of machine learning algorithms in discriminating sacroiliitis features on MRI: a systematic review"],"prefix":"10.1136","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3786-6935","authenticated-orcid":false,"given":"Sun Jae","family":"Moon","sequence":"first","affiliation":[{"name":"Department of Medicine, Santa Marie 24 Clinic, Seongnam-si, Korea (the Republic of)"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5551-4178","authenticated-orcid":false,"given":"Seulkee","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea (the Republic of)"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinseub","family":"Hwang","sequence":"additional","affiliation":[{"name":"Department of Data Science, Daegu University, Gyeongsan-si, Korea (the Republic of)"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jaejoon","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea (the Republic of)"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seonyoung","family":"Kang","sequence":"additional","affiliation":[{"name":"Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea (the Republic of)"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hoon-Suk","family":"Cha","sequence":"additional","affiliation":[{"name":"Department of Medicine, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea (the Republic of)"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2023,11,23]]},"reference":[{"key":"2025090313032505000_9.4.e003783.1","doi-asserted-by":"publisher","DOI":"10.1038\/nrrheum.2014.145"},{"key":"2025090313032505000_9.4.e003783.2","doi-asserted-by":"publisher","DOI":"10.1136\/ard.2009.110767"},{"key":"2025090313032505000_9.4.e003783.3","doi-asserted-by":"crossref","first-page":"55","DOI":"10.4078\/jrd.2021.28.2.55","article-title":"Axial Spondyloarthritis: Current advances, future challenges","volume":"28","author":"Inman","year":"2021","journal-title":"J Rheum Dis"},{"key":"2025090313032505000_9.4.e003783.4","doi-asserted-by":"crossref","DOI":"10.1136\/rmdopen-2019-000950","article-title":"Implementation and role of modern musculoskeletal imaging in Rheumatological practice in member countries of EULAR","volume":"5","author":"Mandl","year":"2019","journal-title":"RMD Open"},{"key":"2025090313032505000_9.4.e003783.5","doi-asserted-by":"crossref","first-page":"1955","DOI":"10.1093\/rheumatology\/kez172","article-title":"Performance of magnetic resonance imaging in the diagnosis of axial Spondyloarthritis: a systematic literature review","volume":"58","author":"Jones","year":"2019","journal-title":"Rheumatology (Oxford)"},{"key":"2025090313032505000_9.4.e003783.6","doi-asserted-by":"crossref","DOI":"10.1136\/rmdopen-2021-002128","article-title":"The value of MRI for detecting Subclinical joint inflammation in clinically suspect arthralgia","volume":"8","author":"Boeren","year":"2022","journal-title":"RMD Open"},{"key":"2025090313032505000_9.4.e003783.7","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1007\/s00256-021-03876-8","article-title":"AI musculoskeletal clinical applications: how can AI increase my day-to-day efficiency","volume":"51","author":"Shin","year":"2022","journal-title":"Skeletal Radiol"},{"key":"2025090313032505000_9.4.e003783.8","doi-asserted-by":"crossref","DOI":"10.1038\/s41746-020-0229-3","article-title":"A systematic review of the applications of artificial intelligence and machine learning in autoimmune diseases","volume":"3","author":"Stafford","year":"2020","journal-title":"NPJ Digit Med"},{"key":"2025090313032505000_9.4.e003783.9","doi-asserted-by":"crossref","first-page":"e185","DOI":"10.1002\/mp.13678","article-title":"Machine and deep learning methods for Radiomics","volume":"47","author":"Avanzo","year":"2020","journal-title":"Med Phys"},{"key":"2025090313032505000_9.4.e003783.10","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1038\/s41584-021-00708-w","article-title":"An introduction to machine learning and analysis of its use in rheumatic diseases","volume":"17","author":"Kingsmore","year":"2021","journal-title":"Nat Rev Rheumatol"},{"key":"2025090313032505000_9.4.e003783.11","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2017171920"},{"key":"2025090313032505000_9.4.e003783.12","doi-asserted-by":"publisher","DOI":"10.1002\/art.42296"},{"key":"2025090313032505000_9.4.e003783.13","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.n2281","article-title":"Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review","volume":"375","author":"Andaur Navarro","year":"2021","journal-title":"BMJ"},{"key":"2025090313032505000_9.4.e003783.14","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.jclinepi.2022.11.015","article-title":"Systematic review identifies the design and methodological conduct of studies on machine learning-based prediction models","volume":"154","author":"Andaur Navarro","year":"2023","journal-title":"Journal of Clinical Epidemiology"},{"issue":"jul21 1","key":"2025090313032505000_9.4.e003783.15","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.b2535","article-title":"Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement","volume":"339","author":"Moher","year":"2009","journal-title":"BMJ"},{"key":"2025090313032505000_9.4.e003783.16","doi-asserted-by":"publisher","DOI":"10.1136\/ard.2009.108217"},{"key":"2025090313032505000_9.4.e003783.17","doi-asserted-by":"publisher","DOI":"10.1038\/s41568-018-0016-5"},{"key":"2025090313032505000_9.4.e003783.18","article-title":"Checklist for artificial intelligence in medical imaging (CLAIM): A guide for authors and reviewers","volume":"2","author":"Mongan","year":"2020","journal-title":"Radiology: Artificial Intelligence"},{"key":"2025090313032505000_9.4.e003783.19","doi-asserted-by":"publisher","DOI":"10.7326\/M18-1377"},{"issue":"Suppl 1","key":"2025090313032505000_9.4.e003783.20","doi-asserted-by":"crossref","first-page":"S56","DOI":"10.1007\/s11606-012-2029-1","article-title":"Chapter 8: meta-analysis of test performance when there is a \"gold standard","volume":"27 Suppl 1","author":"Trikalinos","year":"2012","journal-title":"J Gen Intern Med"},{"key":"2025090313032505000_9.4.e003783.21","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/j.diii.2023.03.008","article-title":"A deep learning model for the diagnosis of Sacroiliitis according to assessment of Spondyloarthritis International society classification criteria with magnetic resonance imaging","volume":"104","author":"Bordner","year":"2023","journal-title":"Diagn Interv Imaging"},{"key":"2025090313032505000_9.4.e003783.22","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1093\/rheumatology\/keab542","article-title":"A predictive clinical-Radiomics Nomogram for diagnosing of axial Spondyloarthritis using MRI and clinical risk factors","volume":"61","author":"Ye","year":"2022","journal-title":"Rheumatology (Oxford)"},{"key":"2025090313032505000_9.4.e003783.23","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1007\/s10278-021-00559-7","article-title":"Radiomic Quantification for MRI assessment of Sacroiliac joints of patients with Spondyloarthritis","volume":"35","author":"Ten\u00f3rio","year":"2022","journal-title":"J Digit Imaging"},{"key":"2025090313032505000_9.4.e003783.24","doi-asserted-by":"crossref","unstructured":"Roels J , De Craemer A-S , Renson T , et al . A machine learning pipeline for predicting bone marrow oedema along the Sacroiliac joints on magnetic resonance imaging. Arthritis Rheumatol July 6, 2023. doi:10.1002\/art.42650","DOI":"10.1002\/art.42650"},{"key":"2025090313032505000_9.4.e003783.25","doi-asserted-by":"crossref","first-page":"e318","DOI":"10.1093\/rheumatology\/keac216","article-title":"Deep learning Algorithms for magnetic resonance imaging of inflammatory Sacroiliitis in axial Spondyloarthritis","volume":"61","author":"Lin","year":"2022","journal-title":"Rheumatology (Oxford)"},{"key":"2025090313032505000_9.4.e003783.26","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1148\/radiol.212526","article-title":"Deep learning detects changes indicative of axial Spondyloarthritis at MRI of Sacroiliac joints","volume":"305","author":"Bressem","year":"2022","journal-title":"Radiology"},{"key":"2025090313032505000_9.4.e003783.27","first-page":"316","article-title":"Development of a deep learning algorithm for the detection of Sacroiliitis on MRI in patients with active axial Spondyloarthritis","volume":"73","author":"Nicolaes","year":"2021","journal-title":"Arthritis Rheumatol"},{"key":"2025090313032505000_9.4.e003783.28","doi-asserted-by":"crossref","DOI":"10.3390\/diagnostics11071156","article-title":"Method for diagnosing the bone marrow edema of Sacroiliac joint in patients with axial Spondyloarthritis using magnetic resonance image analysis based on deep learning","volume":"11","author":"Lee","year":"2021","journal-title":"Diagnostics (Basel)"},{"key":"2025090313032505000_9.4.e003783.29","doi-asserted-by":"crossref","first-page":"109755","DOI":"10.1016\/j.ejrad.2021.109755","article-title":"Differentiation of inflammatory from degenerative changes in the Sacroiliac joints by machine learning supported texture analysis","volume":"140","author":"Kepp","year":"2021","journal-title":"Eur J Radiol"},{"key":"2025090313032505000_9.4.e003783.30","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/s42358-020-00126-8","article-title":"Machine learning techniques for computer-aided classification of active inflammatory Sacroiliitis in magnetic resonance imaging","volume":"60","author":"Faleiros","year":"2020","journal-title":"Adv Rheumatol"},{"key":"2025090313032505000_9.4.e003783.31","doi-asserted-by":"publisher","DOI":"10.3348\/kjr.2017.18.4.570"},{"key":"2025090313032505000_9.4.e003783.32","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2020-048008","article-title":"Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and Prognostic prediction model studies based on artificial intelligence","volume":"11","author":"Collins","year":"2021","journal-title":"BMJ Open"},{"key":"2025090313032505000_9.4.e003783.33","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2016161553"},{"key":"2025090313032505000_9.4.e003783.34","doi-asserted-by":"crossref","DOI":"10.1186\/1471-2288-14-137","article-title":"Modern Modelling techniques are data hungry: a simulation study for predicting Dichotomous endpoints","volume":"14","author":"van der Ploeg","year":"2014","journal-title":"BMC Med Res Methodol"},{"key":"2025090313032505000_9.4.e003783.35","doi-asserted-by":"crossref","DOI":"10.1148\/ryai.2021200126","article-title":"Magician\u2019s corner: 9. performance Metrics for machine learning models","volume":"3","author":"Erickson","year":"2021","journal-title":"Radiol Artif Intell"},{"key":"2025090313032505000_9.4.e003783.36","doi-asserted-by":"publisher","DOI":"10.1016\/j.zemedi.2018.11.002"}],"container-title":["RMD Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/rmdopen-2023-003783","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T20:03:37Z","timestamp":1756929817000},"score":1,"resource":{"primary":{"URL":"https:\/\/rmdopen.bmj.com\/lookup\/doi\/10.1136\/rmdopen-2023-003783"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11]]},"references-count":36,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,11,23]]},"published-print":{"date-parts":[[2023,11]]}},"alternative-id":["10.1136\/rmdopen-2023-003783"],"URL":"https:\/\/doi.org\/10.1136\/rmdopen-2023-003783","relation":{},"ISSN":["2056-5933"],"issn-type":[{"value":"2056-5933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11]]}}}