{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:53:03Z","timestamp":1792788783112,"version":"4.5.0"},"reference-count":30,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2023,2,13]],"date-time":"2023-02-13T00:00:00Z","timestamp":1676246400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"JSPS Grant-in-Aid for Early-Career Scientists","award":["19K17894"],"award-info":[{"award-number":["19K17894"]}]},{"name":"Fujitsu.Ltd."},{"name":"Fujitsu.Ltd."},{"name":"Fujitsu.Ltd."},{"name":"Fujitsu.Ltd."}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>We propose a wrist joint subluxation\/ankylosis classification model for an automatic radiographic scoring system for X-ray images. In managing rheumatoid arthritis, the evaluation of joint destruction is important. The modified total Sharp score (mTSS), which is conventionally used to evaluate joint destruction of the hands and feet, should ideally be automated because the required time depends on the skill of the evaluator, and there is variability between evaluators. Since joint subluxation and ankylosis are given a large score in mTSS, we aimed to estimate subluxation and ankylosis using a deep neural network as a first step in developing an automatic radiographic scoring system for joint destruction. We randomly extracted 216 hand X-ray images from an electronic medical record system for the learning experiments. These images were acquired from patients who visited the rheumatology department of Keio University Hospital in 2015. Using our newly developed annotation tool, well-trained rheumatologists and radiologists labeled the mTSS to the wrist, metacarpal phalangeal joints, and proximal interphalangeal joints included in the images. We identified 21 X-ray images containing one or more subluxation joints and 42 X-ray images with ankylosis. To predict subluxation\/ankylosis, we conducted five-fold cross-validation with deep neural network models: AlexNet, ResNet, DenseNet, and Vision Transformer. The best performance on wrist subluxation\/ankylosis classification was as follows: accuracy, precision, recall, F1 value, and AUC were 0.97\u00b10.01\/0.89\u00b10.04, 0.92\u00b10.12\/0.77\u00b10.15, 0.77\u00b10.16\/0.71\u00b10.13, 0.82\u00b10.11\/0.72\u00b10.09, and 0.92\u00b10.08\/0.85\u00b10.07, respectively. The classification model based on a deep neural network was trained with a relatively small dataset; however, it showed good accuracy. In conclusion, we provided data collection and model training schemes for mTSS prediction and showed an important contribution to building an automated scoring system.<\/jats:p>","DOI":"10.1371\/journal.pone.0281088","type":"journal-article","created":{"date-parts":[[2023,2,13]],"date-time":"2023-02-13T14:37:01Z","timestamp":1676299021000},"page":"e0281088","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":18,"title":["Detecting hand joint ankylosis and subluxation in radiographic images using deep learning: A step in the development of an automatic radiographic scoring system for joint destruction"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8597-0759","authenticated-orcid":true,"given":"Keisuke","family":"Izumi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7122-7649","authenticated-orcid":true,"given":"Kanata","family":"Suzuki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0162-5312","authenticated-orcid":true,"given":"Masahiro","family":"Hashimoto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Toshio","family":"Endoh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kentaro","family":"Doi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuki","family":"Iwai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8241-6565","authenticated-orcid":true,"given":"Masahiro","family":"Jinzaki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shigeru","family":"Ko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tsutomu","family":"Takeuchi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuko","family":"Kaneko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2023,2,13]]},"reference":[{"issue":"3","key":"pone.0281088.ref001","first-page":"743","article-title":"How to read radiographs according to the Sharp\/van der Heijde method","volume":"26","author":"D van der Heijde","year":"1999","journal-title":"Journal of Rheumatology"},{"key":"pone.0281088.ref002","first-page":"1097","article-title":"ImageNet classification with deep convolutional neural networks","author":"A Krizhevsky","year":"2012","journal-title":"In Advances in neural information processing systems"},{"key":"pone.0281088.ref003","first-page":"10781","article-title":"EfficientDet: Scalable and Efficient Object Detection","author":"M Tan","year":"2020","journal-title":"In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"pone.0281088.ref004","first-page":"833","volume-title":"ECCV 2018","author":"LC Chen","year":"2018"},{"issue":"22","key":"pone.0281088.ref005","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs","volume":"316","author":"V Gulshan","year":"2016","journal-title":"JAMA"},{"key":"pone.0281088.ref006","doi-asserted-by":"crossref","first-page":"1836","DOI":"10.1093\/annonc\/mdy166","article-title":"Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists","volume":"29","author":"HA Haenssle","year":"2018","journal-title":"Ann Oncol"},{"issue":"1","key":"pone.0281088.ref007","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.gie.2018.07.037","article-title":"Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks","volume":"89","author":"Y Horie","year":"2019","journal-title":"Gastrointest Endosc"},{"issue":"4","key":"pone.0281088.ref008","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1007\/s10120-018-0793-2","article-title":"Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images","volume":"21","author":"T Hirasawa","year":"2018","journal-title":"Gastric Cancer"},{"key":"pone.0281088.ref009","doi-asserted-by":"crossref","first-page":"104231","DOI":"10.1016\/j.ijmedinf.2020.104231","article-title":"Classification of glomerular pathological findings using deep learning and nephrologist\u2013AI collective intelligence approach","volume":"141","author":"E Uchino","year":"2020","journal-title":"International Journal of Medical Informatics"},{"issue":"8","key":"pone.0281088.ref010","doi-asserted-by":"crossref","first-page":"e0184059","DOI":"10.1371\/journal.pone.0184059","volume":"12","author":"P Burlina","year":"2017","journal-title":"PLoS ONE"},{"issue":"e1","key":"pone.0281088.ref011","doi-asserted-by":"crossref","first-page":"e151","DOI":"10.1136\/amiajnl-2014-002642","article-title":"Automatic identification of methotrexate-induced liver toxicity in patients with rheumatoid arthritis from the electronic medical record. d","volume":"22","author":"C Lin","year":"2015","journal-title":"Journal of the American Medical Informatics Association"},{"key":"pone.0281088.ref012","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.compbiomed.2018.08.029","article-title":"Predicting hospital readmission for lupus patients: An RNN-LSTM-based deep-learning methodology","volume":"101","author":"BK Raddy","year":"2018","journal-title":"Computers in Biology and Medicine"},{"issue":"Issue 2","key":"pone.0281088.ref013","article-title":"Development and validation of a deep-learning model for scoring of radiographic finger joint destruction in rheumatoid arthritis","volume":"Volume 3","author":"T Hirano","year":"2019","journal-title":"Rheumatology Advances in Practice"},{"issue":"suppl 10","key":"pone.0281088.ref014","article-title":"Detecting Hand Joint Ankylosis in Radiographic Images Using Deep Learning: A Step in Developing Automatic Radiographic Scoring System for Bone Destruction","volume":"70","author":"K Izumi","year":"2018","journal-title":"Arthritis Rheumatol"},{"key":"pone.0281088.ref015","unstructured":"Miyato T, Maeda S, Koyama M, Nakae K, Ishii S. Distributional Smoothing with Virtual Adversarial Training. In Proceedings of the International Conference on Learning Representations, 2016."},{"key":"pone.0281088.ref016","article-title":"MixMatch: A Holistic Approach to Semi- Supervised Learning","author":"D Berthelot","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pone.0281088.ref017","unstructured":"Kingma DP, Welling M. Auto-Encoding Variational Bayes. In Proceedings of the International Conference on Learning Representations, 2014."},{"key":"pone.0281088.ref018","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770\u2013778, 2016.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pone.0281088.ref019","doi-asserted-by":"crossref","unstructured":"Huang G, Liu Z, van der Maaten L, Weinberger KQ. Densely Connected Convolutional Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.","DOI":"10.1109\/CVPR.2017.243"},{"key":"pone.0281088.ref020","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In Proceedings of the International Conference on Learning Representations, 2021."},{"key":"pone.0281088.ref021","first-page":"6000","article-title":"Attention is all you need","author":"A Vaswani","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pone.0281088.ref022","unstructured":"Kingma DP, Ba JL. Adam: A Method for Stochastic Optimization. In Proceedings of the International Conference on Learning Representations, pp. 1\u201313, 2015."},{"key":"pone.0281088.ref023","unstructured":"Loshchilov I, Hutter F. Decoupled Weight Decay Regularization. In Proceedings of the International Conference on Learning Representations, 2019."},{"key":"pone.0281088.ref024","doi-asserted-by":"crossref","unstructured":"Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In Proceedings of IEEE International Conference on Computer Vision, pp. 618\u2013626, 2017.","DOI":"10.1109\/ICCV.2017.74"},{"issue":"1","key":"pone.0281088.ref025","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1186\/s13075-022-02914-7","article-title":"Deep learning-based automatic-bone-destruction-evaluation system using contextual information from other joints","volume":"24","author":"K Miyama","year":"2022","journal-title":"Arthritis Res Ther"},{"issue":"8","key":"pone.0281088.ref026","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/09544119221109735","article-title":"Automated evaluation of rheumatoid arthritis from hand radiographs using Machine Learning and deep learning techniques","volume":"236","author":"RK Ahalya","year":"2022","journal-title":"Proc Inst Mech Eng H"},{"issue":"6","key":"pone.0281088.ref027","doi-asserted-by":"crossref","first-page":"1355","DOI":"10.3390\/biomedicines10061355","article-title":"Deep Learning-Based Computer-Aided Diagnosis of Rheumatoid Arthritis with Hand X-ray Images Conforming to Modified Total Sharp\/van der Heijde Score","volume":"10","author":"HJ Wang","year":"2022","journal-title":"Biomedicines"},{"issue":"2","key":"pone.0281088.ref028","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1007\/s10278-021-00564-w","article-title":"Automated Classification of Rheumatoid Arthritis, Osteoarthritis, and Normal Hand Radiographs with Deep Learning Methods","volume":"35","author":"K \u00dcreten","year":"2022","journal-title":"J Digit Imaging"},{"issue":"suppl 2","key":"pone.0281088.ref029","first-page":"1364","article-title":"Automatic detection of hand joint region, ankylosis and subluxation in radiographic images using deep learning: development of artificial intelligence-based radiographic evaluation system for bone destruction","volume":"78","author":"K Izumi","year":"2019","journal-title":"Annals of the Rheumatic Diseases"},{"issue":"suppl 1","key":"pone.0281088.ref030","doi-asserted-by":"crossref","first-page":"1841","DOI":"10.1136\/annrheumdis-2020-eular.4107","article-title":"Automatic finger joint bone erosion score prediction considering 2-time-point X-rays of patients with rheumatoid arthritis by deep learning","volume":"79","author":"K Izumi","year":"2020","journal-title":"Annals of the Rheumatic Diseases"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0281088","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,13]],"date-time":"2023-02-13T14:37:32Z","timestamp":1676299052000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0281088"}},"subtitle":[],"editor":[{"given":"Jeonghwan","family":"Gwak","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2023,2,13]]},"references-count":30,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2,13]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0281088","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,13]]}}}