{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T00:05:36Z","timestamp":1792800336602,"version":"4.5.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T00:00:00Z","timestamp":1720396800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T00:00:00Z","timestamp":1720396800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"the National Research Foundation of Korea (NRF) funded by the Korean government","award":["2019R1A5A2026045"],"award-info":[{"award-number":["2019R1A5A2026045"]}],"id":[{"id":"https:\/\/ror.org\/013aysd81","id-type":"ROR","asserted-by":"crossref"}]},{"name":"the Korea Health Industry Development Institute (KHIDI) funded by the Ministry of Health & Welfare, Republic of Korea","award":["HR21C1003"],"award-info":[{"award-number":["HR21C1003"]}],"id":[{"id":"https:\/\/ror.org\/00fdzyk40","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Recent advances in Vision Transformer (ViT)-based deep learning have significantly improved the accuracy of lung disease prediction from chest X-ray images. However, limited research exists on comparing the effectiveness of different optimizers for lung disease prediction within ViT models. This study aims to systematically evaluate and compare the performance of various optimization methods for ViT-based models in predicting lung diseases from chest X-ray images.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This study utilized a chest X-ray image dataset comprising 19,003 images containing both normal cases and six lung diseases: COVID-19, Viral Pneumonia, Bacterial Pneumonia, Middle East Respiratory Syndrome (MERS), Severe Acute Respiratory Syndrome (SARS), and Tuberculosis. Each ViT model (ViT, FastViT, and CrossViT) was individually trained with each optimization method (Adam, AdamW, NAdam, RAdam, SGDW, and Momentum) to assess their performance in lung disease prediction.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>When tested with ViT on the dataset with balanced-sample sized classes, RAdam demonstrated superior accuracy compared to other optimizers, achieving 95.87%. In the dataset with imbalanced sample size, FastViT with NAdam achieved the best performance with an accuracy of 97.63%.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>We provide comprehensive optimization strategies for developing ViT-based model architectures, which can enhance the performance of these models for lung disease prediction from chest X-ray images.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12911-024-02591-3","type":"journal-article","created":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T08:04:18Z","timestamp":1720425858000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":50,"title":["Optimization of vision transformer-based detection of lung diseases from chest X-ray images"],"prefix":"10.1186","volume":"24","author":[{"given":"Jinsol","family":"Ko","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Soyeon","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyun Goo","family":"Woo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,7,8]]},"reference":[{"key":"2591_CR1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105581","volume":"196","author":"AI Khan","year":"2020","unstructured":"Khan AI, Shah JL, Bhat MM. CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray images. Comput Methods Programs Biomed. 2020;196: 105581.","journal-title":"Comput Methods Programs Biomed"},{"issue":"1","key":"2591_CR2","doi-asserted-by":"publisher","first-page":"19549","DOI":"10.1038\/s41598-020-76550-z","volume":"10","author":"L Wang","year":"2020","unstructured":"Wang L, Lin ZQ, Wong A. COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images. Sci Rep. 2020;10(1):19549.","journal-title":"Sci Rep"},{"issue":"5","key":"2591_CR3","first-page":"821","volume":"28","author":"MM Rahaman","year":"2020","unstructured":"Rahaman MM, Li C, Yao Y, Kulwa F, Rahman MA, Wang Q, Qi S, Kong F, Zhu X, Zhao X. Identification of COVID-19 samples from chest X-Ray images using deep learning: A comparison of transfer learning approaches. J Xray Sci Technol. 2020;28(5):821\u201339.","journal-title":"J Xray Sci Technol"},{"issue":"5","key":"2591_CR4","doi-asserted-by":"publisher","first-page":"820","DOI":"10.1109\/JPROC.2021.3054390","volume":"109","author":"SK Zhou","year":"2021","unstructured":"Zhou SK, Greenspan H, Davatzikos C, Duncan JS, Van Ginneken B, Madabhushi A, Prince JL, Rueckert D, Summers RM. A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises. Proc IEEE. 2021;109(5):820\u201338.","journal-title":"Proc IEEE"},{"key":"2591_CR5","volume-title":"An image is worth 16x16 words: Transformers for image recognition at scale","author":"A Dosovitskiy","year":"2020","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S. An image is worth 16x16 words: Transformers for image recognition at scale. 2020. arXiv preprint arXiv:201011929."},{"key":"2591_CR6","doi-asserted-by":"publisher","unstructured":"Vaswani A, et al. Attention is all you need. Advances in neural information processing systems. 2017;30. https:\/\/doi.org\/10.48550\/arXiv.1706.03762.","DOI":"10.48550\/arXiv.1706.03762"},{"key":"2591_CR7","doi-asserted-by":"publisher","first-page":"1100110","DOI":"10.1109\/JTEHM.2021.3134096","volume":"10","author":"AK Mondal","year":"2022","unstructured":"Mondal AK, Bhattacharjee A, Singla P, Prathosh AP. xViTCOS: Explainable Vision Transformer Based COVID-19 Screening Using Radiography. IEEE J Transl Eng Health Med. 2022;10:1100110.","journal-title":"IEEE J Transl Eng Health Med"},{"issue":"11","key":"2591_CR8","doi-asserted-by":"publisher","first-page":"3013","DOI":"10.3390\/jcm11113013","volume":"11","author":"M Chetoui","year":"2022","unstructured":"Chetoui M, Akhloufi MA. Explainable Vision Transformers and Radiomics for COVID-19 Detection in Chest X-rays. J Clin Med. 2022;11(11):3013.","journal-title":"J Clin Med"},{"key":"2591_CR9","first-page":"644","volume-title":"2021 6th International Conference on Signal Processing, Computing and Control (ISPCC): 2021: IEEE","author":"KS Krishnan","year":"2021","unstructured":"Krishnan KS, Krishnan KS. Vision transformer based COVID-19 detection using chest X-rays. In: 2021 6th International Conference on Signal Processing, Computing and Control (ISPCC): 2021: IEEE. 2021. p. 644\u20138."},{"issue":"21","key":"2591_CR10","doi-asserted-by":"publisher","first-page":"11086","DOI":"10.3390\/ijerph182111086","volume":"18","author":"D Shome","year":"2021","unstructured":"Shome D, Kar T, Mohanty SN, Tiwari P, Muhammad K, AlTameem A, Zhang Y, Saudagar AKJ. COVID-Transformer: Interpretable COVID-19 Detection Using Vision Transformer for Healthcare. Int J Environ Res Public Health. 2021;18(21):11086.","journal-title":"Int J Environ Res Public Health"},{"key":"2591_CR11","volume-title":"FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization","author":"PKA Vasu","year":"2023","unstructured":"Vasu PKA, Gabriel J, Zhu J, Tuzel O, Ranjan A. FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization. 2023. arXiv preprint arXiv:230314189."},{"key":"2591_CR12","first-page":"357","volume-title":"Proceedings of the IEEE\/CVF international conference on computer vision: 2021","author":"C-FR Chen","year":"2021","unstructured":"Chen C-FR, Fan Q, Panda R. Crossvit: Cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE\/CVF international conference on computer vision: 2021. 2021. p. 357\u201366."},{"issue":"11","key":"2591_CR13","doi-asserted-by":"publisher","first-page":"2466","DOI":"10.3390\/math11112466","volume":"11","author":"R Abdulkadirov","year":"2023","unstructured":"Abdulkadirov R, Lyakhov P, Nagornov N. Survey of Optimization Algorithms in Modern Neural Networks. Mathematics. 2023;11(11):2466.","journal-title":"Mathematics"},{"issue":"4","key":"2591_CR14","doi-asserted-by":"publisher","first-page":"2080","DOI":"10.3390\/app12042080","volume":"12","author":"AA AbdElhamid","year":"2022","unstructured":"AbdElhamid AA, AbdElhalim E, Mohamed MA, Khalifa F. Multi-Classification of Chest X-rays for COVID-19 Diagnosis Using Deep Learning Algorithms. Appl Sci. 2022;12(4):2080.","journal-title":"Appl Sci"},{"key":"2591_CR15","doi-asserted-by":"crossref","unstructured":"Appasami G, Nickolas S. A deep learning-based COVID-19 classification from chest X-ray image: case study.\u00a0Eur Phys J Spec Top. 2022;231(18):3767\u201377.","DOI":"10.1140\/epjs\/s11734-022-00647-x"},{"key":"2591_CR16","doi-asserted-by":"publisher","unstructured":"Fan Z, Jamil M, Sadiq MT, Huang X, Yu X. Exploiting Multiple Optimizers with Transfer Learning Techniques for the Identification of COVID-19 Patients.\u00a0J\u00a0Healthc\u00a0Eng.\u00a02020;2020(1):8889412. https:\/\/doi.org\/10.1155\/2020\/8889412.","DOI":"10.1155\/2020\/8889412"},{"key":"2591_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.106646","volume":"155","author":"FJM Shamrat","year":"2023","unstructured":"Shamrat FJM, Azam S, Karim A, Ahmed K, Bui FM, De Boer F. High-precision multiclass classification of lung disease through customized MobileNetV2 from chest X-ray images. Comput Biol Med. 2023;155: 106646.","journal-title":"Comput Biol Med"},{"issue":"2","key":"2591_CR18","doi-asserted-by":"publisher","first-page":"310","DOI":"10.3390\/jpm12020310","volume":"12","author":"MM Al Rahhal","year":"2022","unstructured":"Al Rahhal MM, Bazi Y, Jomaa RM, AlShibli A, Alajlan N, Mekhalfi ML, Melgani F. COVID-19 Detection in CT\/X-ray Imagery Using Vision Transformers. J Pers Med. 2022;12(2):310.","journal-title":"J Pers Med"},{"key":"2591_CR19","volume-title":"Shake-shake regularization","author":"X Gastaldi","year":"2017","unstructured":"Gastaldi X. Shake-shake regularization. 2017. arXiv preprint arXiv:170507485."},{"key":"2591_CR20","volume-title":"Autoaugment: Learning augmentation policies from data","author":"ED Cubuk","year":"2018","unstructured":"Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV. Autoaugment: Learning augmentation policies from data. 2018. arXiv preprint arXiv:180509501."},{"key":"2591_CR21","unstructured":"RSNA Pneumonia Detection Challenge [https:\/\/www.kaggle.com\/c\/rsna-pneumonia-detection-challenge\/data]."},{"key":"2591_CR22","unstructured":"COVID-19 Chest X-Ray Image Repository [https:\/\/figshare.com\/articles\/COVID-19_Chest_X-Ray_Image_Repository\/12580328]."},{"key":"2591_CR23","unstructured":"COVID-CXNet [https:\/\/github.com\/armiro\/COVID-CXNet]."},{"key":"2591_CR24","unstructured":"BIMCV-COVID-19 [https:\/\/bimcv.cipf.es\/bimcv-projects\/bimcv-covid19\/#1590858128006-9e640421-6711]."},{"key":"2591_CR25","unstructured":"Eurorad [https:\/\/eurorad.org]."},{"key":"2591_CR26","unstructured":"covid-chestxray-dataset [https:\/\/github.com\/ieee8023\/covid-chestxray-dataset]."},{"key":"2591_CR27","unstructured":"covid-19-image-repository [https:\/\/github.com\/ml-workgroup\/covid-19-image-repository\/tree\/master\/png]."},{"key":"2591_CR28","unstructured":"Chest X-Ray Images (Pneumonia) [https:\/\/www.kaggle.com\/paultimothymooney\/chest-xray-pneumonia]."},{"key":"2591_CR29","unstructured":"SIRM [https:\/\/sirm.org\/category\/senza-categoria\/covid-19\/]."},{"key":"2591_CR30","unstructured":"Tuberculosis (TB) Chest X-ray Database [https:\/\/www.kaggle.com\/datasets\/tawsifurrahman\/tuberculosis-tb-chest-xray-dataset]."},{"key":"2591_CR31","unstructured":"COVID-19, SARS, MERS X-ray Images Dataset [https:\/\/www.kaggle.com\/datasets\/057e1b6dc41d9691e59dded4445fa8cc2f0b4b5cbcb49aef9583d95233799d5a]."},{"key":"2591_CR32","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","volume-title":"2009 IEEE conference on computer vision and pattern recognition: 2009: Ieee","author":"J Deng","year":"2009","unstructured":"Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition: 2009: Ieee. 2009. p. 248\u201355."},{"issue":"4","key":"2591_CR33","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.icte.2020.04.010","volume":"6","author":"I Kandel","year":"2020","unstructured":"Kandel I, Castelli M. The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset. ICT express. 2020;6(4):312\u20135.","journal-title":"ICT express"},{"key":"2591_CR34","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105532","volume":"194","author":"RM Pereira","year":"2020","unstructured":"Pereira RM, Bertolini D, Teixeira LO, Silla CN Jr, Costa YMG. COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios. Comput Methods Programs Biomed. 2020;194: 105532.","journal-title":"Comput Methods Programs Biomed"},{"key":"2591_CR35","first-page":"345","volume-title":"Proceedings of the 27th European conference on Advances in Information Retrieval Research. Santiago de Compostela, Spain: Springer-Verlag","author":"C Goutte","year":"2005","unstructured":"Goutte C, Gaussier E. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: Proceedings of the 27th European conference on Advances in Information Retrieval Research. Santiago de Compostela, Spain: Springer-Verlag. 2005. p. 345\u201359."},{"issue":"1","key":"2591_CR36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0192-5","volume":"6","author":"JM Johnson","year":"2019","unstructured":"Johnson JM, Khoshgoftaar TM. Survey on deep learning with class imbalance. Journal of Big Data. 2019;6(1):1\u201354.","journal-title":"Journal of Big Data"},{"issue":"6","key":"2591_CR37","doi-asserted-by":"publisher","first-page":"962","DOI":"10.1109\/72.286891","volume":"4","author":"R Anand","year":"1993","unstructured":"Anand R, Mehrotra KG, Mohan CK, Ranka S. An improved algorithm for neural network classification of imbalanced training sets. IEEE Trans Neural Networks. 1993;4(6):962\u20139.","journal-title":"IEEE Trans Neural Networks"},{"issue":"7","key":"2591_CR38","doi-asserted-by":"publisher","first-page":"427","DOI":"10.3390\/brainsci10070427","volume":"10","author":"M Yaqub","year":"2020","unstructured":"Yaqub M, Feng J, Zia MS, Arshid K, Jia K, Rehman ZU, Mehmood A. State-of-the-art CNN optimizer for brain tumor segmentation in magnetic resonance images. Brain Sci. 2020;10(7):427.","journal-title":"Brain Sci"},{"key":"2591_CR39","first-page":"771","volume-title":"2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE): 2021: IEEE","author":"Z Wan","year":"2021","unstructured":"Wan Z, Yuxiang Z, Gong X, Yu B. DenseNet model with RAdam optimization algorithm for cancer image classification. In: 2021 IEEE International Conference on Consumer Electronics and Computer Engineering (ICCECE): 2021: IEEE. 2021. p. 771\u20135."},{"key":"2591_CR40","volume-title":"Journal of Physics: Conference Series: 2021: IOP Publishing","author":"C Liu","year":"2021","unstructured":"Liu C, Yin Q. Automatic diagnosis of covid-19 using a tailored transformer-like network. In: Journal of Physics: Conference Series: 2021: IOP Publishing. 2021. 012175."},{"issue":"1","key":"2591_CR41","doi-asserted-by":"publisher","first-page":"454","DOI":"10.3390\/app13010454","volume":"13","author":"K An","year":"2022","unstructured":"An K, Zhang Y. A Self-Supervised Detail-Sensitive ViT-Based Model for COVID-19 X-ray Image Diagnosis: SDViT. Appl Sci. 2022;13(1):454.","journal-title":"Appl Sci"},{"key":"2591_CR42","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2022.107141","volume":"226","author":"GI Okolo","year":"2022","unstructured":"Okolo GI, Katsigiannis S, Ramzan N. IEViT: An enhanced vision transformer architecture for chest X-ray image classification. Comput Methods Programs Biomed. 2022;226: 107141.","journal-title":"Comput Methods Programs Biomed"},{"key":"2591_CR43","volume-title":"Incorporating nesterov momentum into adam","author":"T Dozat","year":"2016","unstructured":"Dozat T. Incorporating nesterov momentum into adam. 2016."}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-024-02591-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-024-02591-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-024-02591-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T08:04:59Z","timestamp":1720425899000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-024-02591-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,8]]},"references-count":43,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["2591"],"URL":"https:\/\/doi.org\/10.1186\/s12911-024-02591-3","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,8]]},"assertion":[{"value":"22 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 June 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 July 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"191"}}