{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T07:11:39Z","timestamp":1793171499281,"version":"4.5.0"},"reference-count":36,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Osteoporosis is a bone disease characterized by reduced bone mineral density and mass, which increase the risk of fragility fractures in patients. Artificial intelligence can mine imaging features specific to different bone densities, shapes, and structures and fuse other multimodal features for synergistic diagnosis to improve prediction accuracy.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aims to develop a multimodal model that fuses chest X-rays and clinical parameters for opportunistic screening of osteoporosis and to compare and analyze the experimental results with existing methods.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We used multimodal data, including chest X-ray images and clinical data, from a total of 1780 patients at Chongqing Daping Hospital from January 2019 to August 2024. We adopted a probability fusion strategy to construct a multimodal model. In our model, we used a convolutional neural network as the backbone network for image processing and fine-tuned it using a transfer learning technique to suit the specific task of this study. In addition, we introduced a gradient-based wavelet feature extraction method. We combined it with an attention mechanism to assist in feature fusion, which enhanced the model\u2019s focus on key regions of the image and further improved its ability to extract image features.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n The multimodal model proposed in this paper outperforms the traditional methods in the 4 evaluation metrics of area under the curve value, accuracy, sensitivity, and specificity. Compared with using only the X-ray image model, the multimodal model improved the area under the curve value significantly from 0.951 to 0.975 (\n P<\/jats:italic>\n =.004), the accuracy from 89.32% to 92.36% (\n P<\/jats:italic>\n =.045), the sensitivity from 89.82% to 91.23% (\n P<\/jats:italic>\n =.03), and the specificity from 88.64% to 93.92% (\n P<\/jats:italic>\n =.008).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n While the multimodal model that fuses chest X-ray images and clinical data demonstrated superior performance compared to unimodal models and traditional methods, this study has several limitations. The dataset size may not be sufficient to capture the full diversity of the population. The retrospective nature of the study may introduce selection bias, and the lack of external validation limits the generalizability of the findings. Future studies should address these limitations by incorporating larger, more diverse datasets and conducting rigorous external validation to further establish the model\u2019s clinical use.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/70738","type":"journal-article","created":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T19:56:18Z","timestamp":1758225378000},"page":"e70738-e70738","source":"Crossref","is-referenced-by-count":17,"title":["Fusion of X-Ray Images and Clinical Data for a Multimodal Deep Learning Prediction Model of Osteoporosis: Algorithm Development and Validation Study"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-8900-7810","authenticated-orcid":false,"given":"Jun","family":"Tang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1662-6537","authenticated-orcid":false,"given":"Xiang","family":"Yin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0157-7657","authenticated-orcid":false,"given":"Jiangyuan","family":"Lai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3329-3714","authenticated-orcid":false,"given":"Keyu","family":"Luo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7826-5963","authenticated-orcid":false,"given":"Dongdong","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,9,18]]},"reference":[{"issue":"12","key":"R1","doi-asserted-by":"publisher","first-page":"2485","DOI":"10.1007\/s00198-021-06023-0","article-title":"Excess mortality after hip fracture: fracture or pre-fall comorbidity?","volume":"32","author":"Liow","journal-title":"Osteoporos Int"},{"key":"R2","volume":"124","author":"Chiu","journal-title":"Stud Health Technol Inform"},{"issue":"5","key":"R3","doi-asserted-by":"publisher","first-page":"833","DOI":"10.1002\/jbmr.4292","article-title":"Machine learning solutions for osteoporosis-a review","volume":"36","author":"Smets","journal-title":"J Bone Miner Res"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","first-page":"375","DOI":"10.17305\/bb.2024.10725","article-title":"Development and evaluation of interpretable machine learning regressors for predicting femoral neck bone mineral density in elderly men using NHANES data","volume":"25","author":"He","journal-title":"Biomol Biomed"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1007\/s00198-023-06913-5","article-title":"Establish and validate the reliability of predictive models in bone mineral density by deep learning as examination tool for women","volume":"35","author":"Hung","journal-title":"Osteoporos Int"},{"issue":"8","key":"R6","doi-asserted-by":"publisher","first-page":"588","DOI":"10.7326\/0003-4819-158-8-201304160-00003","article-title":"Opportunistic screening for osteoporosis using abdominal computed tomography scans obtained for other indications","volume":"158","author":"Pickhardt","journal-title":"Ann Intern Med"},{"issue":"7","key":"R7","doi-asserted-by":"publisher","first-page":"5097","DOI":"10.1007\/s00330-023-09421-6","article-title":"Automatic segmentation and radiomic texture analysis for osteoporosis screening using chest low-dose computed tomography","volume":"33","author":"Chen","journal-title":"Eur Radiol"},{"issue":"4","key":"R8","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1007\/s00198-021-06129-5","article-title":"Opportunistic use of lumbar magnetic resonance imaging for osteoporosis screening","volume":"33","author":"Kadri","journal-title":"Osteoporos Int"},{"issue":"4","key":"R9","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1007\/s00256-019-03342-6","article-title":"The exploration of feature extraction and machine learning for predicting bone density from simple spine X-ray images in a Korean population","volume":"49","author":"Lee","journal-title":"Skeletal Radiol"},{"issue":"2","key":"R10","doi-asserted-by":"publisher","first-page":"84","DOI":"10.26599\/BDMA.2020.9020012","article-title":"Diagnosis of COVID-19 from chest X-ray images using wavelets-based depthwise convolution network","volume":"4","author":"Singh","journal-title":"Big Data Min Anal"},{"issue":"2","key":"R11","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1002\/jbmr.4477","article-title":"Opportunistic osteoporosis screening using chest radiographs with deep learning: development and external validation with a cohort dataset","volume":"37","author":"Jang","journal-title":"J Bone Miner Res"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1016\/j.bone.2020.115561","article-title":"Deep learning of lumbar spine X-ray for osteopenia and osteoporosis screening: a multicenter retrospective cohort study","volume":"140","author":"Zhang","journal-title":"Bone"},{"issue":"3","key":"R13","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1016\/j.bone.2007.11.001","article-title":"A reference standard for the description of osteoporosis","volume":"42","author":"Kanis","journal-title":"Bone"},{"key":"R14","doi-asserted-by":"publisher","DOI":"10.3389\/fendo.2022.971877","article-title":"Deep learning for screening primary osteopenia and osteoporosis using spine radiographs and patient clinical covariates in a Chinese population","volume":"13","author":"Mao","journal-title":"Front Endocrinol (Lausanne)"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-25779-x","article-title":"Automated bone mineral density prediction and fracture risk assessment using plain radiographs via deep learning","volume":"12","author":"Hsieh","journal-title":"Nat Commun"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","DOI":"10.1186\/s13244-021-01137-9","article-title":"Deep learning approach for automatic segmentation of ulna and radius in dual-energy X-ray imaging","volume":"12","author":"Yang","journal-title":"Insights Imaging"},{"issue":"1","key":"R17","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1109\/TMI.2022.3209648","article-title":"Lumbar bone mineral density estimation from chest X-ray images: anatomy-aware attentive multi-ROI modeling","volume":"42","author":"Wang","journal-title":"IEEE Trans Med Imaging"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-023-02030-2","article-title":"Artificial intelligence-enabled chest X-ray classifies osteoporosis and identifies mortality risk","volume":"48","author":"Tsai","journal-title":"J Med Syst"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-00341-z","article-title":"Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines","volume":"3","author":"Huang","journal-title":"NPJ Digit Med"},{"key":"R20","doi-asserted-by":"crossref","unstructured":"Huang SC Shen L Lungren MP Yeung S . GLoRIA: a multimodal global-local representation learning framework for label-efficient medical image recognition. Presented at: 2021 IEEE\/CVF International Conference on Computer Vision (ICCV); Oct 10-17, 2021:3922-3931; Montreal, QC, Canada. [doi: 10.1109\/ICCV48922.2021.00391]","DOI":"10.1109\/ICCV48922.2021.00391"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.3043125","article-title":"A bilevel integrated model with data-driven layer ensemble for multi-modality image fusion","volume":"30","author":"Liu","journal-title":"IEEE Trans Image Process"},{"issue":"2","key":"R22","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbab569","article-title":"Multimodal deep learning for biomedical data fusion: a review","volume":"23","author":"Stahlschmidt","journal-title":"Brief Bioinform"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-53450-8","article-title":"An explainable longitudinal multi-modal fusion model for predicting neoadjuvant therapy response in women with breast cancer","volume":"15","author":"Gao","journal-title":"Nat Commun"},{"issue":"12","key":"R24","doi-asserted-by":"publisher","first-page":"4335","DOI":"10.1109\/TMI.2020.3017160","article-title":"Hybrid MPI-MRI System for dual-modal in situ cardiovascular assessments of real-time 3D blood flow quantification-a pre-clinical in vivo feasibility investigation","volume":"39","author":"Franke","journal-title":"IEEE Trans Med Imaging"},{"issue":"7","key":"R25","doi-asserted-by":"publisher","first-page":"3500","DOI":"10.1109\/TIP.2018.2820424","article-title":"Region-adaptive deformable registration of CT\/MRI pelvic images via learning-based image synthesis","volume":"27","author":"Cao","journal-title":"IEEE Trans on Image Process"},{"key":"R26","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.107704","article-title":"Fusion of electronic health records and radiographic images for a multimodal deep learning prediction model of atypical femur fractures","volume":"168","author":"Schilcher","journal-title":"Comput Biol Med"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-56527-3","article-title":"Multimodal machine learning-based knee osteoarthritis progression prediction from plain radiographs and clinical data","volume":"9","author":"Tiulpin","journal-title":"Sci Rep"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1109\/JPROC.2020.3004555","article-title":"A comprehensive survey on transfer learning","volume":"109","author":"Zhuang","journal-title":"Proc IEEE"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"Pillai R Sharma A Sharma N Gupta R . Brain tumor classification using VGG 16, ResNet50, and Inception V3 transfer learning models. Presented at: 2023 2nd International Conference for Innovation in Technology (INOCON); Mar 3-5, 2023:1-5; Bangalore, India. [doi: 10.1109\/INOCON57975.2023.10101252]","DOI":"10.1109\/INOCON57975.2023.10101252"},{"key":"R30","doi-asserted-by":"crossref","unstructured":"Holste G Partridge SC Rahbar H Biswas D Lee CI Alessio AM . End-to-end learning of fused image and non-image features for improved breast cancer classification from MRI. Presented at: 2021 IEEE\/CVF International Conference on Computer Vision Workshops (ICCVW); Oct 11-17, 2021:3287-3296; Montreal, BC, Canada. [doi: 10.1109\/ICCVW54120.2021.00368]","DOI":"10.1109\/ICCVW54120.2021.00368"},{"key":"R31","doi-asserted-by":"crossref","unstructured":"Xue J Rao Y Yao Q Wu J . 3H-ObjectDet: a target detection method based on Haar wavelet transform downsampling and hybrid attention mechanism for medical fracture image. Presented at: 2024 4th International Conference on Electronic Information Engineering and Computer Communication (EIECC); 2024:507-513; Wuhan, China. [doi: 10.1109\/EIECC64539.2024.10929242]","DOI":"10.1109\/EIECC64539.2024.10929242"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Roy A Sarkar S Ghosal S Kaplun D Lyanova A Sarkar R . A wavelet guided attention module for skin cancer classification with gradient-based feature fusion. Presented at: 2024 IEEE International Symposium on Biomedical Imaging (ISBI); May 27-30, 2024:1-4; Athens, Greece. [doi: 10.1109\/ISBI56570.2024.10635704]","DOI":"10.1109\/ISBI56570.2024.10635704"},{"key":"R33","unstructured":"Bahdanau D Cho K Bengio Y . Neural machine translation by jointly learning to align and translate. arXiv. Preprint posted online on Sep 1, 2014. [doi: 10.48550\/arXiv.1409.0473]"},{"issue":"8","key":"R34","doi-asserted-by":"publisher","DOI":"10.2196\/38082","article-title":"Predicting mortality in intensive care unit patients with heart failure using an interpretable machine learning model: retrospective cohort study","volume":"24","author":"Li","journal-title":"J Med Internet Res"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","DOI":"10.1186\/s12967-020-02620-5","article-title":"Predicting 30-days mortality for MIMIC-III patients with sepsis-3: a machine learning approach using XGboost","volume":"18","author":"Hou","journal-title":"J Transl Med"},{"issue":"9","key":"R36","doi-asserted-by":"publisher","first-page":"14193","DOI":"10.1007\/s11042-022-13911-y","article-title":"Osteoporosis diagnosis in knee X-rays by transfer learning based on convolution neural network","volume":"82","author":"Wani","journal-title":"Multimed Tools Appl"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e70738","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:59:37Z","timestamp":1782842377000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e70738"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,18]]},"references-count":36,"URL":"https:\/\/doi.org\/10.2196\/70738","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,18]]},"article-number":"v13i12e70738"}}