{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T04:11:57Z","timestamp":1783051917190,"version":"3.54.6"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T00:00:00Z","timestamp":1778112000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T00:00:00Z","timestamp":1779840000000},"content-version":"vor","delay-in-days":20,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Medical image interpretation plays a critical role in lumbar fusion surgery, where accurate analysis of anatomical structures is essential for clinical assessment. However, most existing deep learning approaches rely primarily on visual features and fail to effectively integrate heterogeneous clinical information. This study proposes a multimodal deep learning framework for lumbar spine image interpretation by jointly modeling medical images and associated clinical text. The framework adopts a global\u2013local representation learning strategy to capture both overall anatomical context and fine-grained structural information. A visual encoder extracts hierarchical features from lumbar radiographs and CT scans, while a transformer-based text encoder captures semantic information from clinical reports. These representations are projected into a shared embedding space to enable cross-modal alignment. To enhance feature interaction, a text-guided attention mechanism is introduced to model correspondence between image regions and textual descriptions. The learned multimodal representations are applied to multiple downstream tasks, including cross-modal retrieval, classification, and lumbar structure segmentation. Experimental results show that the proposed framework outperforms image-only baselines and achieves competitive performance compared with existing multimodal approaches. The integration of global and local representations improves feature discrimination and structural modeling. Visualization results provide qualitative evidence that the model focuses on anatomically relevant regions, although such observations should be interpreted with caution. Overall, the proposed framework demonstrates the potential of multimodal representation learning for lumbar spine image analysis and provides a structured approach for integrating heterogeneous clinical data.<\/jats:p>","DOI":"10.1038\/s41598-026-51495-x","type":"journal-article","created":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T12:25:50Z","timestamp":1778156750000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A multimodal deep learning framework for clinical nursing assessment in lumbar fusion surgery via representation learning and feature extraction"],"prefix":"10.1038","volume":"16","author":[{"given":"Chao","family":"Li","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianshu","family":"Cai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoling","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yan","family":"Ni","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuping","family":"Ling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shibiao","family":"Fang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,5,7]]},"reference":[{"key":"51495_CR1","doi-asserted-by":"publisher","first-page":"929","DOI":"10.1016\/j.arth.2024.09.041","volume":"40","author":"MJ Solomito","year":"2025","unstructured":"Solomito, M. J., Witmer, D. K., Lucchio, P. & Kostyun, R. O. Utility of the Patient-Reported Outcomes Measurement Information System Global Mental Health T-Score to identify patients at risk for poor postsurgical outcomes following total hip arthroplasty. J. Arthroplasty 40, 929\u2013934 (2025).","journal-title":"J. Arthroplasty"},{"key":"51495_CR2","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1007\/s11517-019-02074-y","volume":"58","author":"Y Zhang","year":"2020","unstructured":"Zhang, Y. et al. Joined fragment segmentation for fractured bones using GPU-accelerated shape-preserving erosion and dilation. Med. Biol. Eng. Comput. 58, 155\u2013170 (2020).","journal-title":"Med. Biol. Eng. Comput."},{"key":"51495_CR3","doi-asserted-by":"crossref","unstructured":"Cai, J., Chen, J., Yu, T., He, L., Chen, L., Qi, H., Zhan, X., Yu, W., Huang, X., Huang, P. Multidisciplinary expert evaluation of large language models on questions regarding bariatric surgery: a comparative analysis of ERNIE Bot 4.0, ChatGPT-4, Claude 3 Opus, and Gemini Pro. Sci Rep. (2026).","DOI":"10.2139\/ssrn.5181651"},{"key":"51495_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jss.2024.10.029","volume":"305","author":"CJ Clark","year":"2025","unstructured":"Clark, C. J. et al. Colorectal surgery outcomes and healthcare burden for Medicare beneficiaries with dementia. J. Surg. Res. 305, 1\u20139 (2025).","journal-title":"J. Surg. Res."},{"key":"51495_CR5","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.compmedimag.2018.02.001","volume":"66","author":"WG Shadid","year":"2018","unstructured":"Shadid, W. G. & Willis, A. Bone fragment segmentation from 3D CT imagery. Comput. Med. Imaging Graph. 66, 14\u201327 (2018).","journal-title":"Comput. Med. Imaging Graph."},{"key":"51495_CR6","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1186\/s42836-024-00240-8","volume":"6","author":"AG Kim","year":"2024","unstructured":"Kim, A. G. et al. Resilience as a psychiatric factor affecting outcomes after total joint arthroplasty: A systematic review. Arthroplasty 6, 16 (2024).","journal-title":"Arthroplasty"},{"key":"51495_CR7","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/s11548-018-1883-7","volume":"14","author":"A Klein","year":"2019","unstructured":"Klein, A. et al. Automatic bone segmentation in whole-body CT images. Int. J. Comput. Assist. Radiol. Surg. 14, 21\u201329 (2019).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"51495_CR8","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1007\/s11265-019-01507-z","volume":"92","author":"M Sakamoto","year":"2020","unstructured":"Sakamoto, M. et al. Bayesian segmentation of hip and thigh muscles in metal artifact-contaminated CT using convolutional neural network-enhanced normalized metal artifact reduction. J. Signal Process. Syst. 92, 335\u2013344 (2020).","journal-title":"J. Signal Process. Syst."},{"key":"51495_CR9","doi-asserted-by":"publisher","first-page":"648","DOI":"10.1016\/j.ultrasmedbio.2016.11.003","volume":"43","author":"MA Hussain","year":"2017","unstructured":"Hussain, M. A., Hodgson, A. J. & Abugharbieh, R. Strain-initialized robust bone surface detection in 3-D ultrasound. Ultrasound Med. Biol. 43, 648\u2013661 (2017).","journal-title":"Ultrasound Med. Biol."},{"key":"51495_CR10","doi-asserted-by":"publisher","first-page":"2535","DOI":"10.2147\/NDT.S432776","volume":"19","author":"Cheng Junning","year":"2023","unstructured":"Cheng, J., Lao, Y., Chen, X., Qiao, X., Sui, W., Gong, X., Zhuang, Y. Dynamic Nomogram for Subsyndromal Delirium in Adult Intensive Care Unit: A Prospective Cohort Study. Neuropsychiatr Dis Treat.19, 2535-2548 (2023).","journal-title":"Neuropsychiatr Dis Treat."},{"key":"51495_CR11","doi-asserted-by":"publisher","first-page":"951","DOI":"10.1007\/s11548-017-1556-y","volume":"12","author":"I Hacihaliloglu","year":"2017","unstructured":"Hacihaliloglu, I. Enhancement of bone shadow region using local phase-based ultrasound transmission maps. Int. J. Comput. Assist. Radiol. Surg. 12, 951\u2013960 (2017).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"51495_CR12","doi-asserted-by":"crossref","unstructured":"Shi, L., Le, K., Qi, H., Feng, Y., Zhou, L., Wang, J., Xie, L. The safety and efficacy of delayed surgery by simulating clinical progression of observable papillary thyroid microcarcinoma: a retrospective analysis of 524 patients from a single medical center. Front Oncol. 13: 1046014 (2023).","DOI":"10.3389\/fonc.2023.1046014"},{"key":"51495_CR13","doi-asserted-by":"crossref","unstructured":"Hacihaliloglu, I. Localization of bone surfaces from ultrasound data using local phase information and signal transmission maps. In Computational Methods and Clinical Applications in Musculoskeletal Imaging, Cham, Switzerland, ; pp. 1\u201311. (2017).","DOI":"10.1007\/978-3-319-74113-0_1"},{"key":"51495_CR14","doi-asserted-by":"publisher","first-page":"1707","DOI":"10.1007\/s11548-018-1856-x","volume":"13","author":"M Villa","year":"2018","unstructured":"Villa, M. et al. FCN-based approach for the automatic segmentation of bone surfaces in ultrasound images. Int. J. Comput. Assist. Radiol. Surg. 13, 1707\u20131716 (2018).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"51495_CR15","doi-asserted-by":"crossref","unstructured":"Duong, D. Q. et al. Fully automated segmentation of alveolar bone using deep convolutional neural networks from intraoral ultrasound images. In Proceedings of the IEEE Engineering in Medicine and Biology Society (EMBC), USA, ; pp. 6632\u20136635. (2019).","DOI":"10.1109\/EMBC.2019.8857060"},{"key":"51495_CR16","doi-asserted-by":"crossref","unstructured":"Jiang, M., Cai, J., Li, L., Huang, X. Comparison of Effectiveness of Chinese Association of Operating Room Nurses-Based Nursing Care and Routine Nursing in Reducing Intraoperative Pressure Injury in Patients Undergoing Neurosurgery. Adv. Skin Wound Care. 38(4), 210-214 (2025).","DOI":"10.1097\/ASW.0000000000000285"},{"key":"51495_CR17","doi-asserted-by":"publisher","first-page":"1127","DOI":"10.1007\/s11548-020-02184-1","volume":"15","author":"P Wang","year":"2020","unstructured":"Wang, P. et al. Robust real-time bone surfaces segmentation from ultrasound using a local phase tensor-guided CNN. Int. J. Comput. Assist. Radiol. Surg. 15, 1127\u20131135 (2020).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"51495_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.compmedimag.2020.101766","volume":"84","author":"K Luan","year":"2020","unstructured":"Luan, K., Li, Z. & Li, J. An efficient end-to-end CNN for segmentation of bone surfaces from ultrasound. Comput. Med. Imaging Graph. 84, 101766 (2020).","journal-title":"Comput. Med. Imaging Graph."},{"key":"51495_CR19","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1007\/s11548-019-01934-0","volume":"14","author":"AZ Alsinan","year":"2019","unstructured":"Alsinan, A. Z., Patel, V. M. & Hacihaliloglu, I. Automatic segmentation of bone surfaces from ultrasound using a filter-layer-guided CNN. Int. J. Comput. Assist. Radiol. Surg. 14, 775\u2013783 (2019).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"51495_CR20","doi-asserted-by":"publisher","first-page":"1477","DOI":"10.1007\/s11548-020-02221-z","volume":"15","author":"AZ Alsinan","year":"2020","unstructured":"Alsinan, A. Z., Patel, V. M. & Hacihaliloglu, I. Bone shadow segmentation from ultrasound data for orthopedic surgery using GAN. Int. J. Comput. Assist. Radiol. Surg. 15, 1477\u20131485 (2020).","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"51495_CR21","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2021.106580","volume":"214","author":"F Chen","year":"2022","unstructured":"Chen, F. et al. TypeSeg: A type-aware encoder\u2013decoder network for multi-type ultrasound images co-segmentation. Comput. Methods Programs Biomed. 214, 106580 (2022).","journal-title":"Comput. Methods Programs Biomed."},{"key":"51495_CR22","doi-asserted-by":"crossref","unstructured":"Huang, Z. et al. DA-GAN: Learning structured noise removal in ultrasound volume projection imaging for enhanced spine segmentation. In Proceedings of the IEEE International Symposium on Biomedical Imaging (ISBI), USA, 770\u2013774 (2021).","DOI":"10.1109\/ISBI48211.2021.9434136"},{"key":"51495_CR23","doi-asserted-by":"publisher","first-page":"1610","DOI":"10.1109\/TMI.2022.3143953","volume":"41","author":"Z Huang","year":"2022","unstructured":"Huang, Z. et al. Joint spine segmentation and noise removal from ultrasound volume projection images with selective feature sharing. IEEE Trans. Med. Imaging 41, 1610\u20131624 (2022).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"51495_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/j.compmedimag.2021.101896","volume":"89","author":"J Lyu","year":"2021","unstructured":"Lyu, J. et al. Dual-task ultrasound spine transverse vertebrae segmentation network with contour regularization. Comput. Med. Imaging Graph. 89, 101896 (2021).","journal-title":"Comput. Med. Imaging Graph."},{"key":"51495_CR25","doi-asserted-by":"publisher","first-page":"3446","DOI":"10.1109\/TMI.2021.3087857","volume":"40","author":"R Liu","year":"2021","unstructured":"Liu, R. et al. NHBS-Net: A feature fusion attention network for ultrasound neonatal hip bone segmentation. IEEE Trans. Med. Imaging 40, 3446\u20133458 (2021).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"51495_CR26","unstructured":"Hayat, M. Endoscopic image super-resolution algorithm using edge and disparity awareness Chulalongkorn University Theses and Dissertations (Chula ETD). 11935. https:\/\/digital.car.chula.ac.th\/chulaetd\/11935 (2023)."},{"key":"51495_CR27","doi-asserted-by":"publisher","first-page":"196654","DOI":"10.1109\/ACCESS.2025.3634156","volume":"13","author":"M Hayat","year":"2025","unstructured":"Hayat, M. & Aramvith, S. Superpixel-guided graph-attention boundary GAN for adaptive feature refinement in scribble-supervised medical image segmentation. IEEE Access 13, 196654\u2013196668. https:\/\/doi.org\/10.1109\/ACCESS.2025.3634156 (2025).","journal-title":"IEEE Access"},{"key":"51495_CR28","doi-asserted-by":"publisher","unstructured":"Huang, S. C., Shen, L., Lungren, M. P. & Yeung, S. GLoRIA: A Multimodal Global-Local Representation Learning Framework for Label-efficient Medical Image Recognition, 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, pp. 3922\u20133931, https:\/\/doi.org\/10.1109\/ICCV48922.2021.00391 (2021).","DOI":"10.1109\/ICCV48922.2021.00391"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-51495-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-51495-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-51495-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T10:36:08Z","timestamp":1779878168000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-51495-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,7]]},"references-count":28,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["51495"],"URL":"https:\/\/doi.org\/10.1038\/s41598-026-51495-x","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,7]]},"assertion":[{"value":"5 April 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 April 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 May 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"The authors confirm that all methods were carried out in accordance with relevant guidelines and regulations. The authors confirm that all experimental protocols were approved by the Ethics Committee of Sir Run Run Shaw Hospital, Zhejiang University School of Medicine (Approval No. 2025\u2009\u2212\u20090327). The approved project corresponds to the clinical study from which the data used in this work were derived. The authors confirm that informed consent was obtained from all subjects and\/or their legal guardians.The lumbar spine X-ray and CT datasets used in this study were obtained from clinical practice and approved under the above ethical protocol.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}],"article-number":"16428"}}