{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,7]],"date-time":"2026-11-07T04:02:07Z","timestamp":1794024127826,"version":"4.8.0"},"reference-count":55,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2024,11,23]],"date-time":"2024-11-23T00:00:00Z","timestamp":1732320000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background: Lumbar spinal stenosis (LSS) is a major cause of chronic lower back and leg pain, and is traditionally diagnosed through labor-intensive analysis of magnetic resonance imaging (MRI) scans by radiologists. This study aims to streamline the diagnostic process by developing an automated radiology report generation (ARRG) system using a vision\u2013language (VL) model. Methods: We utilized a Generative Image-to-Text (GIT) model, originally designed for visual question answering (VQA) and image captioning. The model was fine-tuned to generate diagnostic reports directly from lumbar spine MRI scans using a modest set of annotated data. Additionally, GPT-4 was used to convert semistructured text into coherent paragraphs for better comprehension by the GIT model. Results: The model effectively generated semantically accurate and grammatically coherent reports. The performance was evaluated using METEOR (0.37), BERTScore (0.886), and ROUGE-L (0.3), indicating its potential to produce clinically relevant content. Conclusions: This study highlights the feasibility of using vision\u2013language models to automate report generation from medical imaging, potentially reducing the diagnostic workload for radiologists.<\/jats:p>","DOI":"10.3390\/jcm13237092","type":"journal-article","created":{"date-parts":[[2024,11,25]],"date-time":"2024-11-25T09:59:34Z","timestamp":1732528774000},"page":"7092","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Auto-Rad: End-to-End Report Generation from Lumber Spine MRI Using Vision\u2013Language Model"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0579-3030","authenticated-orcid":false,"given":"Mohammed","family":"Yeasin","sequence":"first","affiliation":[{"name":"Department of EECE, The University of Memphis, Memphis, TN 38152, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8742-2019","authenticated-orcid":false,"given":"Kazi Ashraf","family":"Moinuddin","sequence":"additional","affiliation":[{"name":"Department of EECE, The University of Memphis, Memphis, TN 38152, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Havugimana","sequence":"additional","affiliation":[{"name":"Department of EECE, The University of Memphis, Memphis, TN 38152, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lijia","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of EECE, The University of Memphis, Memphis, TN 38152, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, College of Medicine, The University of Tennessee Health Sciences, Memphis, TN 38163, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S1047-9651(02)00063-3","article-title":"Lumbar spinal stenosis: Anatomy and pathogenesis","volume":"14","author":"Botwin","year":"2003","journal-title":"Phys. Med. Rehabil. Clin. N. Am."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.berh.2009.11.001","article-title":"Lumbar Spinal Stenosis. Back Pain and Non-Inflammatory Spinal Disorders","volume":"24","author":"Genevay","year":"2010","journal-title":"Best Pract. Res. Clin. Rheumatol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.spinee.2013.08.003","article-title":"An evidence-based clinical guideline for the diagnosis and treatment of lumbar disc herniation with radiculopathy","volume":"14","author":"Kreiner","year":"2013","journal-title":"Spine J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"h6234","DOI":"10.1136\/bmj.h6234","article-title":"Management of lumbar spinal stenosis","volume":"352","author":"Lurie","year":"2016","journal-title":"BMJ"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"101878","DOI":"10.1016\/j.artmed.2020.101878","article-title":"Deep learning in generating radiology reports: A survey","volume":"106","author":"Monshi","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_6","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"194","DOI":"10.4103\/jmss.JMSS_21_20","article-title":"Automatic Generation of Structured Radiology Reports for Volumetric Computed Tomography Images Using Question-Specific Deep Feature Extraction and Learning","volume":"11","author":"Mansoorizadeh","year":"2021","journal-title":"J. Med. Signals Sens."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-L\u00f3pez, C., and Fichtinger, G. (2018, January 16\u201320). Towards Automatic Report Generation in Spine Radiology Using Weakly Supervised Framework. Proceedings of the Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2018, Granada, Spain.","DOI":"10.1007\/978-3-030-00931-1"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., and Chen, T. (2021). Big Self-Supervised Models Advance Medical Image Classification. arXiv.","DOI":"10.1109\/ICCV48922.2021.00346"},{"key":"ref_10","unstructured":"Wang, J., Yang, Z., Hu, X., Li, L., Lin, K., Gan, Z., Liu, Z., Liu, C., and Wang, L. (2022). GIT: A Generative Image-to-text Transformer for Vision and Language. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1148\/radiol.220076","article-title":"Improved Productivity Using Deep Learning-assisted Reporting for Lumbar Spine MRI","volume":"305","author":"Lim","year":"2022","journal-title":"Radiology"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1148\/radiol.2021204289","article-title":"Deep Learning Model for Automated Detection and Classification of Central Canal, Lateral Recess, and Neural Foraminal Stenosis at Lumbar Spine MRI","volume":"300","author":"Hallinan","year":"2021","journal-title":"Radiology"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"104","DOI":"10.3171\/2021.11.SPINE211136","article-title":"Diagnostic triage in patients with central lumbar spinal stenosis using a deep learning system of radiographs","volume":"37","author":"Kim","year":"2022","journal-title":"J. Neurosurg. Spine"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Andrew, J., Divya Varshini, M., Barjo, P., and Tigga, I. (2020, January 6\u20137). Spine Magnetic Resonance Image Segmentation Using Deep Learning Techniques. Proceedings of the 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India.","DOI":"10.1109\/ICACCS48705.2020.9074218"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ahmed, M.K., and Yeasin, M. (2024). MU-Net: Modified U-Net for Precise Localization and Segmentation of Lumber-Spine Regions from Sagittal Views. Authorea, preprints.","DOI":"10.36227\/techrxiv.171441712.29297338\/v1"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1007\/s10278-018-0130-7","article-title":"Automatic Lumbar MRI Detection and Identification Based on Deep Learning","volume":"32","author":"Zhou","year":"2019","journal-title":"J. Digit. Imaging"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Tanwani, A., Barral, J., and Freedman, D. (2022, January 18\u201322). RepsNet: Combining Vision with Language for Automated Medical Reports. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Singapore.","DOI":"10.1007\/978-3-031-16443-9_68"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kaur, N., and Mittal, A. (2022). RadioBERT: A deep learning-based system for medical report generation from chest X-ray images using contextual embeddings. J. Biomed. Inform., 128.","DOI":"10.1016\/j.jbi.2022.104220"},{"key":"ref_19","first-page":"3790","article-title":"Multi-Modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training","volume":"25","author":"Moon","year":"2021","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xue, Y., Xu, T., Long, L.R., Xue, Z., Antani, S.K., Thoma, G., and Huang, X. (2018, January 16\u201320). Multimodal Recurrent Model with Attention for Automated Radiology Report Generation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.","DOI":"10.1007\/978-3-030-00928-1_52"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1569","DOI":"10.5829\/IJE.2023.36.08B.16","article-title":"Cross-modal Deep Learning-based Clinical Recommendation System for Radiology Report Generation from Chest X-rays","volume":"36","author":"Shetty","year":"2023","journal-title":"Int. J. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yuan, J., Liao, H., Luo, R., and Luo, J. (2019, January 13\u201317). Automatic Radiology Report Generation based on Multi-view Image Fusion and Medical Concept Enrichment. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Shenzhen, China.","DOI":"10.1007\/978-3-030-32226-7_80"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, Z., Tang, M., Wang, L., Li, X., and Zhou, L. (2022, January 18\u201322). A Medical Semantic-Assisted Transformer for Radiographic Report Generation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Singapore.","DOI":"10.1007\/978-3-031-16437-8_63"},{"key":"ref_24","unstructured":"Sudirman, S., Al Kafri, A., Natalia, F., Meidia, H., Afriliana, N., Al-Rashdan, W., Bashtawi, M., and Al-Jumaily, M. (2023, September 01). Lumbar Spine MRI Dataset. Mendeley Data, 2019. Available online: https:\/\/doi.org\/10.17632\/k57fr854j2.2."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"43487","DOI":"10.1109\/ACCESS.2019.2908002","article-title":"Boundary Delineation of MRI Images for Lumbar Spinal Stenosis Detection Through Semantic Segmentation Using Deep Neural Networks","volume":"7","author":"Sudirman","year":"2019","journal-title":"IEEE Access"},{"key":"ref_26","unstructured":"OpenAI (2023, July 03). ChatGPT (June 22). 2023. Available online: https:\/\/chat.openai.com\/chat."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1566","DOI":"10.1198\/016214506000000302","article-title":"Hierarchical Dirichlet Processes","volume":"101","author":"Teh","year":"2006","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_28","unstructured":"Hughes, M.C., Stephenson, W., and Sudderth, E.B. (2015, January 7\u201312). Scalable Adaptation of State Complexity for Nonparametric Hidden Markov Models. Proceedings of the 28th International Conference on Neural Information Processing Systems\u2014Volume 1, NIPS\u201915, Montreal, QC, Canada."},{"key":"ref_29","unstructured":"Hughes, M.C., and Sudderth, E.B. (2013, January 5\u201310). Memoized Online Variational Inference for Dirichlet Process Mixture Models. Proceedings of the 26th International Conference on Neural Information Processing Systems\u2014Volume 1, NIPS\u201913, Lake Tahoe, CA, USA."},{"key":"ref_30","unstructured":"(2023, July 12). Hugging Face GIT Implementation. Available online: https:\/\/huggingface.co\/microsoft\/git-base."},{"key":"ref_31","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021). Learning Transferable Visual Models From Natural Language Supervision. arXiv."},{"key":"ref_32","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2023). Attention Is All You Need. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1162\/neco.1989.1.2.270","article-title":"A Learning Algorithm for Continually Running Fully Recurrent Neural Networks","volume":"1","author":"Williams","year":"1989","journal-title":"Neural Comput."},{"key":"ref_34","unstructured":"Loshchilov, I., and Hutter, F. (2019). Decoupled Weight Decay Regularization. arXiv."},{"key":"ref_35","unstructured":"Lin, C.Y. (2004, January 25\u201326). ROUGE: A Package for Automatic Evaluation of Summaries. Proceedings of the Text Summarization Branches Out, Barcelona, Spain."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6\u201312). Bleu: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_37","unstructured":"Banerjee, S., and Lavie, A. (2005, January 29). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, Ann Arbor, MI, USA."},{"key":"ref_38","unstructured":"Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (2019). BERTScore: Evaluating Text Generation with BERT. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"S63","DOI":"10.1121\/1.2016299","article-title":"Perplexity\u2014A measure of the difficulty of speech recognition tasks","volume":"62","author":"Jelinek","year":"1977","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Vedantam, R., Lawrence Zitnick, C., and Parikh, D. (2015, January 7\u201312). CIDEr: Consensus-Based Image Description Evaluation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Anderson, P., Fernando, B., Johnson, M., and Gould, S. (2016, January 11\u201314). SPICE: Semantic Propositional Image Caption Evaluation. Proceedings of the ECCV, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46454-1_24"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1023\/A:1026543900054","article-title":"The Earth Mover\u2019s Distance as a Metric for Image Retrieval","volume":"40","author":"Rubner","year":"2000","journal-title":"Int. J. Comput. Vis."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On Information and Sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Xie, Y., Xing, F., McGough, M., and Yang, L. (2017, January 21\u201326). Mdnet: A semantically and visually interpretable medical image diagnosis network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.378"},{"key":"ref_46","unstructured":"Harzig, P., Chen, Y., Chen, F., and Lienhart, R. (2019). Addressing Data Bias Problems for Chest X-ray Image Report Generation. arXiv."},{"key":"ref_47","unstructured":"Liu, G., Hsu, T.H., McDermott, M.B.A., Boag, W., Weng, W., Szolovits, P., and Ghassemi, M. (2019). Clinically Accurate Chest X-Ray Report Generation. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"100557","DOI":"10.1016\/j.imu.2021.100557","article-title":"Automated radiology report generation using conditioned transformers","volume":"24","author":"Alfarghaly","year":"2021","journal-title":"Inform. Med. Unlocked"},{"key":"ref_49","unstructured":"Xu, W., Qi, C., Xu, Z., and Lukasiewicz, T. (2020). Reinforced Medical Report Generation with X-Linear Attention and Repetition Penalty. arXiv."},{"key":"ref_50","unstructured":"Ziegler, D.M., Stiennon, N., Wu, J., Brown, T.B., Radford, A., Amodei, D., Christiano, P.F., and Irving, G. (2019). Fine-Tuning Language Models from Human Preferences. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"E469","DOI":"10.1097\/BRS.0b013e31828935ac","article-title":"Diagnosis of Lumbar Spinal Stenosis: An Updated Systematic Review of the Accuracy of Diagnostic Tests","volume":"38","author":"Overdevest","year":"2013","journal-title":"Spine"},{"key":"ref_52","first-page":"1233","article-title":"Lumbar Degenerative Disease: Key for Diagnosis","volume":"49 6","author":"Yasuhara","year":"2021","journal-title":"No Shinkei Geka"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1097\/00000542-200703000-00024","article-title":"Pathogenesis, Diagnosis, and Treatment of Lumbar Zygapophysial (Facet) Joint Pain","volume":"106","author":"Cohen","year":"2007","journal-title":"Anesthesiology"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1097\/BRS.0000000000004903","article-title":"Development of a Deep-Learning Model for Diagnosing Lumbar Spinal Stenosis Based on CT Images","volume":"49","author":"Li","year":"2023","journal-title":"Spine"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Prisilla, A.A., Guo, Y.L., Jan, Y., Lin, C.Y., Lin, F.Y., Liau, B.Y., Tsai, J.Y., Ardhianto, P., Pusparani, Y., and Lung, C.W. (2023). An approach to the diagnosis of lumbar disc herniation using deep learning models. Front. Bioeng. Biotechnol., 11.","DOI":"10.3389\/fbioe.2023.1247112"}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/13\/23\/7092\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:38:15Z","timestamp":1760114295000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/13\/23\/7092"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,23]]},"references-count":55,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["jcm13237092"],"URL":"https:\/\/doi.org\/10.3390\/jcm13237092","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,23]]}}}