{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,12]],"date-time":"2026-09-12T05:06:02Z","timestamp":1789189562233,"version":"build-2803163510"},"reference-count":36,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T00:00:00Z","timestamp":1788998400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Early and accurate cancer detection from medical imaging remains challenging because clinically relevant evidence is distributed across local image appearance, structural relationships between suspicious regions, and ordered imaging context. This study proposes a graph-aware and sequence-aware deep learning framework that combines a convolutional neural network (CNN) backbone for spatial feature extraction, a Graph Attention Network (GAT) for lesion-structure modelling, and a Bidirectional Long Short-Term Memory (BiLSTM) module for ordered-view or slice-sequence representation learning. Cross-attention-based multimodal fusion is evaluated exclusively for the RSNA mammography task, where the metadata branch is restricted to patient age and implant status, both available before diagnosis. In contrast, the primary LIDC-IDRI experiment is conducted as an image-only analysis because radiologist malignancy scores and semantic nodule attributes are annotation-derived variables and are not treated as independent clinical predictors. The framework is evaluated on the RSNA Breast Cancer Detection dataset and the LIDC-IDRI lung CT dataset using accuracy, precision, recall, F1-score, specificity, and area under the receiver operating characteristic curve (AUC). Additional ablation experiments assess the contribution of graph learning, sequence-aware modelling, and leakage-safe RSNA metadata fusion, while SHAP analysis quantifies the influence of the included RSNA metadata variables on multimodal predictions. For the RSNA multimodal experiment, the best held-out run achieved 95.2% accuracy and an AUC of 0.978, while three repeated runs yielded 94.0 \u00b1 0.3% accuracy and an AUC of 0.970 \u00b1 0.007 under the internal patient-wise benchmark protocol. These results should be interpreted as public-dataset benchmark outcomes rather than evidence of real-world clinical performance; external multi-centre and prospective validation is required before clinical deployment.<\/jats:p>","DOI":"10.3390\/jimaging12090431","type":"journal-article","created":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T13:04:57Z","timestamp":1789045497000},"page":"431","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Graph-Aware and Sequence-Aware Multimodal Deep Learning Framework for Cancer Detection and Risk Analysis from Medical Imaging"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6246-444X","authenticated-orcid":false,"given":"Chetanpal","family":"Singh","sequence":"first","affiliation":[{"name":"School of Engineering and Technology, Central Queensland University, Melbourne 3000, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5318-8428","authenticated-orcid":false,"given":"Santoso","family":"Wibowo","sequence":"additional","affiliation":[{"name":"School of Engineering and Technology, Central Queensland University, Melbourne 3000, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Srimannarayana","family":"Grandhi","sequence":"additional","affiliation":[{"name":"School of Engineering and Technology, Central Queensland University, Melbourne 3000, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6997-5875","authenticated-orcid":false,"given":"Satria","family":"Mandala","sequence":"additional","affiliation":[{"name":"Human Centric Engineering, Telkom University, Bandung 40257, Indonesia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_3","unstructured":"Jetley, S., Lord, N.A., Lee, N., and Torr, P.H.S. (May, January 30). Learn to pay attention. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.media.2019.01.012","article-title":"Attention gated networks: Learning to leverage salient regions in medical images","volume":"53","author":"Schlemper","year":"2019","journal-title":"Med. Image Anal."},{"key":"ref_5","unstructured":"Kipf, T.N., and Welling, M. (2017, January 24\u201326). Semi-supervised classification with graph convolutional networks. Proceedings of the International Conference on Learning Representations (ICLR), Toulon, France."},{"key":"ref_6","unstructured":"Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., and Bengio, Y. (May, January 30). Graph attention networks. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_8","unstructured":"Graves, A., and Schmidhuber, J. (August, January 31). Framewise phoneme classification with bidirectional LSTM networks. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Montreal, QC, Canada."},{"key":"ref_9","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","article-title":"Multimodal machine learning: A survey and taxonomy","volume":"41","author":"Baltrusaitis","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA."},{"key":"ref_12","unstructured":"Tan, M., and Le, Q. (2019, January 9\u201315). EfficientNet: Rethinking model scaling for convolutional neural networks. Proceedings of the 36th International Conference on Machine Learning (ICML), Long Beach, CA, USA."},{"key":"ref_13","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2021, January 3\u20137). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. Proceedings of the International Conference on Learning Representations (ICLR), Vienna, Austria."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin Transformer: Hierarchical vision transformer using shifted windows. Proceedings of the International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_15","first-page":"102716","article-title":"Swin UNETR: Swin Transformers for semantic segmentation of brain tumours in MRI images","volume":"84","author":"Hatamizadeh","year":"2023","journal-title":"Med. Image Anal."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1118\/1.3528204","article-title":"The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI)","volume":"38","author":"Armato","year":"2011","journal-title":"Med. Physic."},{"key":"ref_17","unstructured":"RSNA (2026, June 25). RSNA Screening Mammography Breast Cancer Detection AI Challenge Dataset. Kaggle. Available online: https:\/\/www.kaggle.com\/competitions\/rsna-breast-cancer-detection."},{"key":"ref_18","unstructured":"(2026, June 26). LIDC-IDRI Dataset. Lung Image Database Consortium and Image Database Resource Initiative. Available online: https:\/\/www.cancerimagingarchive.net\/collection\/lidc-idri\/."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_20","unstructured":"Lundberg, S.M., and Lee, S.I. (2017, January 4\u20139). A unified approach to interpreting model predictions. Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Mienye, I.D., and Viriri, S. (2025). Graph neural networks in medical imaging: Methods, applications and future directions. Information, 16.","DOI":"10.3390\/info16121051"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"14885","DOI":"10.1007\/s00432-023-05464-w","article-title":"Graph neural network-based breast cancer diagnosis using clinically significant imaging features","volume":"149","author":"Chowa","year":"2023","journal-title":"J. Cancer Res. Clin. Oncol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1038\/s43856-023-00388-5","article-title":"Deep learning for the detection of benign and malignant pulmonary nodules in CT scans","volume":"3","author":"Hendrix","year":"2023","journal-title":"Commun. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1038\/s41746-023-00811-0","article-title":"Fusion of medical imaging and electronic health records using deep learning: A systematic review and implementation guidelines","volume":"6","author":"Huang","year":"2023","journal-title":"npj Digit. Med."},{"key":"ref_25","first-page":"362","article-title":"Pan-cancer integrative histology-genomic analysis via multimodal deep learning","volume":"5","author":"Chen","year":"2023","journal-title":"Nat. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e241447","DOI":"10.1148\/radiol.241447","article-title":"Performance of algorithms submitted in the 2023 RSNA Screening Mammography Breast Cancer Detection AI Challenge","volume":"315","author":"Chen","year":"2025","journal-title":"Radiology"},{"key":"ref_27","first-page":"100298","article-title":"A deep learning framework for lung nodule segmentation and lung cancer classification in CT imaging","volume":"5","author":"Crasta","year":"2024","journal-title":"Healthc. Anal."},{"key":"ref_28","first-page":"10408","article-title":"Deep learning innovations in the detection of lung cancer: A review of recent progress","volume":"17","author":"Camacho","year":"2025","journal-title":"Cogn. Comput."},{"key":"ref_29","first-page":"107924","article-title":"Graph-based representation learning for cancer diagnosis from multimodal medical data","volume":"170","author":"Li","year":"2024","journal-title":"Comput. Biol. Med."},{"key":"ref_30","first-page":"102748","article-title":"Multimodal transformer architectures for medical image classification and prognosis prediction: A review","volume":"148","author":"Zhou","year":"2024","journal-title":"Artif. Intell. Med."},{"key":"ref_31","first-page":"3564","article-title":"Explainable graph neural networks for tumour classification and localisation in medical imaging","volume":"28","author":"Wu","year":"2024","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_32","first-page":"123456","article-title":"Cross-attention multimodal fusion for cancer diagnosis using imaging and clinical information","volume":"245","author":"Zhang","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_33","first-page":"110962","article-title":"Temporal representation learning in medical imaging: Recent advances and future directions","volume":"158","author":"Liu","year":"2025","journal-title":"Pattern Recogn."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"102995","DOI":"10.1016\/j.inffus.2025.102995","article-title":"Vision-Language Models in medical image analysis: From simple fusion to general large models","volume":"118","author":"Li","year":"2025","journal-title":"Inform. Fusion"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"103819","DOI":"10.1016\/j.media.2025.103819","article-title":"Knowledge distillation and teacher-student learning in medical imaging: Comprehensive overview, pivotal role, and future directions","volume":"107","author":"Li","year":"2025","journal-title":"Med. Image Anal."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"8712","DOI":"10.1109\/JIOT.2025.3641888","article-title":"PHH-FL: Perceptual Hashing Hypernetwork Personalized Federated Learning for Heterogeneous Medical Image Analysis Tasks","volume":"13","author":"Sun","year":"2025","journal-title":"IEEE Internet Things J."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/9\/431\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,12]],"date-time":"2026-09-12T04:14:03Z","timestamp":1789186443000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/9\/431"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,10]]},"references-count":36,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["jimaging12090431"],"URL":"https:\/\/doi.org\/10.3390\/jimaging12090431","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,10]]}}}