{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T21:08:50Z","timestamp":1777669730701,"version":"3.51.4"},"reference-count":45,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,11,21]],"date-time":"2024-11-21T00:00:00Z","timestamp":1732147200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Colorectal cancer is a major public health issue, causing significant morbidity and mortality worldwide. Treatment for colorectal cancer often has a significant impact on patients\u2019 quality of life, which can vary over time and across individuals. The application of artificial intelligence and machine learning techniques has great potential for optimizing patient outcomes by providing valuable insights. In this paper, we propose a multimodal machine learning framework for the prediction of quality of life indicators in colorectal cancer patients at various temporal stages, leveraging both clinical data and computed tomography scan images. Additionally, we identify key predictive factors for each quality of life indicator, thereby enabling clinicians to make more informed treatment decisions and ultimately enhance patient outcomes. Our approach integrates data from multiple sources, enhancing the performance of our predictive models. The analysis demonstrates a notable improvement in accuracy for some indicators, with results for the Wexner score increasing from 24% to 48% and for the Anorectal Ultrasound score from 88% to 96% after integrating data from different modalities. These results highlight the potential of multimodal learning to provide valuable insights and improve patient care in real-world applications.<\/jats:p>","DOI":"10.3390\/jimaging10120297","type":"journal-article","created":{"date-parts":[[2024,11,21]],"date-time":"2024-11-21T06:11:54Z","timestamp":1732169514000},"page":"297","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Multimodal Machine Learning for Predicting Post-Surgery Quality of Life in Colorectal Cancer Patients"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0147-8466","authenticated-orcid":false,"given":"Maryem","family":"Rhanoui","sequence":"first","affiliation":[{"name":"Laboratory Health Systemic Process (P2S), UR4129, University Claude Bernard Lyon 1, University of Lyon, 69008 Lyon, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6647-5098","authenticated-orcid":false,"given":"Mounia","family":"Mikram","sequence":"additional","affiliation":[{"name":"Meridian Team, LyRICA Laboratory, School of Information Sciences, Rabat 10100, Morocco"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-1005-1004","authenticated-orcid":false,"given":"Kamelia","family":"Amazian","sequence":"additional","affiliation":[{"name":"Higher Institute of Nursing Professions and Health Technology, Fez 30050, Morocco"},{"name":"Human Pathology, Biomedicine and Environment Laboratory, Faculty of Medicine and Pharmacy, Sidi Mohamed Ben Abdellah University, Fez 30000, Morocco"}]},{"given":"Abderrahim","family":"Ait-Abderrahim","sequence":"additional","affiliation":[{"name":"General Surgery Department, Hassan II University Hospital, Fez 30050, Morocco"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0745-8327","authenticated-orcid":false,"given":"Siham","family":"Yousfi","sequence":"additional","affiliation":[{"name":"Meridian Team, LyRICA Laboratory, School of Information Sciences, Rabat 10100, Morocco"}]},{"given":"Imane","family":"Toughrai","sequence":"additional","affiliation":[{"name":"General Surgery Department, Hassan II University Hospital, Fez 30050, Morocco"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"209","DOI":"10.3322\/caac.21660","article-title":"Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","year":"2021","journal-title":"CA A Cancer J. Clin."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2879","DOI":"10.1016\/j.ejca.2010.06.010","article-title":"Quality of life among long-term (\u2a7e5 years) colorectal cancer survivors\u2013systematic review","volume":"46","author":"Jansen","year":"2010","journal-title":"Eur. J. Cancer"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"84486","DOI":"10.1109\/ACCESS.2022.3197671","article-title":"Explainable AI for healthcare 5.0: Opportunities and challenges","volume":"10","author":"Saraswat","year":"2022","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chtouki, K., Rhanoui, M., Mikram, M., Yousfi, S., and Amazian, K. (2023, January 25\u201327). Supervised Machine Learning for Breast Cancer Risk Factors Analysis and Survival Prediction. Proceedings of the International Conference On Big Data and Internet of Things, Tangier, Morocco.","DOI":"10.1007\/978-3-031-28387-1_6"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Mikram, M., Moujahdi, C., Rhanoui, M., Meddad, M., and Khallout, A. (2022, January 17\u201318). Hybrid deep learning models for diabetic retinopathy classification. Proceedings of the International Conference On Big Data and Internet of Things, Rabat, Morocco.","DOI":"10.1007\/978-3-031-07969-6_13"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.ejmp.2021.05.003","article-title":"A review of deep learning based methods for medical image multi-organ segmentation","volume":"85","author":"Fu","year":"2021","journal-title":"Phys. Medica"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.radonc.2020.09.045","article-title":"Clinical evaluation of atlas-and deep learning-based automatic segmentation of multiple organs and clinical target volumes for breast cancer","volume":"153","author":"Choi","year":"2020","journal-title":"Radiother. Oncol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.media.2017.07.003","article-title":"Efficient and robust cell detection: A structured regression approach","volume":"44","author":"Xie","year":"2018","journal-title":"Med Image Anal."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3257","DOI":"10.1109\/TMI.2019.2927182","article-title":"Deep adversarial training for multi-organ nuclei segmentation in histopathology images","volume":"39","author":"Mahmood","year":"2019","journal-title":"IEEE Trans. Med Imaging"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.cmpb.2019.06.022","article-title":"Prediction of early colorectal cancer metastasis by machine learning using digital slide images","volume":"178","author":"Takamatsu","year":"2019","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bychkov, D., Linder, N., Turkki, R., Nordling, S., Kovanen, P.E., Verrill, C., Walliander, M., Lundin, M., Haglund, C., and Lundin, J. (2018). Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci. Rep., 8.","DOI":"10.1038\/s41598-018-21758-3"},{"key":"ref_13","first-page":"423","article-title":"Multimodal machine learning: A survey and taxonomy","volume":"41","author":"Ahuja","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"8294","DOI":"10.1007\/s11227-019-03101-3","article-title":"Multimodal deep learning for finance: Integrating and forecasting international stock markets","volume":"76","author":"Lee","year":"2020","journal-title":"J. Supercomput."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bousnina, N., Ghouzali, S., Mikram, M., and Abdul, W. (2019, January 1\u20133). DTCWT-DCT watermarking method for multimodal biometric authentication. Proceedings of the 2nd International Conference on Networking, Information Systems & Security, Riyadh, Saudi Arabia.","DOI":"10.1145\/3320326.3320409"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1773","DOI":"10.1038\/s41591-022-01981-2","article-title":"Multimodal biomedical AI","volume":"28","author":"Acosta","year":"2022","journal-title":"Nat. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1136\/gut.47.3.444","article-title":"Quality of life measurement in gastrointestinal and liver disorders","volume":"47","author":"Borgaonkar","year":"2000","journal-title":"Gut"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lewandowska, A., Rudzki, G., Lewandowski, T., Pr\u00f3chnicki, M., Rudzki, S., Laskowska, B., and Brudniak, J. (2020). Quality of life of cancer patients treated with chemotherapy. Int. J. Environ. Res. Public Health, 17.","DOI":"10.3390\/ijerph17196938"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1200\/JCO.2004.06.078","article-title":"Measuring quality of life in routine oncology practice improves communication and patient well-being: A randomized controlled trial","volume":"22","author":"Velikova","year":"2004","journal-title":"J. Clin. Oncol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1007\/s10350-006-0791-z","article-title":"Prospective evaluation of quality of life and sexual functioning after laparoscopic total mesorectal excision","volume":"50","author":"Breukink","year":"2007","journal-title":"Dis. Colon Rectum"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1697","DOI":"10.1038\/bjc.2011.155","article-title":"Modifiable and fixed factors predicting quality of life in people with colorectal cancer","volume":"104","author":"Gray","year":"2011","journal-title":"Br. J. Cancer"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Xu, Y., Ju, L., Tong, J., Zhou, C.M., and Yang, J.J. (2020). Machine learning algorithms for predicting the recurrence of stage IV colorectal cancer after tumor resection. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-59115-y"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Valeikaite-Tauginiene, G., Kraujelyte, A., Poskus, E., Jotautas, V., Saladzinskas, Z., Tamelis, A., Lizdenis, P., Dulskas, A., Samalavicius, N.E., and Strupas, K. (2022). Predictors of quality of life six years after curative colorectal cancer surgery: Results of the prospective multicenter study. Medicina, 58.","DOI":"10.3390\/medicina58040482"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e33182","DOI":"10.2196\/33182","article-title":"Machine Learning\u2013Based Short-Term Mortality Prediction Models for Patients With Cancer Using Electronic Health Record Data: Systematic Review and Critical Appraisal","volume":"10","author":"Lu","year":"2022","journal-title":"JMIR Med. Inform."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sim, J.A., Kim, Y., Kim, J.H., Lee, J.M., Kim, M.S., Shim, Y.M., Zo, J.I., and Yun, Y.H. (2020). The major effects of health-related quality of life on 5-year survival prediction among lung cancer survivors: Applications of machine learning. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-67604-3"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Savi\u0107, M., Kurbalija, V., Ili\u0107, M., Ivanovi\u0107, M., Jakoveti\u0107, D., Valachis, A., Autexier, S., Rust, J., and Kosmidis, T. (2021, January 1\u20133). Analysis of Machine Learning Models Predicting Quality of Life for Cancer Patients. Proceedings of the 13th International Conference on Management of Digital EcoSystems, Virtual Event.","DOI":"10.1145\/3444757.3485103"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Nuutinen, M., Korhonen, S., Hiltunen, A.M., Haavisto, I., Poikonen-Saksela, P., Mattson, J., Kondylakis, H., Mazzocco, K., Pat-Horenczyk, R., and Sousa, B. (2022, January 9\u201311). Impact of machine learning assistance on the quality of life prediction for breast cancer patients. Proceedings of the 15th International Joint Conference on Biomedical Engineering Systems and Technologies, Virtual Event.","DOI":"10.5220\/0010786900003123"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhou, C., Hu, J., Wang, Y., Ji, M.H., Tong, J., Yang, J.J., and Xia, H. (2021). A machine learning-based predictor for the identification of the recurrence of patients with gastric cancer after operation. Sci. Rep., 11.","DOI":"10.1038\/s41598-021-81188-6"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ting, W.C., Chang, H.R., Chang, C.C., and Lu, C.J. (2020). Developing a novel machine learning-based classification scheme for predicting SPCs in colorectal cancer survivors. Appl. Sci., 10.","DOI":"10.3389\/fgene.2019.00848"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Karri, R., Chen, Y.P.P., and Drummond, K.J. (2022). Using machine learning to predict health-related quality of life outcomes in patients with low grade glioma, meningioma, and acoustic neuroma. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0267931"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Trebeschi, S., van Griethuysen, J.J., Lambregts, D.M., Lahaye, M.J., Parmar, C., Bakers, F.C., Peters, N.H., Beets-Tan, R.G., and Aerts, H.J. (2017). Deep learning for fully-automated localization and segmentation of rectal cancer on multiparametric MR. Sci. Rep., 7.","DOI":"10.1038\/s41598-017-05728-9"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1883","DOI":"10.4249\/scholarpedia.1883","article-title":"K-nearest neighbor","volume":"4","author":"Peterson","year":"2009","journal-title":"Scholarpedia"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1007\/BF00175354","article-title":"A genetic algorithm tutorial","volume":"4","author":"Whitley","year":"1994","journal-title":"Stat. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yang, X.S. (2012, January 3\u20137). Flower pollination algorithm for global optimization. Proceedings of the International Conference on Unconventional Computing and Natural Computation, Orlean, France.","DOI":"10.1007\/978-3-642-32894-7_27"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1016\/j.knosys.2015.12.022","article-title":"SCA: A sine cosine algorithm for solving optimization problems","volume":"96","author":"Mirjalili","year":"2016","journal-title":"Knowl.-Based Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.advengsoft.2017.07.002","article-title":"Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems","volume":"114","author":"Mirjalili","year":"2017","journal-title":"Adv. Eng. Softw."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Schapire, R.E. (2013). Explaining adaboost. Empirical Inference: Festschrift in Honor of Vladimir N. Vapnik, Springer.","DOI":"10.1007\/978-3-642-41136-6_5"},{"key":"ref_38","unstructured":"Chen, T. (2015). Xgboost: Extreme gradient boosting. R Package Version 0.4-2, 1."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Friedman, J.H. (2001). Greedy function approximation: A gradient boosting machine. Ann. Stat., 1189\u20131232.","DOI":"10.1214\/aos\/1013203451"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., and Liang, J. (2018). Unet++: A nested u-net architecture for medical image segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-00889-5_1"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.isprsjprs.2020.01.013","article-title":"ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data","volume":"162","author":"Diakogiannis","year":"2020","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Jha, D., Smedsrud, P.H., Riegler, M.A., Johansen, D., De Lange, T., Halvorsen, P., and Johansen, H.D. (2019, January 9\u201311). Resunet++: An advanced architecture for medical image segmentation. Proceedings of the 2019 IEEE International Symposium on Multimedia (ISM), San Diego, CA, USA.","DOI":"10.1109\/ISM46123.2019.00049"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"102112","DOI":"10.1016\/j.artmed.2021.102112","article-title":"Improving clinical outcome predictions using convolution over medical entities with multimodal learning","volume":"117","author":"Bardak","year":"2021","journal-title":"Artif. Intell. Med."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Zekaoui, N.E., Yousfi, S., Mikram, M., and Rhanoui, M. (2023, January 22\u201323). Enhancing Large Language Models\u2019 Utility for Medical Question-Answering: A Patient Health Question Summarization Approach. Proceedings of the 2023 14th International Conference on Intelligent Systems: Theories and Applications (SITA), Casablanca, Morocco.","DOI":"10.1109\/SITA60746.2023.10373720"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"4793","DOI":"10.1109\/TNNLS.2020.3027314","article-title":"A survey on explainable artificial intelligence (xai): Toward medical xai","volume":"32","author":"Tjoa","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/12\/297\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:36:31Z","timestamp":1760114191000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/12\/297"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,21]]},"references-count":45,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["jimaging10120297"],"URL":"https:\/\/doi.org\/10.3390\/jimaging10120297","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,21]]}}}