{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T22:23:06Z","timestamp":1784326986547,"version":"3.55.0"},"reference-count":45,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2025,7,4]],"date-time":"2025-07-04T00:00:00Z","timestamp":1751587200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>One of the main issues with paragangliomas and pheochromocytomas is that these tumors have up to a 20% rate of metastatic disease, which cannot be reliably predicted. While machine learning models hold great promise for enhancing predictive accuracy, their often opaque nature limits trust and adoption in critical fields such as healthcare. Understanding the factors driving predictions is essential not only for validating their reliability but also for enabling their integration into clinical decision-making. In this paper, we propose an architecture that combines data mining, machine learning, and explainability techniques to improve predictions of metastatic disease in these types of cancer and enhance trust in the models. A wide variety of algorithms have been applied for the development of predictive models, with a focus on interpreting their outputs to support clinical insights. Our methodology involves a comprehensive preprocessing phase to prepare the data, followed by the application of classification algorithms. Explainability techniques were integrated to provide insights into the key factors driving predictions. Additionally, a feature selection process was performed to identify the most influential variables and explore how their inclusion affects model performance. The best-performing algorithm, Random Forest, achieved an accuracy of 96.3%, precision of 96.5%, and AUC of 0.963, among other metrics, combining strong predictive capability with explainability that fosters trust in clinical applications.<\/jats:p>","DOI":"10.3390\/s25134184","type":"journal-article","created":{"date-parts":[[2025,7,7]],"date-time":"2025-07-07T04:43:37Z","timestamp":1751863417000},"page":"4184","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Prediction of Metastasis in Paragangliomas and Pheochromocytomas Using Machine Learning Models: Explainability Challenges"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-8369-6027","authenticated-orcid":false,"given":"Carmen","family":"Garc\u00eda-Barcel\u00f3","sequence":"first","affiliation":[{"name":"University Institute for Computer Research, University of Alicante, Carretera San Vicente del Raspeig s\/n, 03690 San Vicente del Raspeig, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0791-8298","authenticated-orcid":false,"given":"David","family":"Gil","sequence":"additional","affiliation":[{"name":"University Institute for Computer Research, University of Alicante, Carretera San Vicente del Raspeig s\/n, 03690 San Vicente del Raspeig, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3287-9366","authenticated-orcid":false,"given":"David","family":"Tom\u00e1s","sequence":"additional","affiliation":[{"name":"University Institute for Computer Research, University of Alicante, Carretera San Vicente del Raspeig s\/n, 03690 San Vicente del Raspeig, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-7907-3359","authenticated-orcid":false,"given":"David","family":"Bernabeu","sequence":"additional","affiliation":[{"name":"University Institute for Computer Research, University of Alicante, Carretera San Vicente del Raspeig s\/n, 03690 San Vicente del Raspeig, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,4]]},"reference":[{"key":"ref_1","unstructured":"National Cancer Institute (2024, April 04). Metastasis. NCI Dictionary of Cancer Terms, Available online: https:\/\/www.cancer.gov\/publications\/dictionaries\/cancer-terms\/def\/metastasis."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/S0092-8674(00)81683-9","article-title":"The hallmarks of cancer","volume":"100","author":"Hanahan","year":"2000","journal-title":"Cell"},{"key":"ref_3","unstructured":"National Cancer Institute (2024, April 12). Pheochromocytoma and Paraganglioma Treatment, Available online: https:\/\/www.cancer.gov\/espanol\/tipos\/feocromocitoma."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"e551","DOI":"10.1016\/S2589-7500(23)00094-8","article-title":"Prediction of metastatic pheochromocytoma and paraganglioma: A machine learning modelling study using data from a cross-sectional cohort","volume":"5","author":"Pamporaki","year":"2023","journal-title":"Lancet Digit. Health"},{"key":"ref_5","unstructured":"Miranda-Aguirre, A.P., Hern\u00e1ndez-Garc\u00eda, S., Tenorio-Torres, J.A., Farias-Alarc\u00f3n, M., and Erazo-Valle, S.A. (2012). Experiencia en el manejo de los tumores neuroendocrinos en el Servicio de Oncolog\u00eda del Centro M\u00e9dico Nacional \u201c20 de noviembre\u201d, ISSSTE, en los \u00faltimos 10 a\u00f1os. Clin. Case, 11."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1056\/NEJMra1806651","article-title":"Pheochromocytoma and Paraganglioma","volume":"381","author":"Neumann","year":"2019","journal-title":"N. Engl. J. Med."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Al Subhi, A.R., Boyle, V., and Elston, M.S. (2022). Systematic review: Incidence of pheochromocytoma and paraganglioma over 70 years. J. Endocr. Soc., 6.","DOI":"10.1210\/jendso\/bvac105"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4875","DOI":"10.1007\/s11831-021-09556-z","article-title":"A systematic review of applications of machine learning in cancer prediction and diagnosis","volume":"28","author":"Sharma","year":"2021","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1607","DOI":"10.1007\/s13347-021-00477-0","article-title":"Transparency and the black box problem: Why we do not trust AI","volume":"34","year":"2021","journal-title":"Philos. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1429","DOI":"10.1109\/TAI.2023.3266418","article-title":"A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?","volume":"5","author":"Bharati","year":"2024","journal-title":"IEEE Trans. Artif. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1038\/s41571-019-0252-y","article-title":"Artificial intelligence in digital pathology\u2014New tools for diagnosis and precision oncology","volume":"16","author":"Bera","year":"2019","journal-title":"Nat. Rev. Clin. Oncol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1753-6561-8-S6-S2","article-title":"Classification models for clear cell renal carcinoma stage progression, based on tumor RNAseq expression trained supervised machine learning algorithms","volume":"8","author":"Jagga","year":"2014","journal-title":"BMC Proc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1038\/nrg2484","article-title":"RNA-Seq: A revolutionary tool for transcriptomics","volume":"10","author":"Wang","year":"2009","journal-title":"Nat. Rev. Genet."},{"key":"ref_14","unstructured":"National Cancer Institute (2025, February 12). The Cancer Genome Atlas (TCGA), Available online: https:\/\/www.cancer.gov\/about-nci\/organization\/ccg\/research\/structural-genomics\/tcga."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1097\/00000478-200205000-00002","article-title":"Pheochromocytoma of the Adrenal gland Scaled Score (PASS) to separate benign from malignant neoplasms: A clinicopathologic and immunophenotypic study of 100 cases","volume":"26","author":"Thompson","year":"2002","journal-title":"Am. J. Surg. Pathol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1530\/ERC-13-0494","article-title":"Pathological grading for predicting metastasis in phaeochromocytoma and paraganglioma","volume":"21","author":"Kimura","year":"2014","journal-title":"Endocr.-Relat. Cancer"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1097\/PAS.0b013e318190d12e","article-title":"Observer variation in the application of the Pheochromocytoma of the Adrenal Gland Scaled Score","volume":"33","author":"Wu","year":"2009","journal-title":"Am. J. Surg. Pathol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Stenman, A., Zedenius, J., and Juhlin, C. (2019). The value of histological algorithms to predict the malignancy potential of pheochromocytomas and abdominal paragangliomas\u2014A meta-analysis and systematic review of the literature. Cancers, 11.","DOI":"10.3390\/cancers11020225"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1016\/j.surg.2018.05.001","article-title":"A clinical prediction model to estimate the metastatic potential of pheochromocytoma\/paraganglioma: ASES score","volume":"164","author":"Cho","year":"2018","journal-title":"Surgery"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1016\/j.aej.2024.08.056","article-title":"Machine learning approaches for advanced detection of rare genetic disorders in whole-genome sequencing","volume":"106","author":"Alzahrani","year":"2024","journal-title":"Alex. Eng. J."},{"key":"ref_21","unstructured":"SweGen (2025, February 15). SweGen\u2014Database Commons. Available online: https:\/\/ngdc.cncb.ac.cn\/databasecommons\/database\/id\/4976."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"717","DOI":"10.1210\/jc.2010-1946","article-title":"Clinical risk factors for malignancy and overall survival in patients with pheochromocytomas and sympathetic paragangliomas: Primary tumor size and primary tumor location as prognostic indicators","volume":"96","author":"Feng","year":"2011","journal-title":"J. Clin. Endocrinol. Metab."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1210\/jc.2016-3829","article-title":"Characteristics of pediatric vs adult pheochromocytomas and paragangliomas","volume":"102","author":"Pamporaki","year":"2017","journal-title":"J. Clin. Endocrinol. Metab."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1111\/j.1365-2362.2011.02518.x","article-title":"Metastatic pheochromocytoma: Does the size and age matter?","volume":"41","author":"Zelinka","year":"2011","journal-title":"Eur. J. Clin. Investig."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1739","DOI":"10.1016\/j.ejca.2011.07.016","article-title":"Plasma methoxytyramine: A novel biomarker of metastatic pheochromocytoma and paraganglioma in relation to established risk factors of tumour size, location and SDHB mutation status","volume":"48","author":"Eisenhofer","year":"2012","journal-title":"Eur. J. Cancer"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jhawar, S., Arakawa, Y., Kumar, S., Varghese, D., Kim, Y.S., Roper, N., and Del Rivero, J. (2022). New insights on the genetics of pheochromocytoma and paraganglioma and its clinical implications. Cancers, 14.","DOI":"10.3390\/cancers14030594"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4505","DOI":"10.1210\/jc.2006-0423","article-title":"High frequency of SDHB germline mutations in patients with malignant catecholamine-producing paragangliomas: Implications for genetic testing","volume":"91","author":"Brouwers","year":"2006","journal-title":"J. Clin. Endocrinol. Metab."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Allgaier, J., Mulansky, L., Draelos, R., and Pryss, R. (2023). How does the model make predictions? A systematic literature review on the explainability power of machine learning in healthcare. Artif. Intell. Med., 143.","DOI":"10.1016\/j.artmed.2023.102616"},{"key":"ref_29","unstructured":"Lundberg, S.M., and Lee, S.I. (2017, January 4\u20139). A Unified Approach to Interpreting Model Predictions. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach CA, Long Beach, CA, USA. Advances in Neural Information Processing Systems."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Alabi, R.O., Elmusrati, M., Leivo, I., Almangush, A., and M\u00e4kitie, A.A. (2023). Machine learning explainability in nasopharyngeal cancer survival using LIME and SHAP. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-35795-0"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy should I trust you?\u201d Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Jansen, T., Geleijnse, G., Van Maaren, M., Hendriks, M.P., Ten Teije, A., and Moncada-Torres, A. (2020). Machine learning explainability in breast cancer survival. Digital Personalized Health and Medicine, IOS Press.","DOI":"10.3233\/SHTI200172"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Pathan, R.K., Shorna, I.J., Hossain, M.S., Khandaker, M.U., Almohammed, H.I., and Hamd, Z.Y. (2024). The efficacy of machine learning models in lung cancer risk prediction with explainability. PLoS ONE, 19.","DOI":"10.1371\/journal.pone.0305035"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gilpin, L., Bau, D., Yuan, B., Bajwa, A., Specter, M., and Kagal, L. (2018, January 1\u20133). Explaining explanations: An overview of interpretability of machine learning. Proceedings of the 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), Turin, Italy.","DOI":"10.1109\/DSAA.2018.00018"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Linardatos, P., Papastefanopoulos, V., and Kotsiantis, S. (2020). Explainable AI: A review of machine learning interpretability methods. Entropy, 23.","DOI":"10.3390\/e23010018"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: A gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat."},{"key":"ref_38","unstructured":"Louppe, G., Wehenkel, L., Sutera, A., and Geurts, P. (2013, January 5\u201310). Understanding Variable Importances in Forests of Randomized Trees. Proceedings of the 27th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA. Advances in Neural Information Processing Systems."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1080\/10618600.2014.907095","article-title":"Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation","volume":"24","author":"Goldstein","year":"2015","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1613\/jair.1.12228","article-title":"A survey on the explainability of supervised machine learning","volume":"70","author":"Burkart","year":"2021","journal-title":"J. Artif. Intell. Res."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1258\/acb.2012.012112","article-title":"Analysis of plasma 3-methoxytyramine, normetanephrine and metanephrine by ultraperformance liquid chromatography-tandem mass spectrometry: Utility for diagnosis of dopamine-producing metastatic phaeochromocytoma","volume":"50","author":"Peitzsch","year":"2013","journal-title":"Ann. Clin. Biochem."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Snezhkina, A., Pavlov, V., Dmitriev, A., Melnikova, N., and Kudryavtseva, A. (2021). Potential Biomarkers of Metastasizing Paragangliomas and Pheochromocytomas. Life, 11.","DOI":"10.3390\/life11111179"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Martinelli, S., Amore, F., Canu, L., Maggi, M., and Rapizzi, E. (2023). Tumour microenvironment in pheochromocytoma and paraganglioma. Front. Endocrinol., 14.","DOI":"10.3389\/fendo.2023.1137456"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Liu, C., Zhou, D., Yang, K., Xu, N., Peng, J., and Zhu, Z. (2023). Research progress on the pathogenesis of the SDHB mutation and related diseases. Biomed. Pharmacother., 167.","DOI":"10.1016\/j.biopha.2023.115500"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1038\/s41574-023-00926-0","article-title":"Management of phaeochromocytoma and paraganglioma in patients with germline SDHB pathogenic variants: An international expert consensus statement","volume":"20","author":"Perrier","year":"2024","journal-title":"Nat. Rev. Endocrinol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/13\/4184\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:04:53Z","timestamp":1760033093000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/13\/4184"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,4]]},"references-count":45,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["s25134184"],"URL":"https:\/\/doi.org\/10.3390\/s25134184","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,4]]}}}