{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,3]],"date-time":"2025-10-03T18:12:56Z","timestamp":1759515176358},"reference-count":76,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,3,18]],"date-time":"2023-03-18T00:00:00Z","timestamp":1679097600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,18]],"date-time":"2023-03-18T00:00:00Z","timestamp":1679097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The self-proclaimed first publicly available dataset of Monkeypox skin images consists of medically irrelevant images extracted from Google and photography repositories through a process denominated web-scrapping. Yet, this did not stop other researchers from employing it to build Machine Learning (ML) solutions aimed at computer-aided diagnosis of Monkeypox and other viral infections presenting skin lesions. Neither did it stop the reviewers or editors from publishing these subsequent works in peer-reviewed journals. Several of these works claimed extraordinary performance in the classification of Monkeypox, Chickenpox and Measles, employing ML and the aforementioned dataset. In this work, we analyse the initiator work that has catalysed the development of several ML solutions, and whose popularity is continuing to grow. Further, we provide a rebuttal experiment that showcases the risks of such methodologies, proving that the ML solutions do not necessarily obtain their performance from the features relevant to the diseases at issue.<\/jats:p>","DOI":"10.1007\/s10916-023-01928-1","type":"journal-article","created":{"date-parts":[[2023,3,18]],"date-time":"2023-03-18T06:02:47Z","timestamp":1679119367000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Analysis: Flawed Datasets of Monkeypox Skin Images"],"prefix":"10.1007","volume":"47","author":[{"given":"Carlos","family":"Vega","sequence":"first","affiliation":[]},{"given":"Reinhard","family":"Schneider","sequence":"additional","affiliation":[]},{"given":"Venkata","family":"Satagopam","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,3,18]]},"reference":[{"key":"1928_CR1","unstructured":"Ahsan, M.M., Uddin, M.R., Farjana, M., Sakib, A.N., Momin, K.A., Luna, S.A.: Image data collection and implementation of deep learning-based model in detecting monkeypox disease using modified vgg16. arXiv preprint arXiv:2206.01862 (2022)"},{"key":"1928_CR2","unstructured":"Ahsan, M.M., Uddin, M.R., Luna, S.A.: Monkeypox image data collection. arXiv preprint arXiv:2206.01774 (2022)"},{"key":"1928_CR3","doi-asserted-by":"crossref","unstructured":"Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., Aroyo, L.M.: \u201deveryone wants to do the model work, not the data work\u201d: Data cascades in high-stakes ai. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1\u201315 (2021)","DOI":"10.1145\/3411764.3445518"},{"key":"1928_CR4","doi-asserted-by":"crossref","unstructured":"Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A.I., Etmann, C., McCague, C., Beer, L., et al: Common pitfalls and recommendations for using machine learning to detect and prognosticate for covid-19 using chest radiographs and ct scans. Nature Machine Intelligence 3(3), 199\u2013217 (2021)","DOI":"10.1038\/s42256-021-00307-0"},{"key":"1928_CR5","doi-asserted-by":"crossref","unstructured":"Williams, C.M., Chaturvedi, R., Urman, R.D., Waterman, R.S., Gabriel, R.A.: Artificial intelligence and a pandemic: An analysis of the potential uses and drawbacks. Springer (2021)","DOI":"10.1007\/s10916-021-01705-y"},{"key":"1928_CR6","doi-asserted-by":"crossref","unstructured":"L\u00f3pez-Cabrera, J.D., Orozco-Morales, R., Portal-Diaz, J.A., Lovelle-Enr\u00edquez, O., P\u00e9rez-D\u00edaz, M.: Current limitations to identify covid-19 using artificial intelligence with chest x-ray imaging. Health and Technology 11(2), 411\u2013424 (2021)","DOI":"10.1007\/s12553-021-00520-2"},{"key":"1928_CR7","doi-asserted-by":"crossref","unstructured":"Santa\u00a0Cruz, B.G., Bossa, M.N., S\u00f6lter, J., Husch, A.D.: Public covid-19 x-ray datasets and their impact on model bias\u2013a systematic review of a significant problem. Medical image analysis 74, 102225 (2021)","DOI":"10.1016\/j.media.2021.102225"},{"key":"1928_CR8","doi-asserted-by":"crossref","unstructured":"Bullock, J., Luccioni, A., Pham, K.H., Lam, C.S.N., Luengo-Oroz, M.: Mapping the landscape of artificial intelligence applications against covid-19. Journal of Artificial Intelligence Research 69, 807\u2013845 (2020)","DOI":"10.1613\/jair.1.12162"},{"key":"1928_CR9","unstructured":"Jain, N., Tanasov, A., Chodnekar, S.Y., Rakauskait\u0117, A., Lansiaux, E., Skuja, S., Reinis, A.: Quantitative bibliometric excellence & productivity in monkeypox (mpox) literature"},{"key":"1928_CR10","doi-asserted-by":"crossref","unstructured":"Maguolo, G., Nanni, L.: A critic evaluation of methods for covid-19 automatic detection from x-ray images. Information Fusion 76, 1\u20137 (2021)","DOI":"10.1016\/j.inffus.2021.04.008"},{"key":"1928_CR11","unstructured":"Cohen, J.P.: Github COVID-19 image data collection (2020). https:\/\/github.com\/ieee8023\/covid-chestxray-dataset"},{"key":"1928_CR12","unstructured":"Manjurul, A.M.: Monkeypox dataset 2022. Github (2022). https:\/\/github.com\/mahsan2\/Monkeypox-dataset-2022\/tree\/18145e9feee4993d2bab83430b583a3e981398e4"},{"key":"1928_CR13","unstructured":"Cohen, J.P.: Research Protocol - COVID-19 image data collection (2020). https:\/\/docs.google.com\/document\/d\/14b7cou98YhYcJ2jwOKznChtn5y6-mi9bgjeFv2DxOt0"},{"key":"1928_CR14","doi-asserted-by":"crossref","unstructured":"Tartaglione, E., Barbano, C.A., Berzovini, C., Calandri, M., Grangetto, M.: Unveiling covid-19 from chest x-ray with deep learning: a hurdles race with small data. International Journal of Environmental Research and Public Health 17(18), 6933 (2020)","DOI":"10.3390\/ijerph17186933"},{"key":"1928_CR15","unstructured":"Web Archive Link for Getty image example: https:\/\/web.archive.org\/web\/20230208100600\/https:\/\/raw.githubusercontent.com\/mahsan2\/Monkeypox-dataset-2022\/18145e9feee4993d2bab83430b583a3e981398e4\/Chickenpox_gray\/chicken12.jpg. https:\/\/github.com\/mahsan2\/Monkeypox-dataset-2022\/blob\/18145e9feee4993d2bab83430b583a3e981398e4\/Chickenpox_gray\/chicken12.jpg"},{"key":"1928_CR16","unstructured":"Web Archive Link for Shutterstock image example: https:\/\/web.archive.org\/web\/20230208101648\/https:\/\/raw.githubusercontent.com\/mahsan2\/Monkeypox-dataset-2022\/18145e9feee4993d2bab83430b583a3e981398e4\/Chickenpox_gray\/chicken13.jpg. https:\/\/github.com\/mahsan2\/Monkeypox-dataset-2022\/blob\/18145e9feee4993d2bab83430b583a3e981398e4\/Chickenpox_gray\/chicken13.jpg"},{"key":"1928_CR17","unstructured":"BSIP: Chickenpox, 4-Year-Old Child, 1St Day. Getty Images (2006). https:\/\/www.gettyimages.com\/detail\/news-photo\/chickenpox-4-year-old-child-1st-day-news-photo\/179798100"},{"key":"1928_CR18","unstructured":"Photography, C.K. Shutterstock. https:\/\/www.shutterstock.com\/image-photo\/chicken-pox-rash-viral-infections-chickenpox-591822203"},{"key":"1928_CR19","unstructured":"Find images you can use & share. Google. https:\/\/support.google.com\/websearch\/answer\/29508?"},{"key":"1928_CR20","unstructured":"Royalty-Free license for using our Media. Dreamstime. https:\/\/www.dreamstime.com\/terms#using"},{"key":"1928_CR21","unstructured":"Help center: Using files. Getty Images. https:\/\/www.gettyimages.com\/faq\/working-files"},{"key":"1928_CR22","unstructured":"University of Luxembourg HPC, the Iris Cluster. University of Luxembourg. https:\/\/hpc.uni.lu\/old\/systems\/iris\/"},{"key":"1928_CR23","unstructured":"Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)"},{"key":"1928_CR24","unstructured":"Pytorch Visual Geometry Group models. Pytorch. https:\/\/pytorch.org\/vision\/main\/models\/vgg.html"},{"key":"1928_CR25","doi-asserted-by":"crossref","unstructured":"Islam, T., Hussain, M.A., Chowdhury, F.U.H., Islam, B.R.: Can artificial intelligence detect monkeypox from digital skin images? bioRxiv (2022)","DOI":"10.1101\/2022.08.08.503193"},{"key":"1928_CR26","doi-asserted-by":"crossref","unstructured":"Gul, I., Liu, C., Yuan, X., Du, Z., Zhai, S., Lei, Z., Chen, Q., Raheem, M.A., He, Q., Hu, Q., et al: Current and perspective sensing methods for monkeypox virus. Bioengineering 9(10), 571 (2022)","DOI":"10.3390\/bioengineering9100571"},{"key":"1928_CR27","doi-asserted-by":"crossref","unstructured":"Sahin, V.H., Oztel, I., Yolcu\u00a0Oztel, G.: Human monkeypox classification from skin lesion images with deep pre-trained network using mobile application. Journal of Medical Systems 46(11), 1\u201310 (2022)","DOI":"10.1007\/s10916-022-01863-7"},{"key":"1928_CR28","doi-asserted-by":"crossref","unstructured":"Sitaula, C., Shahi, T.B.: Monkeypox virus detection using pre-trained deep learning-based approaches. Journal of Medical Systems 46(11), 1\u20139 (2022)","DOI":"10.1007\/s10916-022-01868-2"},{"key":"1928_CR29","doi-asserted-by":"crossref","unstructured":"Patel, M., Surti, M., Adnan, M.: Artificial intelligence (ai) in monkeypox infection prevention. Journal of Biomolecular Structure and Dynamics, 1\u20135 (2022)","DOI":"10.1080\/07391102.2022.2134214"},{"key":"1928_CR30","doi-asserted-by":"crossref","unstructured":"Eid, M.M., El-Kenawy, E.-S.M., Khodadadi, N., Mirjalili, S., Khodadadi, E., Abotaleb, M., Alharbi, A.H., Abdelhamid, A.A., Ibrahim, A., Amer, G.M., et al: Meta-heuristic optimization of lstm-based deep network for boosting the prediction of monkeypox cases. Mathematics 10(20), 3845 (2022)","DOI":"10.3390\/math10203845"},{"key":"1928_CR31","doi-asserted-by":"crossref","unstructured":"Alcal\u00e1-Rmz, V., Villagrana-Ba\u00f1uelos, K.E., Celaya-Padilla, J.M., Galv\u00e1n-Tejada, J.I., Gamboa-Rosales, H., Galv\u00e1n-Tejada, C.E.: Convolutional neural network for monkeypox detection. In: International Conference on Ubiquitous Computing and Ambient Intelligence, pp. 89\u2013100 (2023). Springer","DOI":"10.1007\/978-3-031-21333-5_9"},{"key":"1928_CR32","doi-asserted-by":"crossref","unstructured":"Haque, M., Ahmed, M., Nila, R.S., Islam, S., et al.: Classification of human monkeypox disease using deep learning models and attention mechanisms. arXiv preprint arXiv:2211.15459 (2022)","DOI":"10.1109\/ICCIT57492.2022.10055870"},{"key":"1928_CR33","unstructured":"Ali, S.N., Ahmed, M.T., Paul, J., Jahan, T., Sani, S.M.S., Noor, N., Hasan, T.: Monkeypox skin lesion detection using deep learning models: A preliminary feasibility study. arXiv preprint arXiv:2207.03342 (2022)"},{"key":"1928_CR34","doi-asserted-by":"crossref","unstructured":"Islam, T., Hussain, M.A., Chowdhury, F.U.H., Islam, B.R.: A web-scrapped skin image database of monkeypox, chickenpox, smallpox, cowpox, and measles. bioRxiv (2022)","DOI":"10.1101\/2022.08.01.502199"},{"key":"1928_CR35","unstructured":"Ali, S.N.: Monkeypox Skin Lesion Dataset. Kaggle (2022). https:\/\/www.kaggle.com\/datasets\/nafin59\/monkeypox-skin-lesion-dataset"},{"key":"1928_CR36","doi-asserted-by":"crossref","unstructured":"Obradovi\u0107, S.: Publication pressures create knowledge silos. Nature Human Behaviour 3(10), 1028\u20131028 (2019)","DOI":"10.1038\/s41562-019-0674-7"},{"key":"1928_CR37","doi-asserted-by":"crossref","unstructured":"Forti, L.R., Solino, L.A., Szabo, J.K.: Trade-off between urgency and reduced editorial capacity affect publication speed in ecological and medical journals during 2020. Humanities and Social Sciences Communications 8(1), 1\u20139 (2021)","DOI":"10.1057\/s41599-021-00920-9"},{"key":"1928_CR38","doi-asserted-by":"crossref","unstructured":"Maier-Hein, L., Eisenmann, M., Reinke, A., Onogur, S., Stankovic, M., Scholz, P., Arbel, T., Bogunovic, H., Bradley, A.P., Carass, A., et al: Why rankings of biomedical image analysis competitions should be interpreted with care. Nature communications 9(1), 5217 (2018)","DOI":"10.1038\/s41467-018-07619-7"},{"key":"1928_CR39","doi-asserted-by":"crossref","unstructured":"Varoquaux, G., Cheplygina, V.: Machine learning for medical imaging: methodological failures and recommendations for the future. NPJ digital medicine 5(1), 48 (2022)","DOI":"10.1038\/s41746-022-00592-y"},{"key":"1928_CR40","doi-asserted-by":"crossref","unstructured":"Greener, J.G., Kandathil, S.M., Moffat, L., Jones, D.T.: A guide to machine learning for biologists. Nature Reviews Molecular Cell Biology 23(1), 40\u201355 (2022)","DOI":"10.1038\/s41580-021-00407-0"},{"key":"1928_CR41","doi-asserted-by":"crossref","unstructured":"Yang, G., Ye, Q., Xia, J.: Unbox the black-box for the medical explainable ai via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond. Information Fusion 77, 29\u201352 (2022)","DOI":"10.1016\/j.inffus.2021.07.016"},{"key":"1928_CR42","doi-asserted-by":"crossref","unstructured":"Cabitza, F., Campagner, A.: The need to separate the wheat from the chaff in medical informatics: Introducing a comprehensive checklist for the (self)-assessment of medical AI studies. Elsevier (2021)","DOI":"10.1016\/j.ijmedinf.2021.104510"},{"key":"1928_CR43","doi-asserted-by":"crossref","unstructured":"Filipp, F.V.: Opportunities for artificial intelligence in advancing precision medicine. Current genetic medicine reports 7, 208\u2013213 (2019)","DOI":"10.1007\/s40142-019-00177-4"},{"key":"1928_CR44","doi-asserted-by":"crossref","unstructured":"Larrazabal, A.J., Nieto, N., Peterson, V., Milone, D.H., Ferrante, E.: Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences 117(23), 12592\u201312594 (2020)","DOI":"10.1073\/pnas.1919012117"},{"key":"1928_CR45","doi-asserted-by":"crossref","unstructured":"Qayyum, A., Qadir, J., Bilal, M., Al-Fuqaha, A.: Secure and robust machine learning for healthcare: A survey. IEEE Reviews in Biomedical Engineering 14, 156\u2013180 (2020)","DOI":"10.1109\/RBME.2020.3013489"},{"key":"1928_CR46","doi-asserted-by":"crossref","unstructured":"Vega, C., Kratochvil, M., Satagopam, V., Schneider, R.: Translational challenges of biomedical machine learning solutions in clinical and laboratory settings. In: Bioinformatics and Biomedical Engineering: 9th International Work-Conference, IWBBIO 2022, Maspalomas, Gran Canaria, Spain, June 27\u201330, 2022, Proceedings, Part II, pp. 353\u2013358 (2022). Springer","DOI":"10.1007\/978-3-031-07802-6_30"},{"key":"1928_CR47","doi-asserted-by":"crossref","unstructured":"Vega, C.: From hume to wuhan: an epistemological journey on the problem of induction in covid-19 machine learning models and its impact upon medical research. Ieee Access 9, 97243\u201397250 (2021)","DOI":"10.1109\/ACCESS.2021.3095222"},{"key":"1928_CR48","doi-asserted-by":"crossref","unstructured":"Reyes, M., Meier, R., Pereira, S., Silva, C.A., Dahlweid, F.-M., Tengg-Kobligk, H.v., Summers, R.M., Wiest, R.: On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiology: artificial intelligence 2(3), 190043 (2020)","DOI":"10.1148\/ryai.2020190043"},{"key":"1928_CR49","doi-asserted-by":"crossref","unstructured":"Goyal, M., Knackstedt, T., Yan, S., Hassanpour, S.: Artificial intelligence-based image classification methods for diagnosis of skin cancer: Challenges and opportunities. Computers in Biology and Medicine 127, 104065 (2020)","DOI":"10.1016\/j.compbiomed.2020.104065"},{"key":"1928_CR50","doi-asserted-by":"crossref","unstructured":"Hogarty, D.T., Su, J.C., Phan, K., Attia, M., Hossny, M., Nahavandi, S., Lenane, P., Moloney, F.J., Yazdabadi, A.: Artificial intelligence in dermatology-where we are and the way to the future: a review. American journal of clinical dermatology 21, 41\u201347 (2020)","DOI":"10.1007\/s40257-019-00462-6"},{"key":"1928_CR51","doi-asserted-by":"crossref","unstructured":"Liang, W., Tadesse, G.A., Ho, D., Fei-Fei, L., Zaharia, M., Zhang, C., Zou, J.: Advances, challenges and opportunities in creating data for trustworthy ai. Nature Machine Intelligence 4(8), 669\u2013677 (2022)","DOI":"10.1038\/s42256-022-00516-1"},{"key":"1928_CR52","doi-asserted-by":"crossref","unstructured":"Daneshjou, R., Vodrahalli, K., Liang, W., Novoa, R.A., Jenkins, M., Rotemberg, V., Ko, J., Swetter, S.M., Bailey, E.E., Gevaert, O., et al.: Disparities in dermatology AI: Assessments using diverse clinical images. arXiv preprint arXiv:2111.08006 (2021)","DOI":"10.1126\/sciadv.abq6147"},{"key":"1928_CR53","doi-asserted-by":"crossref","unstructured":"Hutchinson, B., Smart, A., Hanna, A., Denton, E., Greer, C., Kjartansson, O., Barnes, P., Mitchell, M.: Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 560\u2013575 (2021)","DOI":"10.1145\/3442188.3445918"},{"key":"1928_CR54","doi-asserted-by":"crossref","unstructured":"Garcia Santa\u00a0Cruz, B., Vega, C., Hertel, F.: The need of standardised metadata to encode causal relationships: Towards safer data-driven machine learning biological solutions. In: Computational Intelligence Methods for Bioinformatics and Biostatistics: 17th International Meeting, CIBB 2021, Virtual Event, November 15\u201317, 2021, Revised Selected Papers, pp. 200\u2013216 (2022). Springer","DOI":"10.1007\/978-3-031-20837-9_16"},{"key":"1928_CR55","doi-asserted-by":"crossref","unstructured":"Holstein, K., Wortman\u00a0Vaughan, J., Daum\u00e9\u00a0III, H., Dudik, M., Wallach, H.: Improving fairness in machine learning systems: What do industry practitioners need? In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1\u201316 (2019)","DOI":"10.1145\/3290605.3300830"},{"key":"1928_CR56","unstructured":"Mustra, M., Delac, K., Grgic, M.: Overview of the dicom standard. In: 2008 50th International Symposium ELMAR, vol. 1, pp. 39\u201344 (2008). IEEE"},{"key":"1928_CR57","doi-asserted-by":"crossref","unstructured":"Funkhouser, W.K.: Pathology: the clinical description of human disease. In: Essential Concepts in Molecular Pathology, pp. 177\u2013190. Elsevier, ??? (2020)","DOI":"10.1016\/B978-0-12-813257-9.00011-5"},{"key":"1928_CR58","doi-asserted-by":"crossref","unstructured":"Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J.W., Wallach, H., Iii, H.D., Crawford, K.: Datasheets for datasets. Communications of the ACM 64(12), 86\u201392 (2021)","DOI":"10.1145\/3458723"},{"key":"1928_CR59","doi-asserted-by":"crossref","unstructured":"Collins, G.S., de Groot, J.A., Dutton, S., Omar, O., Shanyinde, M., Tajar, A., Voysey, M., Wharton, R., Yu, L.-M., Moons, K.G., et al: External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC medical research methodology 14(1), 1\u201311 (2014)","DOI":"10.1186\/1471-2288-14-40"},{"key":"1928_CR60","doi-asserted-by":"crossref","unstructured":"Cabitza, F., Campagner, A., Soares, F., de Guadiana-Romualdo, L.G., Challa, F., Sulejmani, A., Seghezzi, M., Carobene, A.: The importance of being external. methodological insights for the external validation of machine learning models in medicine. Computer Methods and Programs in Biomedicine 208, 106288 (2021)","DOI":"10.1016\/j.cmpb.2021.106288"},{"key":"1928_CR61","doi-asserted-by":"crossref","unstructured":"Rizk, B., Brat, H., Zille, P., Guillin, R., Pouchy, C., Adam, C., Ardon, R., d\u2019Assignies, G.: Meniscal lesion detection and characterization in adult knee mri: a deep learning model approach with external validation. Physica Medica 83, 64\u201371 (2021)","DOI":"10.1016\/j.ejmp.2021.02.010"},{"key":"1928_CR62","doi-asserted-by":"crossref","unstructured":"Bien, N., Rajpurkar, P., Ball, R.L., Irvin, J., Park, A., Jones, E., Bereket, M., Patel, B.N., Yeom, K.W., Shpanskaya, K., et al: Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of mrnet. PLoS medicine 15(11), 1002699 (2018)","DOI":"10.1371\/journal.pmed.1002699"},{"key":"1928_CR63","doi-asserted-by":"crossref","unstructured":"Faes, L., Wagner, S.K., Fu, D.J., Liu, X., Korot, E., Ledsam, J.R., Back, T., Chopra, R., Pontikos, N., Kern, C., et al: Automated deep learning design for medical image classification by health-care professionals with no coding experience: a feasibility study. The Lancet Digital Health 1(5), 232\u2013242 (2019)","DOI":"10.1016\/S2589-7500(19)30108-6"},{"key":"1928_CR64","doi-asserted-by":"crossref","unstructured":"Han, S.S., Moon, I.J., Kim, S.H., Na, J.-I., Kim, M.S., Park, G.H., Park, I., Kim, K., Lim, W., Lee, J.H., et al: Assessment of deep neural networks for the diagnosis of benign and malignant skin neoplasms in comparison with dermatologists: A retrospective validation study. PLoS medicine 17(11), 1003381 (2020)","DOI":"10.1371\/journal.pmed.1003381"},{"key":"1928_CR65","unstructured":"Treveil, M., Team, D.: Introducing MLOps, O\u2019Reilly Media, Inc. (2020)"},{"key":"1928_CR66","doi-asserted-by":"crossref","unstructured":"Webb, G.I., Hyde, R., Cao, H., Nguyen, H.L., Petitjean, F.: Characterizing concept drift. Data Mining and Knowledge Discovery 30(4), 964\u2013994 (2016)","DOI":"10.1007\/s10618-015-0448-4"},{"key":"1928_CR67","doi-asserted-by":"crossref","unstructured":"Stirbu, V., Granlund, T., Mikkonen, T.: Continuous design control for machine learning in certified medical systems. Software Quality Journal, 1\u201327 (2022)","DOI":"10.1007\/s11219-022-09601-5"},{"key":"1928_CR68","doi-asserted-by":"crossref","unstructured":"Miller, T.: Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267, 1\u201338 (2019)","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"1928_CR69","unstructured":"Kim, B., Khanna, R., Koyejo, O.O.: Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems 29 (2016)"},{"key":"1928_CR70","doi-asserted-by":"crossref","unstructured":"Wulf, A.J., Seizov, O.: \u201dplease understand we cannot provide further information\u201d: evaluating content and transparency of gdpr-mandated ai disclosures. AI & SOCIETY, 1\u201322 (2022)","DOI":"10.1007\/s00146-022-01424-z"},{"key":"1928_CR71","unstructured":"Molnar, C.: Interpretable Machine Learning. Lulu. com, ??? (2020)"},{"key":"1928_CR72","unstructured":"Apley, D.W., Zhu, J.: Visualizing the effects of predictor variables in black box supervised learning models. arXiv preprint arXiv:1612.08468 (2016)"},{"key":"1928_CR73","unstructured":"Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017)"},{"key":"1928_CR74","unstructured":"Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)"},{"key":"1928_CR75","doi-asserted-by":"crossref","unstructured":"Olah, C., Mordvintsev, A., Schubert, L.: Feature visualization. Distill 2(11), 7 (2017)","DOI":"10.23915\/distill.00007"},{"key":"1928_CR76","unstructured":"Vega, C.: Code of the Review of Monkeypox Machine Learning datasets and papers. GitHub. https:\/\/github.com\/carlosvega\/DebunkMonkeypoxML"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-023-01928-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-023-01928-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-023-01928-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,29]],"date-time":"2023-12-29T06:10:16Z","timestamp":1703830216000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-023-01928-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,18]]},"references-count":76,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["1928"],"URL":"https:\/\/doi.org\/10.1007\/s10916-023-01928-1","relation":{},"ISSN":["1573-689X"],"issn-type":[{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,18]]},"assertion":[{"value":"23 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 February 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"The authors have no conflicts of interest to declare.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"37"}}