{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T01:21:12Z","timestamp":1775697672577,"version":"3.50.1"},"reference-count":76,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2021,11,2]],"date-time":"2021-11-02T00:00:00Z","timestamp":1635811200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,11,2]],"date-time":"2021-11-02T00:00:00Z","timestamp":1635811200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Fraunhofer-Gesellschaft zur F\u00f6rderung der angewandten Forschung e.V."}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Developers proposing new machine learning for health (ML4H) tools often pledge to match or even surpass the performance of existing tools, yet the reality is usually more complicated. Reliable deployment of ML4H to the real world is challenging as examples from diabetic retinopathy or Covid-19 screening show. We envision an integrated framework of algorithm auditing and quality control that provides a path towards the effective and reliable application of ML systems in healthcare. In this editorial, we give a summary of ongoing work towards that vision and announce a call for participation to the special issue\u00a0 <jats:italic>Machine Learning for Health: Algorithm Auditing &amp; Quality Control<\/jats:italic> in this journal to advance the practice of ML4H auditing.<\/jats:p>","DOI":"10.1007\/s10916-021-01783-y","type":"journal-article","created":{"date-parts":[[2021,11,2]],"date-time":"2021-11-02T21:02:46Z","timestamp":1635886966000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Machine Learning for Health: Algorithm Auditing &amp; Quality Control"],"prefix":"10.1007","volume":"45","author":[{"given":"Luis","family":"Oala","sequence":"first","affiliation":[]},{"given":"Andrew G.","family":"Murchison","sequence":"additional","affiliation":[]},{"given":"Pradeep","family":"Balachandran","sequence":"additional","affiliation":[]},{"given":"Shruti","family":"Choudhary","sequence":"additional","affiliation":[]},{"given":"Jana","family":"Fehr","sequence":"additional","affiliation":[]},{"given":"Alixandro Werneck","family":"Leite","sequence":"additional","affiliation":[]},{"given":"Peter G.","family":"Goldschmidt","sequence":"additional","affiliation":[]},{"given":"Christian","family":"Johner","sequence":"additional","affiliation":[]},{"given":"Elora D. M.","family":"Sch\u00f6rverth","sequence":"additional","affiliation":[]},{"given":"Rose","family":"Nakasi","sequence":"additional","affiliation":[]},{"given":"Martin","family":"Meyer","sequence":"additional","affiliation":[]},{"given":"Federico","family":"Cabitza","sequence":"additional","affiliation":[]},{"given":"Pat","family":"Baird","sequence":"additional","affiliation":[]},{"given":"Carolin","family":"Prabhu","sequence":"additional","affiliation":[]},{"given":"Eva","family":"Weicken","sequence":"additional","affiliation":[]},{"given":"Xiaoxuan","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Markus","family":"Wenzel","sequence":"additional","affiliation":[]},{"given":"Steffen","family":"Vogler","sequence":"additional","affiliation":[]},{"given":"Darlington","family":"Akogo","sequence":"additional","affiliation":[]},{"given":"Shada","family":"Alsalamah","sequence":"additional","affiliation":[]},{"given":"Emre","family":"Kazim","sequence":"additional","affiliation":[]},{"given":"Adriano","family":"Koshiyama","sequence":"additional","affiliation":[]},{"given":"Sven","family":"Piechottka","sequence":"additional","affiliation":[]},{"given":"Sheena","family":"Macpherson","sequence":"additional","affiliation":[]},{"given":"Ian","family":"Shadforth","sequence":"additional","affiliation":[]},{"given":"Regina","family":"Geierhofer","sequence":"additional","affiliation":[]},{"given":"Christian","family":"Matek","sequence":"additional","affiliation":[]},{"given":"Joachim","family":"Krois","sequence":"additional","affiliation":[]},{"given":"Bruno","family":"Sanguinetti","sequence":"additional","affiliation":[]},{"given":"Matthew","family":"Arentz","sequence":"additional","affiliation":[]},{"given":"Pavol","family":"Bielik","sequence":"additional","affiliation":[]},{"given":"Saul","family":"Calderon-Ramirez","sequence":"additional","affiliation":[]},{"given":"Auss","family":"Abbood","sequence":"additional","affiliation":[]},{"given":"Nicolas","family":"Langer","sequence":"additional","affiliation":[]},{"given":"Stefan","family":"Haufe","sequence":"additional","affiliation":[]},{"given":"Ferath","family":"Kherif","sequence":"additional","affiliation":[]},{"given":"Sameer","family":"Pujari","sequence":"additional","affiliation":[]},{"given":"Wojciech","family":"Samek","sequence":"additional","affiliation":[]},{"given":"Thomas","family":"Wiegand","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,11,2]]},"reference":[{"issue":"1","key":"1783_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-020-62724-2","volume":"10","author":"M H\u00e4gele","year":"2020","unstructured":"H\u00e4gele, M., Seegerer, P., Lapuschkin, S., Bockmayr, M., Samek, W., Klauschen, F., M\u00fcller, K.-R., and Binder, A. Resolving challenges in deep learning-based analyses of histopathological images using explanation methods. Scientific Reports 10, 1 (2020), 1\u201312.","journal-title":"Scientific Reports"},{"key":"1783_CR2","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Siddiquee, M. M.\u00a0R., Tajbakhsh, N., and Liang, J. Unet++: A nested u-net architecture for medical image segmentation. In Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, 2018, pp.\u00a03\u201311.","DOI":"10.1007\/978-3-030-00889-5_1"},{"issue":"6","key":"1783_CR3","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/ab10ca","volume":"35","author":"TA Bubba","year":"2019","unstructured":"Bubba, T.\u00a0A., Kutyniok, G., Lassas, M., M\u00e4rz, M., Samek, W., Siltanen, S., and Srinivasan, V. Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography. Inverse Problems 35, 6 (2019), 064002.","journal-title":"Inverse Problems"},{"issue":"7792","key":"1783_CR4","doi-asserted-by":"publisher","first-page":"706","DOI":"10.1038\/s41586-019-1923-7","volume":"577","author":"AW Senior","year":"2020","unstructured":"Senior, A.\u00a0W., Evans, R., Jumper, J., Kirkpatrick, J., Sifre, L., Green, T., Qin, C., \u017d\u00eddek, A., Nelson, A.\u00a0W., Bridgland, A., et\u00a0al. Improved protein structure prediction using potentials from deep learning. Nature 577, 7792 (2020), 706\u2013710.","journal-title":"Nature"},{"issue":"1","key":"1783_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41597-020-0495-6","volume":"7","author":"P Wagner","year":"2020","unstructured":"Wagner, P., Strodthoff, N., Bousseljot, R.-D., Kreiseler, D., Lunze, F.\u00a0I., Samek, W., and Schaeffter, T. Ptb-xl, a large publicly available electrocardiography dataset. Scientific Data 7, 1 (2020), 1\u201315.","journal-title":"Scientific Data"},{"issue":"4","key":"1783_CR6","doi-asserted-by":"publisher","first-page":"582","DOI":"10.1038\/s41591-021-01312-x","volume":"27","author":"E Wu","year":"2021","unstructured":"Wu, E., Wu, K., Daneshjou, R., Ouyang, D., Ho, D.\u00a0E., and Zou, J. How medical ai devices are evaluated: limitations and recommendations from an analysis of fda approvals. Nature Medicine 27, 4 (2021), 582\u2013584.","journal-title":"Nature Medicine"},{"key":"1783_CR7","doi-asserted-by":"crossref","unstructured":"Cabitza, F., Campagner, A., and Sconfienza, L.\u00a0M. As if sand were stone. new concepts and metrics to probe the ground on which to build trustable ai. BMC Medical Informatics and Decision Making 20, 1 (2020), 1\u201321.","DOI":"10.1186\/s12911-020-01224-9"},{"key":"1783_CR8","unstructured":"D\u2019Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M.\u00a0D., et\u00a0al. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint\u00a0arXiv:2011.03395 (2020)."},{"key":"1783_CR9","unstructured":"Gilmer, J., Ford, N., Carlini, N., and Cubuk, E. Adversarial examples are a natural consequence of test error in noise. In International Conference on Machine Learning (2019), PMLR, pp.\u00a02280\u20132289."},{"key":"1783_CR10","doi-asserted-by":"crossref","unstructured":"Raji, I.\u00a0D., Smart, A., White, R.\u00a0N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., and Barnes, P. Closing the ai accountability gap: defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (2020), pp.\u00a033\u201344.","DOI":"10.1145\/3351095.3372873"},{"key":"1783_CR11","unstructured":"Recht, B., Roelofs, R., Schmidt, L., and Shankar, V. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning (2019), PMLR, pp.\u00a05389\u20135400.\u00a0http:\/\/www.bmj.com\/lookup\/doi\/10.1136\/bmj.m3210"},{"key":"1783_CR12","unstructured":"Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013)."},{"key":"1783_CR13","doi-asserted-by":"crossref","unstructured":"Macdonald, J., M\u00e4rz, M., Oala, L., and Samek, W. Interval neural networks as instability detectors for image reconstructions. In Bildverarbeitung f\u00fcr die Medizin 2021 (Wiesbaden, 2021), C.\u00a0Palm, T.\u00a0M. Deserno, H.\u00a0Handels, A.\u00a0Maier, K.\u00a0Maier-Hein, and T.\u00a0Tolxdorff, Eds., Springer Fachmedien Wiesbaden, pp.\u00a0324\u2013329.","DOI":"10.1007\/978-3-658-33198-6_79"},{"key":"1783_CR14","unstructured":"Oala, L., Hei\u00df, C., Macdonald, J., M\u00e4rz, M., Kutyniok, G., and Samek, W. Detecting failure modes in image reconstructions with interval neural network uncertainty. International Journal of Computer Assisted Radiology and Surgery (2021), 1\u20139. https:\/\/arxiv.org\/abs\/2003.11566"},{"key":"1783_CR15","doi-asserted-by":"crossref","unstructured":"Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et\u00a0al. The many faces of robustness: A critical analysis of out-of-distribution generalization. arXiv preprint arXiv:2006.16241 (2020).","DOI":"10.1109\/ICCV48922.2021.00823"},{"key":"1783_CR16","unstructured":"Taori, R., Dave, A., Shankar, V., Carlini, N., Recht, B., and Schmidt, L. Measuring robustness to natural distribution shifts in image classification. arXiv preprint arXiv:2007.00644 (2020)."},{"key":"1783_CR17","unstructured":"Willis, K., and Oala, L. Post-hoc domain adaptation via guided data homogenization. CoRR abs\/2104.03624 (2021).\u00a0https:\/\/arxiv.org\/abs\/2104.03624"},{"issue":"1","key":"1783_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-019-08987-4","volume":"10","author":"S Lapuschkin","year":"2019","unstructured":"Lapuschkin, S., W\u00e4ldchen, S., Binder, A., Montavon, G., Samek, W., and M\u00fcller, K.-R. Unmasking clever hans predictors and assessing what machines really learn. Nature Communications 10, 1 (2019), 1\u20138.","journal-title":"Nature Communications"},{"key":"1783_CR19","unstructured":"Nalisnick, E., Matsukawa, A., Teh, Y.\u00a0W., Gorur, D., and Lakshminarayanan, B. Do deep generative models know what they don\u2019t know? arXiv preprint arXiv:1810.09136 (2018)."},{"key":"1783_CR20","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2021.104393","volume":"133","author":"I Neves","year":"2021","unstructured":"Neves, I., Folgado, D., Santos, S., Barandas, M., Campagner, A., Ronzio, L., Cabitza, F., and Gamboa, H. Interpretable heartbeat classification using local model-agnostic explanations on ecgs. Computers in Biology and Medicine 133 (2021), 104393.","journal-title":"Computers in Biology and Medicine"},{"key":"1783_CR21","doi-asserted-by":"publisher","first-page":"85442","DOI":"10.1109\/ACCESS.2021.3085418","volume":"9","author":"S Calderon-Ramirez","year":"2021","unstructured":"Calderon-Ramirez, S., Yang, S., Moemeni, A., Colreavy-Donnelly, S., Elizondo, D.\u00a0A., Oala, L., Rodr\u00edguez-Capit\u00e1n, J., Jim\u00e9nez-Navarro, M., L\u00f3pez-Rubio, E., and Molina-Cabello, M.\u00a0A. Improving uncertainty estimation with semi-supervised deep learning for covid-19 detection using chest x-ray images. IEEE Access 9 (2021), 85442\u201385454.","journal-title":"IEEE Access"},{"key":"1783_CR22","unstructured":"Guo, C., Pleiss, G., Sun, Y., and Weinberger, K.\u00a0Q. On calibration of modern neural networks. In International Conference on Machine Learning (2017), PMLR, pp.\u00a01321\u20131330."},{"key":"1783_CR23","unstructured":"Minderer, M., Djolonga, J., Romijnders, R., Hubis, F., Zhai, X., Houlsby, N., Tran, D., and Lucic, M. Revisiting the calibration of modern neural networks, 2021."},{"key":"1783_CR24","unstructured":"Kendall, A., and Gal, Y. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (2017), I.\u00a0Guyon, U.\u00a0V. Luxburg, S.\u00a0Bengio, H.\u00a0Wallach, R.\u00a0Fergus, S.\u00a0Vishwanathan, and R.\u00a0Garnett, Eds., vol.\u00a030, Curran Associates, Inc.\u00a0https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf"},{"issue":"3","key":"1783_CR25","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1038\/s42256-021-00307-0","volume":"3","author":"M Roberts","year":"2021","unstructured":"Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A.\u00a0I., Etmann, C., McCague, C., Beer, L., et\u00a0al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for covid-19 using chest radiographs and ct scans. Nature Machine Intelligence 3, 3 (2021), 199\u2013217.","journal-title":"Nature Machine Intelligence"},{"key":"1783_CR26","unstructured":"Heaven, W.\u00a0D. Google\u2019s medical ai was super accurate in a lab. real life was a different story. | mit technology review. https:\/\/www.technologyreview.com\/2020\/04\/27\/1000658\/google-medical-ai-accurate-lab-real-life-clinic-covid-diabetes-retina-disease\/. (Accessed on 06\/10\/2021)."},{"key":"1783_CR27","unstructured":"Oakden-Rayner, L. Ct scanning is just awful for diagnosing covid-19 \u2013 luke oakden-rayner. https:\/\/lukeoakdenrayner.wordpress.com\/2020\/03\/23\/ct-scanning-is-just-awful-for-diagnosing-covid-19\/. (Accessed on 06\/10\/2021)."},{"issue":"10192","key":"1783_CR28","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/S0140-6736(19)30762-7","volume":"394","author":"T Wiegand","year":"2019","unstructured":"Wiegand, T., Krishnamurthy, R., Kuglitsch, M., Lee, N., Pujari, S., Salath\u00e9, M., Wenzel, M., and Xu, S. Who and itu establish benchmarking process for artificial intelligence in health. The Lancet 394, 10192 (2019), 9\u201311.","journal-title":"The Lancet"},{"key":"1783_CR29","unstructured":"Oala, L., Fehr, J., Gilli, L., Balachandran, P., Leite, A.\u00a0W., Calderon-Ramirez, S., Li, D.\u00a0X., Nobis, G., Alvarado, E. A. M.\u00a0n., Jaramillo-Gutierrez, G., Matek, C., Shroff, A., Kherif, F., Sanguinetti, B., and Wiegand, T. Ml4h auditing: From paper to practice. In Proceedings of the Machine Learning for Health NeurIPS Workshop (2020), vol.\u00a0136, PMLR, pp.\u00a0280\u2013317."},{"key":"1783_CR30","unstructured":"Koshiyama, A., Kazim, E., Treleaven, P., Rai, P., Szpruch, L., Pavey, G., Ahamat, G., Leutner, F., Goebel, R., Knight, A., et\u00a0al. Towards algorithm auditing: A survey on managing legal, ethical and technological risks of ai, ml and associated algorithms."},{"issue":"48","key":"1783_CR31","doi-asserted-by":"publisher","first-page":"13538","DOI":"10.1073\/pnas.1618211113","volume":"113","author":"B Shneiderman","year":"2016","unstructured":"Shneiderman, B. Opinion: The dangers of faulty, biased, or malicious algorithms requires independent oversight. Proceedings of the National Academy of Sciences 113, 48 (2016), 13538\u201313540.\u00a0https:\/\/www.pnas.org\/content\/113\/48\/13538","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"1783_CR32","doi-asserted-by":"publisher","unstructured":"Ryan, J.\u00a0R. Software product quality assurance. In Proceedings of the June 7-10, 1982, National Computer Conference (New York, NY, USA, 1982), AFIPS \u201982, Association for Computing Machinery, p.\u00a0393\u2013398.\u00a0https:\/\/doi.org\/10.1145\/1500774.1500823","DOI":"10.1145\/1500774.1500823"},{"key":"1783_CR33","doi-asserted-by":"crossref","unstructured":"Carlini, N., and Wagner, D. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) (2017), IEEE, pp.\u00a039\u201357.","DOI":"10.1109\/SP.2017.49"},{"key":"1783_CR34","unstructured":"Hendrycks, D., and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 (2019)."},{"issue":"3","key":"1783_CR35","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1109\/JPROC.2021.3060483","volume":"109","author":"W Samek","year":"2021","unstructured":"Samek, W., Montavon, G., Lapuschkin, S., Anders, C.\u00a0J., and M\u00fcller, K.-R. Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE 109, 3 (2021), 247\u2013278.","journal-title":"Proceedings of the IEEE"},{"key":"1783_CR36","unstructured":"Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K.\u00a0T., and Ghani, R. Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577 (2018)."},{"key":"1783_CR37","unstructured":"Oala, L., Hei\u00df, C., MacDonald, J., M\u00e4rz, M., Samek, W., and Kutyniok, G. Interval neural networks: Uncertainty scores. CoRR abs\/2003.11566 (2020)."},{"issue":"4","key":"1783_CR38","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1016\/j.carj.2019.06.002","volume":"70","author":"I Balki","year":"2019","unstructured":"Balki, I., Amirabadi, A., Levman, J., Martel, A.\u00a0L., Emersic, Z., Meden, B., Garcia-Pedrero, A., Ramirez, S.\u00a0C., Kong, D., Moody, A.\u00a0R., et\u00a0al. Sample-size determination methodologies for machine learning in medical imaging research: a systematic review. Canadian Association of Radiologists Journal 70, 4 (2019), 344\u2013353.","journal-title":"Canadian Association of Radiologists Journal"},{"key":"1783_CR39","doi-asserted-by":"crossref","unstructured":"Mendez, M., Calderon-Ramirez, S., and Tyrrell, P.\u00a0N. Using cluster analysis to assess the impact of dataset heterogeneity on deep convolutional network accuracy: A first glance. In Latin American High Performance Computing Conference (2019), Springer, pp.\u00a0307\u2013319.","DOI":"10.1007\/978-3-030-41005-6_21"},{"key":"1783_CR40","doi-asserted-by":"crossref","unstructured":"Noseworthy, P.\u00a0A., Attia, Z.\u00a0I., Brewer, L.\u00a0C., Hayes, S.\u00a0N., Yao, X., Kapa, S., Friedman, P.\u00a0A., and Lopez-Jimenez, F. Assessing and mitigating bias in medical artificial intelligence: the effects of race and ethnicity on a deep learning model for ecg analysis. Circulation: Arrhythmia and Electrophysiology 13, 3 (2020), e007988.","DOI":"10.1161\/CIRCEP.119.007988"},{"key":"1783_CR41","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101714","volume":"66","author":"G M\u00e5rtensson","year":"2020","unstructured":"M\u00e5rtensson, G., Ferreira, D., Granberg, T., Cavallin, L., Oppedal, K., Padovani, A., Rektorova, I., Bonanni, L., Pardini, M., Kramberger, M.\u00a0G., et\u00a0al. The reliability of a deep learning model in clinical out-of-distribution mri data: a multicohort study. Medical Image Analysis 66 (2020), 101714.","journal-title":"Medical Image Analysis"},{"key":"1783_CR42","unstructured":"Ram\u00edrez, S.\u00a0C., and Oala, L. More than meets the eye: Semi-supervised learning under non-iid data. CoRR abs\/2104.10223 (2021).\u00a0https:\/\/arxiv.org\/abs\/2104.10223"},{"issue":"15","key":"1783_CR43","doi-asserted-by":"publisher","first-page":"3492","DOI":"10.1158\/1078-0432.CCR-18-0385","volume":"24","author":"C Parmar","year":"2018","unstructured":"Parmar, C., Barry, J.\u00a0D., Hosny, A., Quackenbush, J., and Aerts, H.\u00a0J. Data analysis strategies in medical imaging. Clinical cancer research 24, 15 (2018), 3492\u20133499.","journal-title":"Clinical cancer research"},{"key":"1783_CR44","unstructured":"FG-AI4H. Data and artificial intelligence assessment methods (daisam) reference. Reference document DEL 7.3 on FG-AI4H server (2020).\u00a0https:\/\/extranet.itu.int\/sites\/itu-t\/focusgroups\/ai4h\/SitePages\/Home.aspx"},{"key":"1783_CR45","unstructured":"Johner, C., Balachandran, P., Oala, L., Lee, A.\u00a0Y., Werneck\u00a0Leite, A., Murchison, A., Lin, A., Molnar, C., Rumball-Smith, J., Baird, P., Goldschmidt, P.\u00a0G., Quartarolo, P., Xu, S., Piechottka, S., and Hornberger, Z. Good practices for health applications of machine learning: Considerations for manufacturers and regulators. In ITU\/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting K (2021), L.\u00a0Oala, Ed., vol.\u00a0K, ITU.\u00a0https:\/\/extranet.itu.int\/sites\/itu-t\/focusgroups\/ai4h\/SitePages\/Home.aspx"},{"key":"1783_CR46","unstructured":"The Supreme Audit Institutions of Finland, Germany, the Netherlands, Norway and the UK. Auditing machine learning algorithms. https:\/\/auditingalgorithms.net\/, 2020. (Accessed on 07\/02\/2021)."},{"key":"1783_CR47","unstructured":"EUROPEAN-COMMISSION. Meddev 2.7\/1 revision 4, clinical evaluation: a guide for manufacturers and notified bodies. https:\/\/ec.europa.eu\/docsroom\/documents\/17522\/attachments\/1\/translations\/en\/renditions\/native, 2016. (Accessed on 07\/01\/2021)."},{"issue":"6","key":"1783_CR48","doi-asserted-by":"publisher","first-page":"807","DOI":"10.1038\/s41591-020-0941-1","volume":"26","author":"V Sounderajah","year":"2020","unstructured":"Sounderajah, V., Ashrafian, H., Aggarwal, R., De Fauw, J., Denniston, A.\u00a0K., Greaves, F., Karthikesalingam, A., King, D., Liu, X., Markar, S.\u00a0R., McInnes, M.\u00a0D., Panch, T., Pearson-Stuttard, J., Ting, D.\u00a0S., Golub, R.\u00a0M., Moher, D., Bossuyt, P.\u00a0M., and Darzi, A. Developing specific reporting guidelines for diagnostic accuracy studies assessing AI interventions: The STARD-AI Steering Group. Nature Medicine 26, 6 (2020), 807\u2013808.\u00a0https:\/\/doi.org\/10.1038\/s41591-020-0941-1","journal-title":"Nature Medicine"},{"key":"1783_CR49","doi-asserted-by":"crossref","unstructured":"Liu, X., Cruz Rivera, S., Moher, D., Calvert, M., Denniston, A.\u00a0K., Spirit-ai, T., and Group, C.-a.\u00a0W. CONSORT-AI extension. Nature Medicine 26, September (2020), 1364\u20131374.","DOI":"10.1038\/s41591-020-1034-x"},{"key":"1783_CR50","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m3210","volume":"370","author":"SC Rivera","year":"2020","unstructured":"Rivera, S.\u00a0C., Liu, X., Chan, A.-W., Denniston, A.\u00a0K., and Calvert, M.\u00a0J. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension. Bmj 370 (2020), m3210.","journal-title":"Bmj"},{"key":"1783_CR51","doi-asserted-by":"crossref","unstructured":"Cabitza, F., and Campagner, A. The need to separate the wheat from the chaff in medical informatics. International Journal of Medical Informatics (2021), 104510.","DOI":"10.1016\/j.ijmedinf.2021.104510"},{"issue":"12","key":"1783_CR52","doi-asserted-by":"publisher","first-page":"2011","DOI":"10.1093\/jamia\/ocaa088","volume":"27","author":"T Hernandez-Boussard","year":"2020","unstructured":"Hernandez-Boussard, T., Bozkurt, S., Ioannidis, J.\u00a0P., and Shah, N.\u00a0H. Minimar (minimum information for medical ai reporting): developing reporting standards for artificial intelligence in health care. Journal of the American Medical Informatics Association 27, 12 (2020), 2011\u20132015.","journal-title":"Journal of the American Medical Informatics Association"},{"key":"1783_CR53","doi-asserted-by":"publisher","DOI":"10.1016\/j.jdent.2021.103610","volume":"107","author":"F Schwendicke","year":"2021","unstructured":"Schwendicke, F., Singh, T., Lee, J.-H., Gaudin, R., Chaurasia, A., Wiegand, T., Uribe, S., and Krois, J. Artificial intelligence in dental research: Checklist for authors, reviewers, readers. Journal of Dentistry 107 (2021), 103610.\u00a0https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0300571221000312","journal-title":"Journal of Dentistry"},{"key":"1783_CR54","first-page":"1","volume":"28","author":"I Scott","year":"2021","unstructured":"Scott, I., Carter, S., and Coiera, E. Clinician checklist for assessing suitability of machine learning applications in healthcare. BMJ Health & Care Informatics 28, 1 (2021).","journal-title":"BMJ Health & Care Informatics"},{"issue":"4","key":"1783_CR55","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1177\/0022034520972335","volume":"100","author":"F Schwendicke","year":"2021","unstructured":"Schwendicke, F., Rossi, J., G\u00f6stemeyer, G., Elhennawy, K., Cantu, A., Gaudin, R., Chaurasia, A., Gehrung, S., and Krois, J. Cost-effectiveness of artificial intelligence for proximal caries detection. Journal of Dental Research 100, 4 (2021), 369\u2013376.\u00a0https:\/\/doi.org\/10.1177\/0022034520972335. PMID: 33198554.","journal-title":"Journal of Dental Research"},{"key":"1783_CR56","unstructured":"FG-AI4H. Clinical evaluation of ai for health. Reference document DEL 7.4 on FG-AI4H server (2021).\u00a0https:\/\/extranet.itu.int\/sites\/itu-t\/focusgroups\/ai4h\/SitePages\/Home.aspx"},{"key":"1783_CR57","doi-asserted-by":"publisher","unstructured":"Kaushal, A., Altman, R., and Langlotz, C. Geographic Distribution of US Cohorts Used to Train Deep Learning Algorithms. JAMA 324, 12 (09 2020), 1212\u20131213.\u00a0https:\/\/doi.org\/10.1001\/jama.2020.12067","DOI":"10.1001\/jama.2020.12067"},{"key":"1783_CR58","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.m689","volume":"360","author":"M Nagendran","year":"2020","unstructured":"Nagendran, M., Chen, Y., Lovejoy, C.\u00a0A., Gordon, A.\u00a0C., Komorowski, M., Harvey, H., Topol, E.\u00a0J., Ionnidis, J. P.\u00a0A., Collins, G.\u00a0S., and Maruthappu, M. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and clains of deep learning studies. British Medical Journal 360 (2020), m689.","journal-title":"British Medical Journal"},{"key":"1783_CR59","unstructured":"EU. Regulation (eu) 2017\/746 of the european parliament and of the council on medical devices, (2017).\u00a0https:\/\/eur-lex.europa.eu\/eli\/reg\/2017\/745\/oj"},{"key":"1783_CR60","unstructured":"EU. Regulation (eu) 2017\/746 of the european parliament and of the council on in vitro diagnostic medical devices, (2017).\u00a0https:\/\/eur-lex.europa.eu\/eli\/reg\/2017\/746\/oj"},{"key":"1783_CR61","unstructured":"FDA. Code of federal regulations, title 21 on foods and drugs.\u00a0https:\/\/www.ecfr.gov\/cgi-bin\/text-idx?SID=cc74806513924f0197b7809c8efbefc8&mc=true&tpl=\/ecfrbrowse\/Title21\/21tab_02.tpl"},{"key":"1783_CR62","unstructured":"IEC. Medical device software \u2013 software life cycle processes \u2013 amendment 1 (2015).\u00a0https:\/\/www.iso.org\/standard\/64686.html"},{"key":"1783_CR63","unstructured":"IEC. Medical devices \u2013 part 1: Application of usability engineering to medical devices \u2013 amendment 1 (2020).\u00a0https:\/\/www.iso.org\/standard\/73007.html"},{"key":"1783_CR64","unstructured":"ISO. Medical devices \u2013 application of risk management to medical devices (2019).\u00a0https:\/\/www.iso.org\/standard\/72704.html"},{"key":"1783_CR65","unstructured":"FDA. Fda guidance documents.\u00a0https:\/\/www.fda.gov\/regulatory-information\/search-fda-guidance-documents"},{"key":"1783_CR66","unstructured":"IMDRF. Documents by international medical device regulators forum.\u00a0http:\/\/www.imdrf.org\/documents\/documents.asp"},{"key":"1783_CR67","unstructured":"AAMI. Techical report (tr) 57 principals for medical device security - risk management.\u00a0https:\/\/store.aami.org\/s\/store#\/store\/browse\/detail\/a152E000006j60WQAQ"},{"key":"1783_CR68","unstructured":"EUROPEAN-COMMISSION. Eur-lex - 52021pc0206 - en - eur-lex. https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=CELEX:52021PC0206, 2021. (Accessed on 07\/01\/2021)."},{"key":"1783_CR69","unstructured":"US-FDA. Aiml\\_samd\\_action\\_plan. https:\/\/www.fda.gov\/media\/145022\/download?utm_medium=email&utm_source=govdelivery, 2021. (Accessed on 07\/01\/2021)."},{"key":"1783_CR70","unstructured":"Verks, B., and Oala, L. Daisam audit reporting template. In ITU\/WHO Focus Group on Artificial Intelligence for Health (FG-AI4H) - Meeting J (2020), vol.\u00a0J, ITU.\u00a0https:\/\/extranet.itu.int\/sites\/itu-t\/focusgroups\/ai4h\/SitePages\/Home.aspx"},{"key":"1783_CR71","unstructured":"FG-AI4H. Data sharing practices. Reference document DEL 5.6 on FG-AI4H server (2021).\u00a0https:\/\/extranet.itu.int\/sites\/itu-t\/focusgroups\/ai4h\/SitePages\/Home.aspx"},{"key":"1783_CR72","unstructured":"Yadav, D., Jain, R., Agrawal, H., Chattopadhyay, P., Singh, T., Jain, A., Singh, S., Lee, S., and Batra, D. Evalai: Towards better evaluation systems for AI agents. CoRR abs\/1902.03570 (2019).\u00a0http:\/\/arxiv.org\/abs\/1902.03570"},{"key":"1783_CR73","doi-asserted-by":"publisher","unstructured":"Chen, A., Chow, A., Davidson, A., DCunha, A., Ghodsi, A., Hong, S.\u00a0A., Konwinski, A., Mewald, C., Murching, S., Nykodym, T., Ogilvie, P., Parkhe, M., Singh, A., Xie, F., Zaharia, M., Zang, R., Zheng, J., and Zumar, C. Developments in mlflow: A system to accelerate the machine learning lifecycle. In Proceedings of the Fourth International Workshop on Data Management for End-to-End Machine Learning (New York, NY, USA, 2020), DEEM\u201920, Association for Computing Machinery.\u00a0https:\/\/doi.org\/10.1145\/3399579.3399867","DOI":"10.1145\/3399579.3399867"},{"key":"1783_CR74","unstructured":"FG-AI4H. Model questionnaire. Reference document J-038 on FG-AI4H server (2020).\u00a0https:\/\/extranet.itu.int\/sites\/itu-t\/focusgroups\/ai4h\/SitePages\/Home.aspx"},{"key":"1783_CR75","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1186\/s12916-019-1426-2","volume":"17","author":"CJ Kelly","year":"2019","unstructured":"Kelly, C.\u00a0J., Karthikesalingam, A., Suleyman, M., Corrado, G., and King, D. Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine 17 (2019), 195.","journal-title":"BMC Medicine"},{"key":"1783_CR76","unstructured":"Hardt, M., and Recht, B. Patterns, predictions, and actions: A story about machine learning. https:\/\/mlstory.org\u00a0(2021)."}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-021-01783-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-021-01783-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-021-01783-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,12]],"date-time":"2022-08-12T06:08:13Z","timestamp":1660284493000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-021-01783-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,2]]},"references-count":76,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1783"],"URL":"https:\/\/doi.org\/10.1007\/s10916-021-01783-y","relation":{},"ISSN":["0148-5598","1573-689X"],"issn-type":[{"value":"0148-5598","type":"print"},{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,2]]},"assertion":[{"value":"3 September 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 October 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 November 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2022","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Missing Open Access funding information has been added in the Funding Note.","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"105"}}