{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T15:31:47Z","timestamp":1775575907407,"version":"3.50.1"},"reference-count":66,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,10,2]],"date-time":"2021-10-02T00:00:00Z","timestamp":1633132800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,2]],"date-time":"2021-10-02T00:00:00Z","timestamp":1633132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"This work was supported by the ITFoC project (Information Technology for the Future of Cancer) \u2013 FLAG-ERA support."}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Artificial intelligence (AI) has the potential to transform our healthcare systems significantly. New AI technologies based on machine learning approaches should play a key role in clinical decision-making in the future. However, their implementation in health care settings remains limited, mostly due to a lack of robust validation procedures. There is a need to develop reliable assessment frameworks for the clinical validation of AI. We present here an approach for assessing AI for predicting treatment response in triple-negative breast cancer (TNBC), using real-world data and molecular -omics data from clinical data warehouses and biobanks.\n<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>The European \u201cITFoC (Information Technology for the Future Of Cancer)\u201d consortium designed a framework for the clinical validation of AI technologies for predicting treatment response in oncology.\n<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>This framework is based on seven key steps specifying: (1) the intended use of AI, (2) the target population, (3) the timing of AI evaluation, (4) the datasets used for evaluation, (5) the procedures used for ensuring data safety (including data quality, privacy and security), (6) the metrics used for measuring performance, and (7) the procedures used to ensure that the AI is explainable. This framework forms the basis of a validation platform that we are building for the \u201cITFoC Challenge\u201d. This community-wide competition will make it possible to assess and compare AI algorithms for predicting the response to TNBC treatments with external real-world datasets.\n<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The predictive performance and safety of AI technologies must be assessed in a robust, unbiased and transparent manner before their implementation in healthcare settings. We believe that the consideration of the ITFoC consortium will contribute to the safe transfer and implementation of AI in clinical settings, in the context of precision oncology and personalized care.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-021-01634-3","type":"journal-article","created":{"date-parts":[[2021,10,4]],"date-time":"2021-10-04T14:45:42Z","timestamp":1633358742000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":91,"title":["A framework for validating AI in precision medicine: considerations from the European ITFoC consortium"],"prefix":"10.1186","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9406-5547","authenticated-orcid":false,"given":"Rosy","family":"Tsopra","sequence":"first","affiliation":[]},{"given":"Xose","family":"Fernandez","sequence":"additional","affiliation":[]},{"given":"Claudio","family":"Luchinat","sequence":"additional","affiliation":[]},{"given":"Lilia","family":"Alberghina","sequence":"additional","affiliation":[]},{"given":"Hans","family":"Lehrach","sequence":"additional","affiliation":[]},{"given":"Marco","family":"Vanoni","sequence":"additional","affiliation":[]},{"given":"Felix","family":"Dreher","sequence":"additional","affiliation":[]},{"given":"O.Ugur","family":"Sezerman","sequence":"additional","affiliation":[]},{"given":"Marc","family":"Cuggia","sequence":"additional","affiliation":[]},{"given":"Marie","family":"de Tayrac","sequence":"additional","affiliation":[]},{"given":"Edvins","family":"Miklasevics","sequence":"additional","affiliation":[]},{"given":"Lucian Mihai","family":"Itu","sequence":"additional","affiliation":[]},{"given":"Marius","family":"Geanta","sequence":"additional","affiliation":[]},{"given":"Lesley","family":"Ogilvie","sequence":"additional","affiliation":[]},{"given":"Florence","family":"Godey","sequence":"additional","affiliation":[]},{"given":"Cristian Nicolae","family":"Boldisor","sequence":"additional","affiliation":[]},{"given":"Boris","family":"Campillo-Gimenez","sequence":"additional","affiliation":[]},{"given":"Cosmina","family":"Cioroboiu","sequence":"additional","affiliation":[]},{"given":"Costin Florian","family":"Ciusdel","sequence":"additional","affiliation":[]},{"given":"Simona","family":"Coman","sequence":"additional","affiliation":[]},{"given":"Oliver","family":"Hijano Cubelos","sequence":"additional","affiliation":[]},{"given":"Alina","family":"Itu","sequence":"additional","affiliation":[]},{"given":"Bodo","family":"Lange","sequence":"additional","affiliation":[]},{"given":"Matthieu","family":"Le Gallo","sequence":"additional","affiliation":[]},{"given":"Alexandra","family":"Lespagnol","sequence":"additional","affiliation":[]},{"given":"Giancarlo","family":"Mauri","sequence":"additional","affiliation":[]},{"given":"H.Okan","family":"Soykam","sequence":"additional","affiliation":[]},{"given":"Bastien","family":"Rance","sequence":"additional","affiliation":[]},{"given":"Paola","family":"Turano","sequence":"additional","affiliation":[]},{"given":"Leonardo","family":"Tenori","sequence":"additional","affiliation":[]},{"given":"Alessia","family":"Vignoli","sequence":"additional","affiliation":[]},{"given":"Christoph","family":"Wierling","sequence":"additional","affiliation":[]},{"given":"Nora","family":"Benhabiles","sequence":"additional","affiliation":[]},{"given":"Anita","family":"Burgun","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,10,2]]},"reference":[{"key":"1634_CR1","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1055\/s-0039-1677898","volume":"28","author":"C Paton","year":"2019","unstructured":"Paton C, Kobayashi S. An open science approach to artificial intelligence in healthcare. Yearb Med Inform. 2019;28:47\u201351.","journal-title":"Yearb Med Inform"},{"key":"1634_CR2","doi-asserted-by":"publisher","first-page":"94","DOI":"10.7861\/futurehosp.6-2-94","volume":"6","author":"T Davenport","year":"2019","unstructured":"Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Future Healthc J. 2019;6:94\u20138.","journal-title":"Future Healthc J"},{"key":"1634_CR3","doi-asserted-by":"publisher","first-page":"2402","DOI":"10.1001\/jama.2016.17216","volume":"316","author":"V Gulshan","year":"2016","unstructured":"Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316:2402\u201310.","journal-title":"JAMA"},{"key":"1634_CR4","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1038\/s41746-018-0040-6","volume":"1","author":"MD Abr\u00e0moff","year":"2018","unstructured":"Abr\u00e0moff MD, Lavin PT, Birch M, Shah N, Folk JC. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit Med. 2018;1:39.","journal-title":"NPJ Digit Med"},{"key":"1634_CR5","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115\u20138.","journal-title":"Nature"},{"key":"1634_CR6","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.compbiomed.2016.05.003","volume":"74","author":"JS Calvert","year":"2016","unstructured":"Calvert JS, Price DA, Chettipally UK, Barton CW, Feldman MD, Hoffman JL, et al. A computational approach to early sepsis detection. Comput Biol Med. 2016;74:69\u201373.","journal-title":"Comput Biol Med"},{"key":"1634_CR7","unstructured":"FDA. Artificial Intelligence and Machine Learning in Software as a Medical Device. FDA. 2019. http:\/\/www.fda.gov\/medical-devices\/software-medical-device-samd\/artificial-intelligence-and-machine-learning-software-medical-device. Accessed 15 Nov 2019."},{"key":"1634_CR8","first-page":"2795","volume":"2018","author":"J Ding","year":"2018","unstructured":"Ding J, Li X. An approach for validating quality of datasets for machine learning. IEEE Int Conf Big Data Big Data. 2018;2018:2795\u2013803.","journal-title":"IEEE Int Conf Big Data Big Data"},{"key":"1634_CR9","doi-asserted-by":"publisher","first-page":"405","DOI":"10.3348\/kjr.2019.0025","volume":"20","author":"DW Kim","year":"2019","unstructured":"Kim DW, Jang HY, Kim KW, Shin Y, Park SH. Design characteristics of studies reporting the performance of artificial intelligence algorithms for diagnostic analysis of medical images: results from recently published papers. Korean J Radiol. 2019;20:405\u201310.","journal-title":"Korean J Radiol"},{"key":"1634_CR10","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1136\/bmjqs-2018-008370","volume":"28","author":"R Challen","year":"2019","unstructured":"Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K. Artificial intelligence, bias and clinical safety. BMJ Qual Saf. 2019;28:231\u20137.","journal-title":"BMJ Qual Saf"},{"key":"1634_CR11","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1001\/jama.2017.7797","volume":"318","author":"F Cabitza","year":"2017","unstructured":"Cabitza F, Rasoini R, Gensini GF. Unintended consequences of machine learning in medicine. JAMA. 2017;318:517\u20138.","journal-title":"JAMA"},{"key":"1634_CR12","doi-asserted-by":"publisher","DOI":"10.3346\/jkms.2018.33.e152","author":"SH Park","year":"2018","unstructured":"Park SH, Kressel HY. Connecting technological innovation in artificial intelligence to real-world medical practice through rigorous clinical validation: what peer-reviewed medical journals could do. J Korean Med Sci. 2018. https:\/\/doi.org\/10.3346\/jkms.2018.33.e152.","journal-title":"J Korean Med Sci"},{"key":"1634_CR13","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1016\/j.csbj.2014.11.005","volume":"13","author":"K Kourou","year":"2015","unstructured":"Kourou K, Exarchos TP, Exarchos KP, Karamouzis MV, Fotiadis DI. Machine learning applications in cancer prognosis and prediction. Comput Struct Biotechnol J. 2015;13:8\u201317.","journal-title":"Comput Struct Biotechnol J"},{"key":"1634_CR14","doi-asserted-by":"publisher","first-page":"2739","DOI":"10.1016\/S0140-6736(17)31540-4","volume":"390","author":"The Lancet null","year":"2018","unstructured":"The Lancet null. Artificial intelligence in health care: within touching distance. Lancet Lond Engl. 2018;390:2739.","journal-title":"Lancet Lond Engl"},{"key":"1634_CR15","doi-asserted-by":"publisher","first-page":"800","DOI":"10.1148\/radiol.2017171920","volume":"286","author":"SH Park","year":"2018","unstructured":"Park SH, Han K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology. 2018;286:800\u20139.","journal-title":"Radiology"},{"key":"1634_CR16","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.18","author":"MD Wilkinson","year":"2016","unstructured":"Wilkinson MD, Dumontier M, Aalbersberg IJJ, Appleton G, Axton M, Baak A, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016. https:\/\/doi.org\/10.1038\/sdata.2016.18.","journal-title":"Sci Data."},{"key":"1634_CR17","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.ijmedinf.2017.02.006","volume":"102","author":"A-S Jannot","year":"2017","unstructured":"Jannot A-S, Zapletal E, Avillach P, Mamzer M-F, Burgun A, Degoulet P. The Georges Pompidou University hospital clinical data warehouse: a 8-years follow-up experience. Int J Med Inf. 2017;102:21\u20138.","journal-title":"Int J Med Inf"},{"key":"1634_CR18","unstructured":"European Medicines Agency. EMA Regulatory Science to 2025. Strategic reflection. 2018."},{"key":"1634_CR19","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1093\/jamiaopen\/ooaa033","volume":"3","author":"Y Park","year":"2020","unstructured":"Park Y, Jackson GP, Foreman MA, Gruen D, Hu J, Das AK. Evaluating artificial intelligence in medicine: phases of clinical research. JAMIA Open. 2020;3:326\u201331.","journal-title":"JAMIA Open"},{"key":"1634_CR20","unstructured":"IT Future of Cancer. https:\/\/itfoc.eu\/. Accessed 30 Apr 2020."},{"key":"1634_CR21","unstructured":"Breast cancer statistics. World Cancer Research Fund. 2018. https:\/\/www.wcrf.org\/dietandcancer\/cancer-trends\/breast-cancer-statistics. Accessed 13 Dec 2019."},{"issue":"Suppl6","key":"1634_CR22","doi-asserted-by":"publisher","first-page":"vi7","DOI":"10.1093\/annonc\/mds187","volume":"23","author":"P Boyle","year":"2012","unstructured":"Boyle P. Triple-negative breast cancer: epidemiological considerations and recommendations. Ann Oncol Off J Eur Soc Med Oncol. 2012;23(Suppl6):vi7-12.","journal-title":"Ann Oncol Off J Eur Soc Med Oncol"},{"key":"1634_CR23","doi-asserted-by":"publisher","first-page":"967","DOI":"10.1111\/tbj.13369","volume":"25","author":"P Khosravi-Shahi","year":"2019","unstructured":"Khosravi-Shahi P, Cabez\u00f3n-Guti\u00e9rrez L, Aparicio Salcedo MI. State of art of advanced triple negative breast cancer. Breast J. 2019;25:967\u201370.","journal-title":"Breast J"},{"key":"1634_CR24","first-page":"46","volume":"45","author":"T Ovcaricek","year":"2010","unstructured":"Ovcaricek T, Frkovic SG, Matos E, Mozina B, Borstnar S. Triple negative breast cancer\u2014prognostic factors and survival. Radiol Oncol. 2010;45:46\u201352.","journal-title":"Radiol Oncol"},{"issue":"Suppl 4","key":"1634_CR25","first-page":"95","volume":"14","author":"LA Ogilvie","year":"2015","unstructured":"Ogilvie LA, Wierling C, Kessler T, Lehrach H, Lange BMH. Predictive modeling of drug treatment in the area of personalized medicine. Cancer Inform. 2015;14(Suppl 4):95\u2013103.","journal-title":"Cancer Inform"},{"key":"1634_CR26","doi-asserted-by":"publisher","first-page":"567","DOI":"10.1016\/j.cels.2018.10.013","volume":"7","author":"F Fr\u00f6hlich","year":"2018","unstructured":"Fr\u00f6hlich F, Kessler T, Weindl D, Shadrin A, Schmiester L, Hache H, et al. Efficient parameter estimation enables the prediction of drug response using a mechanistic pan-cancer pathway model. Cell Syst. 2018;7:567-579.e6.","journal-title":"Cell Syst"},{"key":"1634_CR27","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1055\/s-0039-1677903","volume":"28","author":"F Magrabi","year":"2019","unstructured":"Magrabi F, Ammenwerth E, McNair JB, De Keizer NF, Hypp\u00f6nen H, Nyk\u00e4nen P, et al. Artificial intelligence in clinical decision support: challenges for evaluating AI and practical implications. Yearb Med Inform. 2019;28:128\u201334.","journal-title":"Yearb Med Inform"},{"key":"1634_CR28","unstructured":"Meurier A-L, Ghafoor Z, Foehrenbach C, Hartmann C, Herzog J, Madzou L, et al. Mission assigned by the Prime Minister \u00c9douard Philippe, p. 154."},{"key":"1634_CR29","unstructured":"AI HLEG (High-Level Expert Group on Artificial Intelligence), set up by the European Commission. Ethics Guidelines for Trustworthy AI. 2018."},{"key":"1634_CR30","doi-asserted-by":"crossref","unstructured":"Kolesnikov N, Hastings E, Keays M, Melnichuk O, Tang YA, Williams E, et al. ArrayExpress update--simplifying data submissions. Nucleic Acids Res. 2015;43 Database issue:D1113-1116.","DOI":"10.1093\/nar\/gku1057"},{"key":"1634_CR31","doi-asserted-by":"crossref","unstructured":"Barrett T, Wilhite SE, Ledoux P, Evangelista C, Kim IF, Tomashevsky M, et al. NCBI GEO: archive for functional genomics data sets\u2014update. Nucleic Acids Res. 2013;41 Database issue:D991\u20135.","DOI":"10.1093\/nar\/gks1193"},{"key":"1634_CR32","unstructured":"Committee on Strategies for Responsible Sharing of Clinical Trial Data, Board on Health Sciences Policy, Institute of Medicine. Sharing Clinical Trial Data: Maximizing Benefits, Minimizing Risk. Washington (DC): National Academies Press (US); 2015. http:\/\/www.ncbi.nlm.nih.gov\/books\/NBK269030\/. Accessed 18 Sep 2019."},{"key":"1634_CR33","first-page":"702","volume":"210","author":"D Delamarre","year":"2015","unstructured":"Delamarre D, Bouzille G, Dalleau K, Courtel D, Cuggia M. Semantic integration of medication data into the EHOP Clinical Data Warehouse. Stud Health Technol Inform. 2015;210:702\u20136.","journal-title":"Stud Health Technol Inform"},{"key":"1634_CR34","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1136\/amiajnl-2011-000681","volume":"20","author":"NG Weiskopf","year":"2013","unstructured":"Weiskopf NG, Weng C. Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. J Am Med Inform Assoc JAMIA. 2013;20:144\u201351.","journal-title":"J Am Med Inform Assoc JAMIA"},{"key":"1634_CR35","doi-asserted-by":"publisher","DOI":"10.3346\/jkms.2018.33.e213","author":"H-S Kim","year":"2018","unstructured":"Kim H-S, Lee S, Kim JH. Real-world evidence versus randomized controlled trial: clinical research based on electronic medical records. J Korean Med Sci. 2018. https:\/\/doi.org\/10.3346\/jkms.2018.33.e213.","journal-title":"J Korean Med Sci"},{"key":"1634_CR36","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.ijmedinf.2018.03.015","volume":"115","author":"R Tsopra","year":"2018","unstructured":"Tsopra R, Peckham D, Beirne P, Rodger K, Callister M, White H, et al. The impact of three discharge coding methods on the accuracy of diagnostic coding and hospital reimbursement for inpatient medical care. Int J Med Inf. 2018;115:35\u201342.","journal-title":"Int J Med Inf"},{"key":"1634_CR37","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1111\/jep.13020","volume":"25","author":"R Tsopra","year":"2019","unstructured":"Tsopra R, Wyatt JC, Beirne P, Rodger K, Callister M, Ghosh D, et al. Level of accuracy of diagnoses recorded in discharge summaries: a cohort study in three respiratory wards. J Eval Clin Pract. 2019;25:36\u201343.","journal-title":"J Eval Clin Pract"},{"key":"1634_CR38","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1197\/jamia.M2470","volume":"14","author":"RL Richesson","year":"2007","unstructured":"Richesson RL, Krischer J. Data standards in clinical research: gaps, overlaps, challenges and future directions. J Am Med Inform Assoc JAMIA. 2007;14:687\u201396.","journal-title":"J Am Med Inform Assoc JAMIA"},{"key":"1634_CR39","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1093\/bib\/bbw114","volume":"19","author":"C Manzoni","year":"2018","unstructured":"Manzoni C, Kia DA, Vandrovcova J, Hardy J, Wood NW, Lewis PA, et al. Genome, transcriptome and proteome: the rise of omics data and their integration in biomedical sciences. Brief Bioinform. 2018;19:286\u2013302.","journal-title":"Brief Bioinform"},{"key":"1634_CR40","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1038\/ng1201-365","volume":"29","author":"A Brazma","year":"2001","unstructured":"Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, et al. Minimum information about a microarray experiment (MIAME)-toward standards for microarray data. Nat Genet. 2001;29:365\u201371.","journal-title":"Nat Genet"},{"key":"1634_CR41","doi-asserted-by":"crossref","unstructured":"Spellman PT, Miller M, Stewart J, Troup C, Sarkans U, Chervitz S, et al. Design and implementation of microarray gene expression markup language (MAGE-ML). Genome Biol. 2002;3:research0046.1-research0046.9.","DOI":"10.1186\/gb-2002-3-9-research0046"},{"key":"1634_CR42","unstructured":"OMOP Common Data Model \u2013 OHDSI. https:\/\/www.ohdsi.org\/data-standardization\/the-common-data-model\/. Accessed 13 Dec 2019."},{"key":"1634_CR43","unstructured":"OSIRIS\u202f: a national data sharing project\u2014www.en.ecancer.fr. https:\/\/en.e-cancer.fr\/OSIRIS-a-national-data-sharing-project. Accessed 4 Nov 2019."},{"key":"1634_CR44","doi-asserted-by":"crossref","unstructured":"Georgiou A, Magrabi F, Hypp\u00f6nen H, Wong ZS-Y, Nyk\u00e4nen P, Scott PJ, et al. The safe and effective use of shared data underpinned by stakeholder engagement and evaluation practice. Yearb Med Inform. 2018;27:25\u20138.","DOI":"10.1055\/s-0038-1641194"},{"key":"1634_CR45","unstructured":"EU data protection rules. European Commission - European Commission. https:\/\/ec.europa.eu\/commission\/priorities\/justice-and-fundamental-rights\/data-protection\/2018-reform-eu-data-protection-rules\/eu-data-protection-rules_en. Accessed 1 Nov 2019."},{"key":"1634_CR46","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1016\/S1470-2045(16)30560-5","volume":"18","author":"J Guinney","year":"2017","unstructured":"Guinney J, Wang T, Laajala TD, Winner KK, Bare JC, Neto EC, et al. Prediction of overall survival for patients with metastatic castration-resistant prostate cancer: development of a prognostic model through a crowdsourced challenge with open clinical trial data. Lancet Oncol. 2017;18:132\u201342.","journal-title":"Lancet Oncol"},{"key":"1634_CR47","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.cmpb.2016.11.004","volume":"139","author":"T Doel","year":"2017","unstructured":"Doel T, Shakir DI, Pratt R, Aertsen M, Moggridge J, Bellon E, et al. GIFT-Cloud: a data sharing and collaboration platform for medical imaging research. Comput Methods Programs Biomed. 2017;139:181\u201390.","journal-title":"Comput Methods Programs Biomed"},{"key":"1634_CR48","doi-asserted-by":"publisher","first-page":"38","DOI":"10.2214\/AJR.18.20224","volume":"212","author":"GS Handelman","year":"2019","unstructured":"Handelman GS, Kok HK, Chandra RV, Razavi AH, Huang S, Brooks M, et al. Peering into the black box of artificial intelligence: evaluation metrics of machine learning methods. AJR Am J Roentgenol. 2019;212:38\u201343.","journal-title":"AJR Am J Roentgenol"},{"key":"1634_CR49","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-77244-8","volume-title":"Clinical prediction models: a practical approach to development, validation, and updating","author":"E Steyerberg","year":"2009","unstructured":"Steyerberg E. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer; 2009. https:\/\/doi.org\/10.1007\/978-0-387-77244-8."},{"key":"1634_CR50","first-page":"307","volume":"32","author":"DG Altman","year":"1983","unstructured":"Altman DG, Bland JM. Measurement in medicine: the analysis of method comparison studies. J R Stat Soc Ser Stat. 1983;32:307\u201317.","journal-title":"J R Stat Soc Ser Stat"},{"key":"1634_CR51","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1186\/s12916-019-1426-2","volume":"17","author":"CJ Kelly","year":"2019","unstructured":"Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 2019;17:195.","journal-title":"BMC Med"},{"key":"1634_CR52","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1038\/s41746-018-0048-y","volume":"1","author":"PA Keane","year":"2018","unstructured":"Keane PA, Topol EJ. With an eye to AI and autonomous diagnosis. NPJ Digit Med. 2018;1:40.","journal-title":"NPJ Digit Med"},{"key":"1634_CR53","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.l6927","author":"S Vollmer","year":"2020","unstructured":"Vollmer S, Mateen BA, Bohner G, Kir\u00e1ly FJ, Ghani R, Jonsson P, et al. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ. 2020. https:\/\/doi.org\/10.1136\/bmj.l6927.","journal-title":"BMJ"},{"key":"1634_CR54","doi-asserted-by":"publisher","first-page":"1320","DOI":"10.1038\/s41591-020-1041-y","volume":"26","author":"B Norgeot","year":"2020","unstructured":"Norgeot B, Quer G, Beaulieu-Jones BK, Torkamani A, Dias R, Gianfrancesco M, et al. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nat Med. 2020;26:1320\u20134.","journal-title":"Nat Med"},{"key":"1634_CR55","doi-asserted-by":"publisher","unstructured":"Haibe-Kains B, Adam GA, Hosny A, Khodakarami F, Board MS, Waldron L, et al. The importance of transparency and reproducibility in artificial intelligence research. 2020.https:\/\/doi.org\/10.1038\/s41586-020-2766-y","DOI":"10.1038\/s41586-020-2766-y"},{"key":"1634_CR56","doi-asserted-by":"publisher","first-page":"e1917094","DOI":"10.1001\/jamanetworkopen.2019.17094","volume":"2","author":"L Moja","year":"2019","unstructured":"Moja L, Polo Friz H, Capobussi M, Kwag K, Banzi R, Ruggiero F, et al. Effectiveness of a hospital-based computerized decision support system on clinician recommendations and patient outcomes: a randomized clinical trial. JAMA Netw Open. 2019;2:e1917094.","journal-title":"JAMA Netw Open"},{"key":"1634_CR57","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1197\/jamia.M2408","volume":"15","author":"\u00d6 Uzuner","year":"2008","unstructured":"Uzuner \u00d6, Goldstein I, Luo Y, Kohane I. Identifying patient smoking status from medical discharge records. J Am Med Inform Assoc JAMIA. 2008;15:14\u201324.","journal-title":"J Am Med Inform Assoc JAMIA"},{"key":"1634_CR58","doi-asserted-by":"publisher","first-page":"181re1","DOI":"10.1126\/scitranslmed.3006112","volume":"5","author":"AA Margolin","year":"2013","unstructured":"Margolin AA, Bilal E, Huang E, Norman TC, Ottestad L, Mecham BH, et al. Systematic analysis of challenge-driven improvements in molecular prognostic models for breast cancer. Sci Transl Med. 2013;5:181re1.","journal-title":"Sci Transl Med"},{"key":"1634_CR59","doi-asserted-by":"publisher","first-page":"181ra50","DOI":"10.1126\/scitranslmed.3005974","volume":"5","author":"W-Y Cheng","year":"2013","unstructured":"Cheng W-Y, Ou Yang T-H, Anastassiou D. Development of a prognostic model for breast cancer survival in an open challenge environment. Sci Transl Med. 2013;5:181ra50.","journal-title":"Sci Transl Med"},{"key":"1634_CR60","doi-asserted-by":"publisher","first-page":"e1003047","DOI":"10.1371\/journal.pcbi.1003047","volume":"9","author":"E Bilal","year":"2013","unstructured":"Bilal E, Dutkowski J, Guinney J, Jang IS, Logsdon BA, Pandey G, et al. Improving breast cancer survival analysis through competition-based multidimensional modeling. PLoS Comput Biol. 2013;9:e1003047.","journal-title":"PLoS Comput Biol"},{"key":"1634_CR61","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1002\/wsbm.1198","volume":"5","author":"R Chen","year":"2013","unstructured":"Chen R, Snyder M. Promise of Personalized Omics to Precision Medicine. Wiley Interdiscip Rev Syst Biol Med. 2013;5:73\u201382.","journal-title":"Wiley Interdiscip Rev Syst Biol Med"},{"key":"1634_CR62","doi-asserted-by":"publisher","first-page":"572","DOI":"10.1016\/j.cmet.2017.02.002","volume":"25","author":"J Nielsen","year":"2017","unstructured":"Nielsen J. Systems biology of metabolism: a driver for developing personalized and precision medicine. Cell Metab. 2017;25:572\u20139.","journal-title":"Cell Metab"},{"key":"1634_CR63","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.bbcan.2018.06.005","volume":"1870","author":"L Sun","year":"2018","unstructured":"Sun L, Suo C, Li S-T, Zhang H, Gao P. Metabolic reprogramming for cancer cells and their microenvironment: beyond the Warburg effect. Biochim Biophys Acta Rev Cancer. 2018;1870:51\u201366.","journal-title":"Biochim Biophys Acta Rev Cancer"},{"key":"1634_CR64","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1055\/s-0039-1677908","volume":"28","author":"F Wang","year":"2019","unstructured":"Wang F, Preininger A. AI in health: state of the art, challenges, and future directions. Yearb Med Inform. 2019;28:16\u201326.","journal-title":"Yearb Med Inform"},{"key":"1634_CR65","doi-asserted-by":"publisher","first-page":"260","DOI":"10.4338\/ACI-2015-09-RA-0125","volume":"7","author":"B Rance","year":"2016","unstructured":"Rance B, Canuel V, Countouris H, Laurent-Puig P, Burgun A. Integrating heterogeneous biomedical data for cancer research: the CARPEM infrastructure. Appl Clin Inform. 2016;7:260\u201374.","journal-title":"Appl Clin Inform"},{"key":"1634_CR66","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.jbi.2014.02.003","volume":"52","author":"I Danciu","year":"2014","unstructured":"Danciu I, Cowan JD, Basford M, Wang X, Saip A, Osgood S, et al. Secondary use of clinical data: the Vanderbilt approach. J Biomed Inform. 2014;52:28\u201335.","journal-title":"J Biomed Inform"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01634-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01634-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01634-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,4]],"date-time":"2021-10-04T18:26:52Z","timestamp":1633372012000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01634-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,2]]},"references-count":66,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1634"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01634-3","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,2]]},"assertion":[{"value":"18 June 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 October 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Hans Lehrach is a member of the board of Alacris Theranostics GmbH. Felix Dreher is an employee of Alacris Theranostics GmbH. Lesley Ogilvie is an employee of Alacris Theranostics GmbH. Bodo Lange is the CEO of Alacris Theranostics GmbH. Christoph Wierling is an employee of Alacris Theranostics GmbH. The other authors have no conflicts of interest to declare.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"274"}}