{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T19:22:45Z","timestamp":1762111365078,"version":"3.41.0"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,9,10]],"date-time":"2024-09-10T00:00:00Z","timestamp":1725926400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,10]],"date-time":"2024-09-10T00:00:00Z","timestamp":1725926400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"publisher","award":["EP\/X03870X\/1"],"award-info":[{"award-number":["EP\/X03870X\/1"]}],"id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100023699","name":"Health Data Research UK","doi-asserted-by":"publisher","award":["218529\/Z\/19\/Z"],"award-info":[{"award-number":["218529\/Z\/19\/Z"]}],"id":[{"id":"10.13039\/501100023699","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI Ethics"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Clinical prediction models are statistical or machine learning models used to quantify the risk of a certain health outcome using patient data. These can then inform potential interventions on patients, causing an effect called performative prediction: predictions inform interventions which influence the outcome they were trying to predict, leading to a potential underestimation of risk in some patients if a model is updated on this data. One suggested resolution to this is the use of hold-out sets, in which a set of patients do not receive model derived risk scores, such that a model can be safely retrained. We present an overview of clinical and research ethics regarding potential implementation of hold-out sets for clinical prediction models in health settings. We focus on the ethical principles of beneficence, non-maleficence, autonomy and justice. We also discuss informed consent, clinical equipoise, and truth-telling. We present illustrative cases of potential hold-out set implementations and discuss statistical issues arising from different hold-out set sampling methods. We also discuss differences between hold-out sets and randomised control trials, in terms of ethics and statistical issues. Finally, we give practical recommendations for researchers interested in the use hold-out sets for clinical prediction models.<\/jats:p>","DOI":"10.1007\/s43681-024-00561-z","type":"journal-article","created":{"date-parts":[[2024,9,10]],"date-time":"2024-09-10T10:03:40Z","timestamp":1725962620000},"page":"2435-2444","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Ethical considerations of use of hold-out sets in clinical prediction model management"],"prefix":"10.1007","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2578-3625","authenticated-orcid":false,"given":"Louis","family":"Chislett","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2211-233X","authenticated-orcid":false,"given":"Louis J. M.","family":"Aslett","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8066-7264","authenticated-orcid":false,"given":"Alisha R.","family":"Davies","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3638-1960","authenticated-orcid":false,"given":"Catalina A.","family":"Vallejos","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0049-8238","authenticated-orcid":false,"given":"James","family":"Liley","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,9,10]]},"reference":[{"key":"561_CR1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-21606-5","volume-title":"The Elements of Statistical Learning","author":"T Hastie","year":"2001","unstructured":"Hastie, T., Friedman, J., Tibshirani, R.: The Elements of Statistical Learning. Springer, New York, NY (2001)"},{"key":"561_CR2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0300-7","author":"EJ Topol","year":"2019","unstructured":"Topol, E.J.: High-performance medicine: the convergence of human and artificial intelligence. Nat. Publ. Group (2019). https:\/\/doi.org\/10.1038\/s41591-018-0300-7","journal-title":"Nat. Publ. Group"},{"issue":"1","key":"561_CR3","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1186\/s41512-019-0060-y","volume":"3","author":"LE Cowley","year":"2019","unstructured":"Cowley, L.E., Farewell, D.M., Maguire, S., Kemp, A.M.: Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn. Progn. Res. 3(1), 16 (2019). https:\/\/doi.org\/10.1186\/s41512-019-0060-y","journal-title":"Diagn. Progn. Res."},{"issue":"4","key":"561_CR4","doi-asserted-by":"publisher","first-page":"734","DOI":"10.1093\/ejcts\/ezs043","volume":"41","author":"SAM Nashef","year":"2012","unstructured":"Nashef, S.A.M., Roques, F., Sharples, L.D., Nilsson, J., Smith, C., Goldstone, A.R., Lockowandt, U.: Euroscore II. Eur. J. Cardiothorac. Surg. 41(4), 734\u2013745 (2012). https:\/\/doi.org\/10.1093\/ejcts\/ezs043","journal-title":"Eur. J. Cardiothorac. Surg."},{"key":"561_CR5","unstructured":"\u017dliobait\u0117, I.: Learning under Concept Drift: An Overview. arXiv preprint (2010). arXiv:1010.4784 [cs.AI]"},{"key":"561_CR6","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2020.103611","volume":"112","author":"SE Davis","year":"2020","unstructured":"Davis, S.E., Greevy, R.A., Lasko, T.A., Walsh, C.G., Matheny, M.E.: Detection of calibration drift in clinical prediction models to inform model updating. J. Biomed. Inform. 112, 103611 (2020). https:\/\/doi.org\/10.1016\/j.jbi.2020.103611","journal-title":"J. Biomed. Inform."},{"issue":"3","key":"561_CR7","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1056\/nejmc2104626","volume":"385","author":"SG Finlayson","year":"2021","unstructured":"Finlayson, S.G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I.S., Saria, S.: The clinician and dataset shift in artificial intelligence. N. Engl. J. Med. 385(3), 283\u2013286 (2021). https:\/\/doi.org\/10.1056\/nejmc2104626","journal-title":"N. Engl. J. Med."},{"key":"561_CR8","unstructured":"Perdomo, J.C., Zrnic, T., Mendler-D\u00fcnner, C., Hardt, M.: Performative prediction. In: International Conference on Machine Learning (2020)"},{"issue":"11","key":"561_CR9","doi-asserted-by":"publisher","first-page":"1085","DOI":"10.1016\/j.jclinepi.2008.04.008","volume":"61","author":"DB Toll","year":"2008","unstructured":"Toll, D.B., Janssen, K.J.M., Vergouwe, Y., Moons, K.G.M.: Validation, updating and impact of clinical prediction rules: a review. J. Clin. Epidemiol. 61(11), 1085\u20131094 (2008). https:\/\/doi.org\/10.1016\/j.jclinepi.2008.04.008","journal-title":"J. Clin. Epidemiol."},{"key":"561_CR10","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-020-0221-y","author":"RT Sutton","year":"2020","unstructured":"Sutton, R.T., Pincock, D., Baumgart, D.C., Sadowski, D.C., Fedorak, R.N., Kroeker, K.I.: An overview of clinical decision support systems: benefits, risks, and strategies for success. Nat. Res. (2020). https:\/\/doi.org\/10.1038\/s41746-020-0221-y","journal-title":"Nat. Res."},{"key":"561_CR11","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.j2099","author":"J Hippisley-Cox","year":"2017","unstructured":"Hippisley-Cox, J., Coupland, C., Brindle, P.: Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ (2017). https:\/\/doi.org\/10.1136\/bmj.j2099","journal-title":"BMJ"},{"key":"561_CR12","unstructured":"Liley, J., Emerson, S.R., Mateen, B.A., Vallejos, C.A., Aslett, L., Vollmer, S.J.: Model updating after interventions paradoxically introduces bias. In: International Conference on Artificial Intelligence and Statistics, vol. 130 (2021).https:\/\/www.who.int\/news-room\/"},{"key":"561_CR13","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz145","volume-title":"Prognostic Models will be Victims of their Own Success, Unless","author":"MC Lenert","year":"2019","unstructured":"Lenert, M.C., Matheny, M.E., Walsh, C.G.: Prognostic Models will be Victims of their Own Success, Unless. Oxford University Press, Oxford (2019)"},{"key":"561_CR14","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocz197","volume-title":"Explicit Causal Reasoning is Needed to Prevent Prognostic Models Being Victims of their Own Success","author":"M Sperrin","year":"2019","unstructured":"Sperrin, M., Jenkins, D., Martin, G.P., Peek, N.: Explicit Causal Reasoning is Needed to Prevent Prognostic Models Being Victims of their Own Success. Oxford University Press, Oxford (2019)"},{"issue":"2","key":"561_CR15","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1177\/0890334420906850","volume":"36","author":"AE Berndt","year":"2020","unstructured":"Berndt, A.E.: Sampling methods. J. Hum. Lact. 36(2), 224\u2013226 (2020). https:\/\/doi.org\/10.1177\/0890334420906850","journal-title":"J. Hum. Lact."},{"key":"561_CR16","doi-asserted-by":"publisher","unstructured":"Haidar-Wehbe, S., Emerson, S.R., Aslett, L.J.M., Liley, J.: Optimal Sizing of a Holdout Set for Safe Predictive Model Updating. arXiv preprint (2022) https:\/\/doi.org\/10.48550\/arXiv.2202.06374","DOI":"10.48550\/arXiv.2202.06374"},{"key":"561_CR17","doi-asserted-by":"publisher","DOI":"10.1159\/000509119","volume-title":"Principles of Clinical Ethics and Their Application to Practice","author":"B Varkey","year":"2021","unstructured":"Varkey, B.: Principles of Clinical Ethics and Their Application to Practice. S. Karger AG, Germany (2021)"},{"issue":"1","key":"561_CR18","doi-asserted-by":"publisher","first-page":"8","DOI":"10.2174\/1874944500801010008","volume":"1","author":"SS Coughlin","year":"2008","unstructured":"Coughlin, S.S.: How many principles for public health ethics? Open Public Health J. 1(1), 8\u201316 (2008). https:\/\/doi.org\/10.2174\/1874944500801010008","journal-title":"Open Public Health J."},{"key":"561_CR19","first-page":"41","volume-title":"Health Care Ethics","author":"J Summers","year":"2009","unstructured":"Summers, J., Morrison, E.: Principles of healthcare ethics. In: Health Care Ethics, 2nd edn., pp. 41\u201358. Jones and Bartlett Publishers, USA (2009)","edition":"2"},{"issue":"3","key":"561_CR20","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1016\/j.jmau.2014.03.003","volume":"2","author":"SY Guraya","year":"2014","unstructured":"Guraya, S.Y., London, N.J.M., Guraya, S.S.: Ethics in medical research. J. Microsc. Ultrastruct. 2(3), 121 (2014). https:\/\/doi.org\/10.1016\/j.jmau.2014.03.003","journal-title":"J. Microsc. Ultrastruct."},{"key":"561_CR21","doi-asserted-by":"publisher","unstructured":"Chen, R.J., Chen, T.Y., Lipkova, J., Wang, J.J., Williamson, D.F.K., Lu, M.Y., Sahai, S., Mahmood, F.: Algorithm Fairness in AI for Medicine and Healthcare. arXiv preprint (2021). https:\/\/doi.org\/10.48550\/arXiv.2110.00603","DOI":"10.48550\/arXiv.2110.00603"},{"key":"561_CR22","doi-asserted-by":"publisher","DOI":"10.2196\/JMIR.9134","author":"RA Verheij","year":"2018","unstructured":"Verheij, R.A., Curcin, V., Delaney, B.C., McGilchrist, M.M.: Possible sources of bias in primary care electronic health record data use and reuse. J. Med. Internet Res. (2018). https:\/\/doi.org\/10.2196\/JMIR.9134","journal-title":"J. Med. Internet Res."},{"issue":"9","key":"561_CR23","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1016\/j.puhe.2010.02.006","volume":"124","author":"D Walsh","year":"2010","unstructured":"Walsh, D., Bendel, N., Jones, R., Hanlon, P.: It\u2019s not \u2018just deprivation\u2019: why do equally deprived UK cities experience different health outcomes? Public Health 124(9), 487\u2013495 (2010). https:\/\/doi.org\/10.1016\/j.puhe.2010.02.006","journal-title":"Public Health"},{"key":"561_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(12)61179-9","volume-title":"The UK Biobank and Selection Bias","author":"JM Swanson","year":"2012","unstructured":"Swanson, J.M.: The UK Biobank and Selection Bias. Elsevier B.V, Amsterdam (2012)"},{"key":"561_CR25","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2016-011847","author":"RM Taylor","year":"2016","unstructured":"Taylor, R.M., Fern, L.A., Aslam, N., Whelan, J.S.: Direct access to potential research participants for a cohort study using a confidentiality waiver included in UK National Health Service legal statutes. BMJ Open (2016). https:\/\/doi.org\/10.1136\/bmjopen-2016-011847","journal-title":"BMJ Open"},{"key":"561_CR26","unstructured":"NHS: Protecting patient data (2022). https:\/\/digital.nhs.uk\/services\/national-data-opt-out\/understanding-the-national-data-opt-out\/protecting-patient-data"},{"key":"561_CR27","doi-asserted-by":"publisher","DOI":"10.1179\/106698111X12899036752014","author":"C Cook","year":"2011","unstructured":"Cook, C., Sheets, C.: Clinical equipoise and personal equipoise: two necessary ingredients for reducing bias in manual therapy trials. J Man Manip Ther (2011). https:\/\/doi.org\/10.1179\/106698111X12899036752014","journal-title":"J Man Manip Ther"},{"key":"561_CR28","doi-asserted-by":"publisher","unstructured":"Gillon, R.: Defending the four principles approach as a good basis for good medical practice and therefore for good medical ethics. Technical Report\u00a01 (2015). https:\/\/doi.org\/10.1136\/medethics-2014-102282","DOI":"10.1136\/medethics-2014-102282"},{"key":"561_CR29","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1191\/0969733004ne728oa","volume":"11","author":"AG Tuckett","year":"2004","unstructured":"Tuckett, A.G.: Truth-telling in clinical practice and the arguments for and against: a review of the literature. Nurs Ethics 11, 500\u2013513 (2004). https:\/\/doi.org\/10.1191\/0969733004ne728oa","journal-title":"Nurs Ethics"},{"issue":"3","key":"561_CR30","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1136\/jme.27.3.192","volume":"27","author":"RJ Sullivan","year":"2001","unstructured":"Sullivan, R.J., Menapace, L.W., White, R.M.: Truth-telling and patient diagnoses. J. Med. Ethics 27(3), 192\u2013197 (2001). https:\/\/doi.org\/10.1136\/jme.27.3.192","journal-title":"J. Med. Ethics"},{"key":"561_CR31","doi-asserted-by":"publisher","DOI":"10.1101\/2021.08.06.21261593","author":"J Liley","year":"2023","unstructured":"Liley, J., Bohner, G., Emerson, S.R., Mateen, B.A., Borland, K., Carr, D., Heald, S., Oduro, S.D., Ireland, J., Moffat, K., Porteous, R., Riddell, S., Cunningham, N., Holmes, C., Payne, K., Vollmer, S.J., Vallejos, C.A., Aslett, L.J.M.: Development and assessment of a machine learning tool for predicting emergency admission in Scotland. medRxiv (2023). https:\/\/doi.org\/10.1101\/2021.08.06.21261593","journal-title":"medRxiv"},{"issue":"8","key":"561_CR32","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1001\/jamainternmed.2021.2626","volume":"181","author":"A Wong","year":"2021","unstructured":"Wong, A., Otles, E., Donnelly, J.P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., Ghous, M., Singh, K.: External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern. Med. 181(8), 1065\u20131070 (2021). https:\/\/doi.org\/10.1001\/jamainternmed.2021.2626","journal-title":"JAMA Intern. Med."},{"issue":"3","key":"561_CR33","doi-asserted-by":"publisher","first-page":"396","DOI":"10.1097\/ALN.0000000000003871","volume":"135","author":"SJ Staffa","year":"2021","unstructured":"Staffa, S.J., Zurakowski, D.: Statistical development and validation of clinical prediction models. Anesthesiology 135(3), 396\u2013405 (2021). https:\/\/doi.org\/10.1097\/ALN.0000000000003871","journal-title":"Anesthesiology"},{"issue":"9","key":"561_CR34","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1136\/bmjqs-2018-007976","volume":"28","author":"H Snooks","year":"2019","unstructured":"Snooks, H., Bailey-Jones, K., Burge-Jones, D., Dale, J., Davies, J., Evans, B.A., Farr, A., Fitzsimmons, D., Heaven, M., Howson, H., Hutchings, H., John, G., Kingston, M., Lewis, L., Phillips, C., Porter, A., Sewell, B., Warm, D., Watkins, A., Whitman, S., Williams, V., Russell, I.: Effects and costs of implementing predictive risk stratification in primary care: a randomised stepped wedge trial. BMJ Qual. Saf 28(9), 697\u2013705 (2019). https:\/\/doi.org\/10.1136\/bmjqs-2018-007976","journal-title":"BMJ Qual. Saf"}],"container-title":["AI and Ethics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-024-00561-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s43681-024-00561-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-024-00561-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T09:11:45Z","timestamp":1748077905000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s43681-024-00561-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,10]]},"references-count":34,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["561"],"URL":"https:\/\/doi.org\/10.1007\/s43681-024-00561-z","relation":{},"ISSN":["2730-5953","2730-5961"],"issn-type":[{"type":"print","value":"2730-5953"},{"type":"electronic","value":"2730-5961"}],"subject":[],"published":{"date-parts":[[2024,9,10]]},"assertion":[{"value":"8 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 August 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 September 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}