{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T06:22:51Z","timestamp":1792736571187,"version":"4.5.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2021,6,26]],"date-time":"2021-06-26T00:00:00Z","timestamp":1624665600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,26]],"date-time":"2021-06-26T00:00:00Z","timestamp":1624665600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Assist Reprod Genet"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Embryo selection within in vitro fertilization (IVF) is the process of evaluating qualities of fertilized oocytes (embryos) and selecting the best embryo(s) available within a patient cohort for subsequent transfer or cryopreservation. In recent years, artificial intelligence (AI) has been used extensively to improve and automate the embryo ranking and selection procedure by extracting relevant information from embryo microscopy images. The AI models are evaluated based on their ability to identify the embryo(s) with the highest chance(s) of achieving a successful pregnancy. Whether such evaluations should be based on ranking performance or pregnancy prediction, however, seems to divide studies. As such, a variety of performance metrics are reported, and comparisons between studies are often made on different outcomes and data foundations. Moreover, superiority of AI methods over manual human evaluation is often claimed based on retrospective data, without any mentions of potential bias. In this paper, we provide a technical view on some of the major topics that divide how current AI models are trained, evaluated and compared. We explain and discuss the most common evaluation metrics and relate them to the two separate evaluation objectives, ranking and prediction. We also discuss when and how to compare AI models across studies and explain in detail how a selection bias is inevitable when comparing AI models against current embryo selection practice in retrospective cohort studies.<\/jats:p>","DOI":"10.1007\/s10815-021-02254-6","type":"journal-article","created":{"date-parts":[[2021,6,26]],"date-time":"2021-06-26T12:02:40Z","timestamp":1624708960000},"page":"1675-1689","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":118,"title":["Embryo selection with artificial intelligence: how to evaluate and compare methods?"],"prefix":"10.1007","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0829-1450","authenticated-orcid":false,"given":"Mikkel Fly","family":"Kragh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Henrik","family":"Karstoft","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,6,26]]},"reference":[{"issue":"1","key":"2254_CR1","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.fertnstert.2019.05.019","volume":"112","author":"N Zaninovic","year":"2019","unstructured":"Zaninovic N, Elemento O, Rosenwaks Z. Artificial intelligence: its applications in reproductive medicine and the assisted reproductive technologies. Fertil Steril 2019;112(1):28\u201330. https:\/\/doi.org\/10.1016\/j.fertnstert.2019.05.019, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0015028219304765.","journal-title":"Fertil Steril"},{"issue":"5","key":"2254_CR2","doi-asserted-by":"publisher","first-page":"934","DOI":"10.1016\/j.fertnstert.2020.10.040","volume":"114","author":"CL Curchoe","year":"2020","unstructured":"Curchoe CL, Malmsten J, Bormann C, Shafiee H, Flores-Saiffe Farias A, Mendizabal G, Chavez-Badiola A, Sigaras A, Alshubbar H, Chambost J, Jacques C, Pena C-A, Drakeley A, Freour T, Hajirasouliha I, Hickman CFL, Elemento O, Zaninovic N, Rosenwaks Z. Predictive modeling in reproductive medicine: Where will the future of artificial intelligence research take us?. Fertil Steril 2020;114(5):934\u2013940. https:\/\/doi.org\/10.1016\/j.fertnstert.2020.10.040, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0015028220325541.","journal-title":"Fertil Steril"},{"issue":"4","key":"2254_CR3","doi-asserted-by":"publisher","first-page":"R139","DOI":"10.1530\/REP-18-0523","volume":"158","author":"R Wang","year":"2019","unstructured":"Wang R, Pan W, Jin L, Li Y, Geng Y, Gao C, Chen G, Wang H, Ma D, Liao S. Artificial intelligence in reproductive medicine. Reproduction 2019;158(4):R139\u2013R154. https:\/\/doi.org\/10.1530\/REP-18-0523, https:\/\/rep.bioscientifica.com\/view\/journals\/rep\/158\/4\/REP-18-0523.xml.","journal-title":"Reproduction"},{"issue":"10","key":"2254_CR4","doi-asserted-by":"publisher","first-page":"2359","DOI":"10.1007\/s10815-020-01881-9","volume":"37","author":"EI Fernandez","year":"2020","unstructured":"Fernandez EI, Ferreira AS, Cec\u00edlio M H M, Ch\u00e9les D S, de Souza RCM, Nogueira MFG, Rocha JC. Artificial intelligence in the IVF laboratory: overview through the application of different types of algorithms for the classification of reproductive data. J Assist Reprod Genet 2020;37(10):2359\u20132376. https:\/\/doi.org\/10.1007\/s10815-020-01881-9.","journal-title":"J Assist Reprod Genet"},{"issue":"11","key":"2254_CR5","doi-asserted-by":"publisher","first-page":"2817","DOI":"10.1007\/s10815-020-01950-z","volume":"37","author":"J Swain","year":"2020","unstructured":"Swain J, VerMilyea MT, Meseguer M, Ezcurra D. AI in the treatment of fertility: key considerations. J Assist Reprod Genet 2020;37(11):2817\u20132824. https:\/\/doi.org\/10.1007\/s10815-020-01950-z.","journal-title":"J Assist Reprod Genet"},{"issue":"5","key":"2254_CR6","doi-asserted-by":"publisher","first-page":"914","DOI":"10.1016\/j.fertnstert.2020.09.157","volume":"114","author":"N Zaninovic","year":"2020","unstructured":"Zaninovic N, Rosenwaks Z. Artificial intelligence in human in vitro fertilization and embryology. Fertil Steril 2020;114(5):914\u2013920. https:\/\/doi.org\/10.1016\/j.fertnstert.2020.09.157, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0015028220323992.","journal-title":"Fertil Steril"},{"key":"2254_CR7","doi-asserted-by":"publisher","unstructured":"Santiago ID, Polansky L. 2020. Big data and artificial intelligence (AI ) are poised to transform infertility healthcare. https:\/\/doi.org\/10.20944\/preprints202010.0356.v1.","DOI":"10.20944\/preprints202010.0356.v1"},{"issue":"5","key":"2254_CR8","doi-asserted-by":"publisher","first-page":"921","DOI":"10.1016\/j.fertnstert.2020.09.159","volume":"114","author":"CL Curchoe","year":"2020","unstructured":"Curchoe CL, Flores-Saiffe Farias A, Mendizabal-Ruiz G, Chavez-Badiola A. Evaluating predictive models in reproductive medicine. Fertil Steril 2020;114(5):921\u2013926. https:\/\/doi.org\/10.1016\/j.fertnstert.2020.09.159, https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0015028220324018.","journal-title":"Fertil Steril"},{"issue":"10","key":"2254_CR9","doi-asserted-by":"publisher","first-page":"1115","DOI":"10.1038\/nbt.1686","volume":"28","author":"CC Wong","year":"2010","unstructured":"Wong CC, Loewke KE, Bossert NL, Behr B, De Jonge C J, Baer TM, Pera RAR. Non-invasive imaging of human embryos before embryonic genome activation predicts development to the blastocyst stage. Nat Biotechnol 2010;28(10):1115\u20131121. https:\/\/doi.org\/10.1038\/nbt.1686.","journal-title":"Nat Biotechnol"},{"issue":"10","key":"2254_CR10","doi-asserted-by":"publisher","first-page":"2231","DOI":"10.1093\/humrep\/dew188","volume":"31","author":"BM Petersen","year":"2016","unstructured":"Petersen BM, Boel M, Montag M, Gardner DK. Development of a generally applicable morphokinetic algorithm capable of predicting the implantation potential of embryos transferred on Day 3. Hum Reprod 2016;31(10):2231\u20132244. https:\/\/doi.org\/10.1093\/humrep\/dew188.","journal-title":"Hum Reprod"},{"issue":"6","key":"2254_CR11","doi-asserted-by":"publisher","first-page":"1011","DOI":"10.1093\/humrep\/dez064","volume":"34","author":"D Tran","year":"2019","unstructured":"Tran D, Cooke S, Illingworth PJ, Gardner DK. Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Hum Reprod 2019;34(6):1011\u20131018. https:\/\/doi.org\/10.1093\/humrep\/dez064, https:\/\/academic.oup.com\/humrep\/article\/34\/6\/1011\/5491340.","journal-title":"Hum Reprod"},{"key":"2254_CR12","doi-asserted-by":"publisher","unstructured":"VerMilyea M, Hall JMM, Diakiw SM, Johnston A, Nguyen T, Perugini D, Miller A, Picou A, Murphy AP, Perugini M. Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF. Hum Reprod. 2020:1\u201315. https:\/\/doi.org\/10.1093\/humrep\/deaa013.","DOI":"10.1093\/humrep\/deaa013"},{"issue":"0","key":"2254_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.rbmo.2020.07.003","volume":"00","author":"A Chavez-Badiola","year":"2020","unstructured":"Chavez-Badiola A, Flores-Saiffe-Far\u00edas A, Mendizabal-Ruiz G, Drakeley A, Cohen J. Embryo Ranking Intelligent Classification Algorithm (ERICA), an artificial intelligence clinical assistant with embryo ploidy and implantation predicting capabilities. Reprod BioMed Online 2020; 00 (0): 1\u20139. https:\/\/doi.org\/10.1016\/j.rbmo.2020.07.003.","journal-title":"Reprod BioMed Online"},{"key":"2254_CR14","unstructured":"Silver DH, Feder M, Gold-zamir Y, Bronstein AM. Data-Driven Prediction of Embryo Implantation Probability Using IVF Time-lapse Imaging. Medical Imaging with Deep Learning Montr\u00e9al. 2020:1\u20136. arXiv:2006.01035v2."},{"key":"2254_CR15","doi-asserted-by":"crossref","unstructured":"Berntsen J, Rimestad J, Lassen JT, Tran D, Kragh MF. 2021. Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences. arXiv:2103.07262.","DOI":"10.1371\/journal.pone.0262661"},{"issue":"1","key":"2254_CR16","doi-asserted-by":"publisher","first-page":"W1","DOI":"10.7326\/M14-0698","volume":"162","author":"KGM Moons","year":"2015","unstructured":"Moons KGM, Altman DG, Reitsma JB, Ioannidis JPA, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): Explanation and elaboration. Ann Intern Med 2015;162(1): W1\u2013W73. https:\/\/doi.org\/10.7326\/M14-0698.","journal-title":"Ann Intern Med"},{"issue":"1","key":"2254_CR17","doi-asserted-by":"publisher","first-page":"W1","DOI":"10.7326\/M18-1377","volume":"170","author":"KGM Moons","year":"2019","unstructured":"Moons KGM, Wolff RF, Riley RD, Whiting PF, Westwood M, Collins GS, Reitsma JB, Kleijnen J, Mallett S. PROBAST: A tool to assess risk of bias and applicability of prediction model studies: Explanation and elaboration. Ann Intern Med 2019;170(1):W1\u2013W33. https:\/\/doi.org\/10.7326\/M18-1377.","journal-title":"Ann Intern Med"},{"issue":"10181","key":"2254_CR18","doi-asserted-by":"publisher","first-page":"1577","DOI":"10.1016\/S0140-6736(19)30037-6","volume":"393","author":"GS Collins","year":"2019","unstructured":"Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. Lancet 2019; 393(10181):1577\u20131579. https:\/\/doi.org\/10.1016\/S0140-6736(19)30037-6.","journal-title":"Lancet"},{"key":"2254_CR19","doi-asserted-by":"crossref","unstructured":"Steyerberg EW. 2019. Clinical Prediction Models. Statistics for Biology and Health. 2nd edn.","DOI":"10.1007\/978-3-030-16399-0"},{"key":"2254_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7554\/eLife.55301","volume":"9","author":"CL Bormann","year":"2020","unstructured":"Bormann CL, Kanakasabapathy MK, Thirumalaraju P, Gupta R, Pooniwala R, Kandula H, Hariton E, Souter I, Dimitriadis I, Ramirez LB, Curchoe CL, Swain JE, Boehnlein LM, Shafiee H. Performance of a deep learning based neural network in the selection of human blastocysts for implantation. eLife 2020;9:1\u201314. https:\/\/doi.org\/10.7554\/elife.55301.","journal-title":"eLife"},{"key":"2254_CR21","doi-asserted-by":"publisher","unstructured":"Chavez-Badiola A, Flores-Saiffe-Far\u00edas A, Mendizabal-Ruiz G, Garcia-sanchez R, Drakeley AJ, Garcia-sandoval JP. Predicting pregnancy test results after embryo transfer by image feature extraction and analysis using machine learning. 2020:1\u20136. https:\/\/doi.org\/10.1038\/s41598-020-61357-9.","DOI":"10.1038\/s41598-020-61357-9"},{"key":"2254_CR22","doi-asserted-by":"publisher","first-page":"2000080","DOI":"10.1002\/aisy.202000080","volume":"2000080","author":"Y Kan-Tor","year":"2020","unstructured":"Kan-Tor Y, Zabari N, Erlich I, Szeskin A, Amitai T, Richter D, Or Y, Shoham Z, Hurwitz A, Har-Vardi I, Gavish M, Ben-Meir A, Buxboim A. Automated Evaluation of Human Embryo Blastulation and Implantation Potential using Deep-Learning. Adv Intell Syst 2020;2000080: 2000080. https:\/\/doi.org\/10.1002\/aisy.202000080.","journal-title":"Adv Intell Syst"},{"key":"2254_CR23","doi-asserted-by":"publisher","unstructured":"Cao Q, Liao SS, Meng X, Ye H, Yan Z, Wang P. Identification of Viable Embryos Using Deep Learning for Medical Image. Proceedings of the 2018 5th International Conference on Bioinformatics Research and Applications. New York: ACM; 2018. p. 69\u201372. https:\/\/doi.org\/10.1145\/3309129.3309143.","DOI":"10.1145\/3309129.3309143"},{"issue":"2","key":"2254_CR24","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1002\/rmb2.12266","volume":"18","author":"Y Miyagi","year":"2019","unstructured":"Miyagi Y, Habara T, Hirata R, Hayashi N. Feasibility of deep learning for predicting live birth from a blastocyst image in patients classified by age. Reprod Med Biol 2019;18(2):190\u2013203. https:\/\/doi.org\/10.1002\/rmb2.12266.","journal-title":"Reprod Med Biol"},{"issue":"2","key":"2254_CR25","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1002\/rmb2.12267","volume":"18","author":"Y Miyagi","year":"2019","unstructured":"Miyagi Y, Habara T, Hirata R, Hayashi N. Feasibility of artificial intelligence for predicting live birth without aneuploidy from a blastocyst image. Reprod Med Biol 2019;18(2):204\u2013211. https:\/\/doi.org\/10.1002\/rmb2.12267.","journal-title":"Reprod Med Biol"},{"issue":"4","key":"2254_CR26","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1002\/rmb2.12284","volume":"18","author":"Y Miyagi","year":"2019","unstructured":"Miyagi Y, Habara T, Hirata R, Hayashi N. Feasibility of predicting live birth by combining conventional embryo evaluation with artificial intelligence applied to a blastocyst image in patients classified by age. Reprod Med Biol 2019;18(4):344\u2013356. https:\/\/doi.org\/10.1002\/rmb2.12284.","journal-title":"Reprod Med Biol"},{"issue":"3","key":"2254_CR27","doi-asserted-by":"publisher","first-page":"94","DOI":"10.35711\/aimi.v1.i3.94","volume":"1","author":"Y Miyagi","year":"2020","unstructured":"Miyagi Y, Habara T, Hirata R, Hayashi N. Predicting a live birth by artificial intelligence incorporating both the blastocyst image and conventional embryo evaluation parameters. Artif Intell Med Imaging 2020;1 (3):94\u2013107. https:\/\/doi.org\/10.35711\/aimi.v1.i3.94, https:\/\/www.wjgnet.com\/2644-3260\/full\/v1\/i3\/94.htm.","journal-title":"Artif Intell Med Imaging"},{"issue":"2","key":"2254_CR28","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1016\/j.rbmo.2014.04.016","volume":"29","author":"L Van Loendersloot","year":"2014","unstructured":"Van Loendersloot L, Van Wely M, Van Der Veen F, Bossuyt P, Repping S. Selection of embryos for transfer in IVF: Ranking embryos based on their implantation potential using morphological scoring. Reprod BioMed Online 2014;29(2):222\u2013230. https:\/\/doi.org\/10.1016\/j.rbmo.2014.04.016.","journal-title":"Reprod BioMed Online"},{"key":"2254_CR29","doi-asserted-by":"publisher","first-page":"e3318","DOI":"10.1136\/bmj.e3318","volume":"344","author":"GCM Siontis","year":"2012","unstructured":"Siontis GCM, Tzoulaki I, Siontis KC, Ioannidis JPA. Comparisons of established risk prediction models for cardiovascular disease: systematic review. BMJ 2012;344:e3318\u2013e3318. https:\/\/doi.org\/10.1136\/bmj.e3318.","journal-title":"BMJ"},{"key":"2254_CR30","unstructured":"D\u2019Amour A, Heller K, Moldovan D, Adlam B, Alipanahi B, Beutel A, Chen C, Deaton J, Eisenstein J, Hoffman MD, Hormozdiari F, Houlsby N, Hou S, Jerfel G, Karthikesalingam A, Lucic M, Ma Y, McLean C, Mincu D, Mitani A, Montanari A, Nado Z, Natarajan V, Nielson C, Osborne TF, Raman R, Ramasamy K, Sayres R, Schrouff J, Seneviratne M, Sequeira S, Suresh H, Veitch V, Vladymyrov M, Wang X, Webster K, Yadlowsky S, Yun T, Zhai X, Sculley D. 2020. Underspecification Presents Challenges for Credibility in Modern Machine Learning. arXiv:2011.03395."},{"issue":"4","key":"2254_CR31","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1007\/s10815-019-01408-x","volume":"36","author":"CL Curchoe","year":"2019","unstructured":"Curchoe CL, Bormann CL. Artificial intelligence and machine learning for human reproduction and embryology presented at ASRM and ESHRE 2018. J Assist Reprod Genet 2019;36(4):591\u2013600. https:\/\/doi.org\/10.1007\/s10815-019-01408-x.","journal-title":"J Assist Reprod Genet"},{"issue":"3","key":"2254_CR32","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis (IJCV) 2015;115(3):211\u2013252. https:\/\/doi.org\/10.1007\/s11263-015-0816-y.","journal-title":"Int J Comput Vis (IJCV)"},{"issue":"01","key":"2254_CR33","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1142\/S2661318219500051","volume":"01","author":"T-J Chen","year":"2019","unstructured":"Chen T-J, Zheng W-L, Liu C-H, Huang I, Lai H-H, Liu M. Using Deep Learning with Large Dataset of Microscope Images to Develop an Automated Embryo Grading System. Fertility Reprod 2019;01(01):51\u201356. https:\/\/doi.org\/10.1142\/S2661318219500051.","journal-title":"Fertility Reprod"},{"issue":"1","key":"2254_CR34","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1002\/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3","volume":"3","author":"WJ Youden","year":"1950","unstructured":"Youden WJ. Index for rating diagnostic tests. Cancer 1950;3(1):32\u201335. https:\/\/doi.org\/10.1002\/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.","journal-title":"Cancer"},{"issue":"1","key":"2254_CR35","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1186\/1471-2288-14-5","volume":"14","author":"D van Klaveren","year":"2014","unstructured":"van Klaveren D, Steyerberg EW, Perel P, Vergouwe Y. Assessing discriminative ability of risk models in clustered data. BMC Med Res Methodol 2014;14(1):5. https:\/\/doi.org\/10.1186\/1471-2288-14-5.","journal-title":"BMC Med Res Methodol"},{"issue":"4","key":"2254_CR36","doi-asserted-by":"publisher","first-page":"422","DOI":"10.1145\/582415.582418","volume":"20","author":"K J\u00e4rvelin","year":"2002","unstructured":"J\u00e4rvelin K, Kek\u00e4l\u00e4inen J. Cumulated gain-based evaluation of IR techniques. ACM Trans Inf Syst 2002;20(4):422\u2013446. https:\/\/doi.org\/10.1145\/582415.582418.","journal-title":"ACM Trans Inf Syst"},{"issue":"8","key":"2254_CR37","doi-asserted-by":"publisher","first-page":"1774","DOI":"10.1093\/humrep\/dep109","volume":"24","author":"SFPJ Coppus","year":"2009","unstructured":"Coppus SFPJ, van der Veen F, Opmeer BC, Mol BWJ, Bossuyt PMM. Evaluating prediction models in reproductive medicine. Hum Reprod 2009;24(8):1774\u20131778. https:\/\/doi.org\/10.1093\/humrep\/dep109.","journal-title":"Hum Reprod"},{"issue":"1","key":"2254_CR38","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","volume":"17","author":"B Van Calster","year":"2019","unstructured":"Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Calibration: the Achilles heel of predictive analytics. BMC Med 2019;17(1):230. https:\/\/doi.org\/10.1186\/s12916-019-1466-7.","journal-title":"BMC Med"},{"key":"2254_CR39","doi-asserted-by":"crossref","unstructured":"Cleveland WS, Grosse E, Shyu WM. Local regression models. Statistical Models in S. In: Chambers J M and Hastie T J, editors. Boca Raton: Routledge; 1992. p. 309\u2013376.","DOI":"10.1201\/9780203738535-8"},{"key":"2254_CR40","doi-asserted-by":"publisher","DOI":"10.1002\/9781118548387","volume-title":"Applied Logistic Regression Wiley Series in Probability and Statistics","author":"DW Hosmer","year":"2013","unstructured":"Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression Wiley Series in Probability and Statistics. New York: Wiley; 2013. https:\/\/doi.org\/10.1002\/9781118548387."},{"issue":"1","key":"2254_CR41","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1186\/1471-2288-14-137","volume":"14","author":"T van der Ploeg","year":"2014","unstructured":"van der Ploeg T, Austin PC, Steyerberg EW. Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Med Res Methodol 2014;14(1):137. https:\/\/doi.org\/10.1186\/1471-2288-14-137.","journal-title":"BMC Med Res Methodol"},{"issue":"2","key":"2254_CR42","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1002\/sim.6787","volume":"35","author":"GS Collins","year":"2016","unstructured":"Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med 2016;35(2):214\u2013226. https:\/\/doi.org\/10.1002\/sim.6787.","journal-title":"Stat Med"},{"issue":"7859","key":"2254_CR43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1136\/bmj.e3186","volume":"344","author":"GS Collins","year":"2012","unstructured":"Collins GS, Moons KGM. Comparing risk prediction models: Should be routine when deriving a new model for the same purpose. BMJ (Online) 2012;344(7859):1\u20132. https:\/\/doi.org\/10.1136\/bmj.e3186.","journal-title":"BMJ (Online)"}],"container-title":["Journal of Assisted Reproduction and Genetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10815-021-02254-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10815-021-02254-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10815-021-02254-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T16:36:12Z","timestamp":1672590972000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10815-021-02254-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,26]]},"references-count":43,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["2254"],"URL":"https:\/\/doi.org\/10.1007\/s10815-021-02254-6","relation":{},"ISSN":["1058-0468","1573-7330"],"issn-type":[{"value":"1058-0468","type":"print"},{"value":"1573-7330","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,26]]},"assertion":[{"value":"15 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 June 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Mikkel Fly Kragh is employed by Vitrolife A\/S.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of interest"}}]}}