{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T03:03:12Z","timestamp":1790996592208,"version":"4.1.0"},"reference-count":60,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004359","name":"Vetenskapsr\u00e5det","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004359","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100013342","name":"NIHR Imperial Biomedical Research Centre","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100013342","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003130","name":"Fonds Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003130","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004040","name":"KU Leuven","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004040","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Res Methodol"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Assessing malignancy risk is important to choose appropriate management of ovarian tumors. We compared six algorithms to estimate the probabilities that an ovarian tumor is benign, borderline malignant, stage I primary invasive, stage II-IV primary invasive, or secondary metastatic.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This retrospective cohort study used 5909 patients recruited from 1999 to 2012 for model development, and 3199 patients recruited from 2012 to 2015 for model validation. Patients were recruited at oncology referral or general centers and underwent an ultrasound examination and surgery\u2009\u2264\u2009120 days later. We developed models using standard multinomial logistic regression (MLR), Ridge MLR, random forest (RF), XGBoost, neural networks (NN), and support vector machines (SVM). We used nine clinical and ultrasound predictors but developed models with or without CA125.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Most tumors were benign (3980 in development and 1688 in validation data), secondary metastatic tumors were least common (246 and 172). The c-statistic (AUROC) to discriminate benign from any type of malignant tumor ranged from 0.89 to 0.92 for models with CA125, from 0.89 to 0.91 for models without. The multiclass c-statistic ranged from 0.41 (SVM) to 0.55 (XGBoost) for models with CA125, and from 0.42 (SVM) to 0.51 (standard MLR) for models without. Multiclass calibration was best for RF and XGBoost. Estimated probabilities for a benign tumor in the same patient often differed by more than 0.2 (20% points) depending on the model. Net Benefit for diagnosing malignancy was similar for algorithms at the commonly used 10% risk threshold, but was slightly higher for RF at higher thresholds. Comparing models, between 3% (XGBoost vs. NN, with CA125) and 30% (NN vs. SVM, without CA125) of patients fell on opposite sides of the 10% threshold.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Although several models had similarly good performance, individual probability estimates varied substantially.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12874-023-02103-3","type":"journal-article","created":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T04:02:29Z","timestamp":1700798549000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Multiclass risk models for ovarian malignancy: an illustration of prediction uncertainty due to the choice of algorithm"],"prefix":"10.1186","volume":"23","author":[{"given":"Ashleigh","family":"Ledger","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jolien","family":"Ceusters","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lil","family":"Valentin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antonia","family":"Testa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Caroline","family":"Van Holsbeke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dorella","family":"Franchi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom","family":"Bourne","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wouter","family":"Froyman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dirk","family":"Timmerman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1613-7450","authenticated-orcid":false,"given":"Ben","family":"Van Calster","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,11,24]]},"reference":[{"key":"2103_CR1","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1016\/j.ygyno.2012.04.012","volume":"126","author":"YL Woo","year":"2012","unstructured":"Woo YL, Kyrgiou M, Bryant A, et al. Centralisation of services for gynaecological cancers \u2013 a Cochrane systematic review. Gynecol Oncol. 2012;126:286\u201390.","journal-title":"Gynecol Oncol"},{"key":"2103_CR2","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1093\/jnci\/djn033","volume":"100","author":"F Vernooij","year":"2008","unstructured":"Vernooij F, Heintz APM, Witteveen PO, et al. Specialized care and survival of Ovarian cancer patients in the Netherlands: nationwide cohort study. J Natl Cancer Inst. 2008;100:399\u2013406.","journal-title":"J Natl Cancer Inst"},{"key":"2103_CR3","doi-asserted-by":"publisher","first-page":"448","DOI":"10.1016\/S1470-2045(18)30837-4","volume":"20","author":"W Froyman","year":"2019","unstructured":"Froyman W, Landolfo C, De Cock B, et al. Risk of Complications in patients with conservatively managed ovarian tumours (IOTA5): a 2-year interim analysis of a multicentre, prospective, cohort study. Lancet Oncol. 2019;20:448\u201358.","journal-title":"Lancet Oncol"},{"key":"2103_CR4","doi-asserted-by":"publisher","first-page":"W1","DOI":"10.7326\/M14-0698","volume":"162","author":"KGM Moons","year":"2015","unstructured":"Moons KGM, Altman DG, Reitsma JB, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162:W1\u201373.","journal-title":"Ann Intern Med"},{"key":"2103_CR5","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-16399-0","volume-title":"Clinical prediction models: a practical approach to development, validation, and updating","author":"EW Steyerberg","year":"2019","unstructured":"Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. 2nd ed. Cham: Springer; 2019.","edition":"2"},{"key":"2103_CR6","doi-asserted-by":"publisher","first-page":"g5920","DOI":"10.1136\/bmj.g5920","volume":"349","author":"B Van Calster","year":"2014","unstructured":"Van Calster B, Van Hoorde K, Valentin L, et al. Evaluating the risk of Ovarian cancer before Surgery using the ADNEX model to differentiate between benign, borderline, early and advanced stage invasive, and secondary metastatic tumours: prospective multicentre diagnostic study. BMJ. 2014;349:g5920.","journal-title":"BMJ"},{"key":"2103_CR7","first-page":"32","volume":"7","author":"B Van Calster","year":"2015","unstructured":"Van Calster B, Van Hoorde K, Froyman W, et al. Practical guidance for applying the ADNEX model from the IOTA group to discriminate between different subtypes of adnexal tumors. Facts Views Vis Obgyn. 2015;7:32\u201341.","journal-title":"Facts Views Vis Obgyn"},{"key":"2103_CR8","doi-asserted-by":"publisher","first-page":"m2614","DOI":"10.1136\/bmj.m2614","volume":"370","author":"B Van Calster","year":"2020","unstructured":"Van Calster B, Valentin L, Froyman W, et al. Validation of models to diagnose Ovarian cancer in patients managed surgically or conservatively: multicentre cohort study. BMJ. 2020;370:m2614.","journal-title":"BMJ"},{"key":"2103_CR9","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1002\/uog.23635","volume":"58","author":"D Timmerman","year":"2021","unstructured":"Timmerman D, Planchamp F, Bourne T, et al. ESGO\/ISUOG\/IOTA\/ESGE Consensus Statement on pre-operative diagnosis of ovarian tumors. Ultrasound Obstet Gynecol. 2021;58:148\u201368.","journal-title":"Ultrasound Obstet Gynecol"},{"key":"2103_CR10","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","volume":"110","author":"E Christodoulou","year":"2019","unstructured":"Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019;110:12\u201322.","journal-title":"J Clin Epidemiol"},{"key":"2103_CR11","doi-asserted-by":"publisher","first-page":"e677","DOI":"10.1016\/S2589-7500(20)30200-4","volume":"2","author":"J Wilkinson","year":"2020","unstructured":"Wilkinson J, Arnold KF, Murray EJ, et al. Time to reality check the promises of machine learning-powered prediction medicine. Lancet Digit Health. 2020;2:e677\u201380.","journal-title":"Lancet Digit Health"},{"key":"2103_CR12","doi-asserted-by":"publisher","first-page":"e048008","DOI":"10.1136\/bmjopen-2020-048008","volume":"11","author":"GS Collins","year":"2021","unstructured":"Collins GS, Dhiman P, Andour Navarro CL, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open. 2021;11:e048008.","journal-title":"BMJ Open"},{"key":"2103_CR13","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1038\/s41746-019-0209-7","volume":"3","author":"PD Myers","year":"2020","unstructured":"Myers PD, Ng K, Severson K, et al. Identifying unreliable predictions in clinical risk models. NPJ Digit Med. 2020;3:8.","journal-title":"NPJ Digit Med"},{"key":"2103_CR14","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1038\/s41746-020-00367-3","volume":"4","author":"B Kompa","year":"2021","unstructured":"Kompa B, Snoek J, Beam AL. Second opinion needed: communicating uncertainty in medical machine learning. NPJ Digit Med. 2021;4:4.","journal-title":"NPJ Digit Med"},{"key":"2103_CR15","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1007\/BF01704747","volume":"21","author":"S Lemeshow","year":"1995","unstructured":"Lemeshow S, Klar J, Teres D. Outcome prediction for individual intensive care patients: useful, misused, or abused? Intensive Care Med. 1995;21:770\u20136.","journal-title":"Intensive Care Med"},{"key":"2103_CR16","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1186\/s41512-020-00082-3","volume":"4","author":"A Pate","year":"2020","unstructured":"Pate A, Emsley R, Sperrin M, et al. Impact of sample size on the stability of risk scores from clinical prediction models: a case study in Cardiovascular Disease. Diagn Progn Res. 2020;4:14.","journal-title":"Diagn Progn Res"},{"key":"2103_CR17","doi-asserted-by":"crossref","unstructured":"Riley RD, Collins GS. Stability of clinical prediction models developed using statistical or machine learning methods. Biom J. 2023;e2200302.","DOI":"10.1002\/bimj.202200302"},{"key":"2103_CR18","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1186\/s12916-019-1368-8","volume":"17","author":"A Pate","year":"2019","unstructured":"Pate A, Emsley R, Ashcroft DM, et al. The uncertainty with using risk prediction models for individual decision making: an exemplar cohort study examining the prediction of Cardiovascular Disease in English primary care. BMC Med. 2019;17:134.","journal-title":"BMC Med"},{"key":"2103_CR19","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1016\/j.jclinepi.2004.07.008","volume":"58","author":"EW Steyerbeg","year":"2005","unstructured":"Steyerbeg EW, Eijkemans MJC, Boersma E, et al. Equally valid models gave divergent predictions for mortality in acute Myocardial Infarction patients in a comparison of logistic regression models. J Clin Epidemiol. 2005;58:383\u201390.","journal-title":"J Clin Epidemiol"},{"key":"2103_CR20","doi-asserted-by":"publisher","first-page":"8794","DOI":"10.1200\/JCO.2005.01.7632","volume":"23","author":"D Timmerman","year":"2005","unstructured":"Timmerman D, Testa AC, Bourne T, et al. Logistic regression model to distinguish between the benign and malignant adnexal mass before Surgery: a multicenter study by the International Ovarian Tumor Analysis Group. J Clin Oncol. 2005;23:8794\u2013801.","journal-title":"J Clin Oncol"},{"key":"2103_CR21","doi-asserted-by":"publisher","first-page":"684","DOI":"10.1158\/1078-0432.CCR-08-0113","volume":"15","author":"C Van Holsbeke","year":"2009","unstructured":"Van Holsbeke C, Van Calster B, Testa AC, et al. Prospective internal validation of mathematical models to predict malignancy in adnexal masses: results from the international ovarian Tumor analysis study. Clin Cancer Res. 2009;15:684\u201391.","journal-title":"Clin Cancer Res"},{"key":"2103_CR22","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1002\/uog.7636","volume":"36","author":"D Timmerman","year":"2010","unstructured":"Timmerman D, Van Calster B, Testa AC, et al. Ovarian cancer prediction in adnexal masses using ultrasound-based logistic regression models. A temporal and external validation study by the IOTA group. Ultrasound Obstet Gynecol. 2010;36:226\u201334.","journal-title":"Ultrasound Obstet Gynecol"},{"key":"2103_CR23","doi-asserted-by":"publisher","first-page":"680","DOI":"10.1038\/bjc.2014.333","volume":"111","author":"A Testa","year":"2014","unstructured":"Testa A, Kaijser J, Wynants L, et al. Strategies to diagnose Ovarian cancer: new evidence from phase 3 of the multicentre international IOTA study. Br J Cancer. 2014;111:680\u20138.","journal-title":"Br J Cancer"},{"key":"2103_CR24","doi-asserted-by":"publisher","first-page":"e071018","DOI":"10.1136\/bmj-2022-071018","volume":"380","author":"TPA Debray","year":"2023","unstructured":"Debray TPA, Collins GS, Riley RD, et al. Transparent reporting of multivariate prediction models developed or validated using clustered data (TRIPOD-Cluster): explanation and elaboration. BMJ. 2023;380:e071018.","journal-title":"BMJ"},{"key":"2103_CR25","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1046\/j.1469-0705.2000.00287.x","volume":"16","author":"D Timmerman","year":"2000","unstructured":"Timmerman D, Valentin L, Bourne TH, et al. Terms, definitions and measurements to describe the sonographic features of adnexal tumors: a consensus opinion from the International Ovarian Tumor Analysis (IOTA) group. Ultrasound Obstet Gynecol. 2000;16:500\u20135.","journal-title":"Ultrasound Obstet Gynecol"},{"key":"2103_CR26","doi-asserted-by":"crossref","unstructured":"Heintz APM, Odicino F, Maisonneuve P et al. Carcinoma of the ovary. FIGO 26th Annual Report on the Results of Treatment in Gynecological Cancer. Int J Gynaecol Obstet. 2006;95:S161\u201392.","DOI":"10.1016\/S0020-7292(06)60033-7"},{"key":"2103_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ijgo.2013.10.001","volume":"124","author":"J Prat","year":"2014","unstructured":"Prat J. Staging classification for cancer of the ovary, fallopian tube, and peritoneum. Int J Gynaecol Obstet. 2014;124:1\u20135.","journal-title":"Int J Gynaecol Obstet"},{"key":"2103_CR28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v033.i01","volume":"33","author":"J Friedman","year":"2010","unstructured":"Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33:1\u201322.","journal-title":"J Stat Softw"},{"key":"2103_CR29","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-6849-3","volume-title":"Applied Predictive Modelling","author":"M Kuhn","year":"2013","unstructured":"Kuhn M, Johnson K. Applied Predictive Modelling. New York: Springer; 2013."},{"key":"2103_CR30","doi-asserted-by":"publisher","first-page":"191","DOI":"10.2307\/2347628","volume":"41","author":"S Le Cessie","year":"1992","unstructured":"Le Cessie S, van Houwelingen JC. Ridge estimators in logistic regression. Appl Statist. 1992;41:191\u2013201.","journal-title":"Appl Statist"},{"key":"2103_CR31","unstructured":"Chen TQ, Guestrin C, XGBoost. A scalable tree boosting system. arXiv. 2016; 1603.02754v3. https:\/\/arxiv.org\/abs\/1603.02754."},{"key":"2103_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-19425-7","volume-title":"Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis","author":"FE Harrell Jr","year":"2015","unstructured":"Harrell FE Jr. Regression modeling strategies: with applications to linear models, logistic and ordinal regression, and survival analysis. 2nd ed. Cham: Springer; 2015.","edition":"2"},{"key":"2103_CR33","doi-asserted-by":"publisher","first-page":"2610","DOI":"10.1002\/sim.5321","volume":"31","author":"B Van Calster","year":"2012","unstructured":"Van Calster B, Van Belle V, Vergouwe Y, et al. Extending the c-statistic to nominal polytomous outcomes: the polytomous discrimination index. Stat Med. 2012;31:2610\u201326.","journal-title":"Stat Med"},{"key":"2103_CR34","doi-asserted-by":"publisher","first-page":"761","DOI":"10.1007\/s10654-012-9733-3","volume":"27","author":"B Van Calster","year":"2012","unstructured":"Van Calster B, Vergouwe Y, Looman CWN, et al. Assessing the discriminative ability of risk models for more than two outcomes categories. Eur J Epidemiol. 2012;27:761\u201370.","journal-title":"Eur J Epidemiol"},{"key":"2103_CR35","doi-asserted-by":"publisher","first-page":"2585","DOI":"10.1002\/sim.6114","volume":"33","author":"K Van Hoorde","year":"2014","unstructured":"Van Hoorde K, Vergouwe Y, Timmerman D, et al. Assessing calibration of multinomial risk prediction models. Stat Med. 2014;33:2585\u201396.","journal-title":"Stat Med"},{"key":"2103_CR36","doi-asserted-by":"publisher","first-page":"1334","DOI":"10.1002\/sim.9281","volume":"41","author":"M Edlinger","year":"2022","unstructured":"Edlinger M, van Smeden M, Alber HF, et al. Risk prediction models for discrete ordinal outcomes: calibration and the impact of the proportional odds assumption. Stat Med. 2022;41:1334\u201360.","journal-title":"Stat Med"},{"key":"2103_CR37","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1177\/0272989X06295361","volume":"26","author":"AJ Vickers","year":"2006","unstructured":"Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26:565\u201374.","journal-title":"Med Decis Making"},{"key":"2103_CR38","doi-asserted-by":"publisher","first-page":"i6","DOI":"10.1136\/bmj.i6","volume":"352","author":"AJ Vickers","year":"2016","unstructured":"Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers and diagnostic tests. BMJ. 2016;352:i6.","journal-title":"BMJ"},{"key":"2103_CR39","doi-asserted-by":"publisher","first-page":"2034","DOI":"10.1002\/sim.7653","volume":"37","author":"L Wynants","year":"2018","unstructured":"Wynants L, Riley RD, Timmerman D, et al. Random-effects meta-analysis of the clinical utility of tests and prediction models. Stat Med. 2018;37:2034\u201352.","journal-title":"Stat Med"},{"key":"2103_CR40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3310\/hta22440","volume":"22","author":"M Westwood","year":"2018","unstructured":"Westwood M, Ramaekers B, Lang S, et al. Risk scores to guide referral decisions for people with suspected Ovarian cancer in secondary care: a systematic review and cost-effectiveness analysis. Health Technol Assess. 2018;22:1\u2013264.","journal-title":"Health Technol Assess"},{"key":"2103_CR41","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1046\/j.1469-0705.1999.13010017.x","volume":"13","author":"D Timmerman","year":"1999","unstructured":"Timmerman D, Verrelst H, Bourne TH, et al. Artificial neural network models for the preoperative discrimination between malignant and benign adnexal masses. Ultrasound Obstet Gynecol. 1999;13:17\u201325.","journal-title":"Ultrasound Obstet Gynecol"},{"key":"2103_CR42","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1148\/radiology.210.2.r99fe18399","volume":"210","author":"R Biagiotti","year":"1999","unstructured":"Biagiotti R, Desii C, Vanzi E, et al. Predicting ovarian malignancy: application of artificial neural networks to transvaginal and color doppler flow us. Radiology. 1999;210:399\u2013403.","journal-title":"Radiology"},{"key":"2103_CR43","doi-asserted-by":"publisher","first-page":"496","DOI":"10.1002\/uog.3996","volume":"29","author":"B Van Calster","year":"2007","unstructured":"Van Calster B, Timmerman D, Lu C, et al. Preoperative diagnosis of ovarian tumors using bayesian kernel-based methods. Ultrasound Obstet Gynecol. 2007;29:496\u2013504.","journal-title":"Ultrasound Obstet Gynecol"},{"key":"2103_CR44","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1186\/1471-2288-10-96","volume":"10","author":"B Van Calster","year":"2010","unstructured":"Van Calster B, Valentin L, Van Holsbeke C, et al. Polytomous diagnosis of ovarian tumors as benign, borderline, primary invasive or metastatic: development and validation of standard and kernel-based risk prediction models. BMC Med Res Methodol. 2010;10:96.","journal-title":"BMC Med Res Methodol"},{"key":"2103_CR45","doi-asserted-by":"publisher","first-page":"4795","DOI":"10.21873\/anticanres.14482","volume":"40","author":"M Akazawa","year":"2020","unstructured":"Akazawa M, Hashimoto K. Artificial intelligence in Ovarian cancer diagnosis. Anticancer Res. 2020;40:4795\u2013800.","journal-title":"Anticancer Res"},{"key":"2103_CR46","doi-asserted-by":"publisher","first-page":"104195","DOI":"10.1016\/j.ijmedinf.2020.104195","volume":"141","author":"M Lu","year":"2020","unstructured":"Lu M, Fan Z, Xu B, et al. Using machine learning to predict Ovarian cancer. Int J Med Inform. 2020;141:104195.","journal-title":"Int J Med Inform"},{"key":"2103_CR47","doi-asserted-by":"publisher","first-page":"2376","DOI":"10.1007\/s00261-020-02668-3","volume":"46","author":"H Park","year":"2021","unstructured":"Park H, Qin L, Guerra P, et al. Decoding incidental ovarian lesions: use of texture analysis and machine learning for characterization and detection of malignancy. Abdom Radiol (NY). 2021;46:2376\u2013383.","journal-title":"Abdom Radiol (NY)"},{"key":"2103_CR48","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1097\/IGC.0b013e3182000528","volume":"21","author":"E Vaes","year":"2011","unstructured":"Vaes E, Manchanda R, Nir R, et al. Mathematical models to discriminate between benign and malignant adnexal masses: potential diagnostic improvement using ovarian HistoScanning. Int J Gynecol Cancer. 2011;21:35\u201343.","journal-title":"Int J Gynecol Cancer"},{"key":"2103_CR49","doi-asserted-by":"publisher","first-page":"1078","DOI":"10.1111\/j.1471-0528.1999.tb08117.x","volume":"106","author":"RD Clayton","year":"1999","unstructured":"Clayton RD, Snowden S, Weston MJ, et al. Neural networks in the diagnosis of malignant ovarian tumours. Br J Obstet Gynaecol. 1999;106:1078\u201382.","journal-title":"Br J Obstet Gynaecol"},{"key":"2103_CR50","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1016\/S0933-3657(03)00051-4","volume":"28","author":"C Lu","year":"2003","unstructured":"Lu C, Van Gestel T, Suykens JAK, et al. Preoperative prediction of malignancy of ovarian tumors using least squares support vector machines. Artif Intell Med. 2003;28:281\u2013306.","journal-title":"Artif Intell Med"},{"key":"2103_CR51","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1111\/j.1525-1438.2006.00277.x","volume":"16","author":"R Moszynski","year":"2006","unstructured":"Moszynski R, Szpurek D, Smolen A, et al. Comparison of diagnostic usefulness of predictive models in preliminary differentiation of adnexal masses. Int J Gynecol Cancer. 2006;16:45\u201351.","journal-title":"Int J Gynecol Cancer"},{"key":"2103_CR52","doi-asserted-by":"publisher","first-page":"e201900115","DOI":"10.1002\/jbio.201900115","volume":"12","author":"Y Zeng","year":"2019","unstructured":"Zeng Y, Nandy S, Rao B, et al. Histogram analysis of en face scattering coefficient map predicts malignancy in human ovarian tissue. J Biophotonics. 2019;12:e201900115.","journal-title":"J Biophotonics"},{"key":"2103_CR53","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1007\/s10994-021-05946-3","volume":"110","author":"E H\u00fcllermeier","year":"2021","unstructured":"H\u00fcllermeier E, Waegeman W. Aleatory and epistemic uncertainty in machine learning: an introduction to concepts and methods. Mach Learn. 2021;110:457\u2013506.","journal-title":"Mach Learn"},{"key":"2103_CR54","doi-asserted-by":"publisher","first-page":"1723","DOI":"10.1177\/0962280216668555","volume":"27","author":"L Wynants","year":"2018","unstructured":"Wynants L, Vergouwe Y, Van Huffel S, et al. Does ignoring clustering in multicenter data influence the performance of prediction models? A simulation study. Stat Methods Med Res. 2018;27:1723\u201336.","journal-title":"Stat Methods Med Res"},{"key":"2103_CR55","doi-asserted-by":"publisher","first-page":"m441","DOI":"10.1136\/bmj.m441","volume":"368","author":"RD Riley","year":"2020","unstructured":"Riley RD, Ensor J, Snell KIE, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020;368:m441.","journal-title":"BMJ"},{"key":"2103_CR56","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1177\/09622802231151220","volume":"32","author":"A Pate","year":"2023","unstructured":"Pate A, Riley RD, Collins GS, et al. Minimum sample size for developing a multivariable prediction model using multinomial logistic regression. Stat Methods Med Res. 2023;32:555\u201371.","journal-title":"Stat Methods Med Res"},{"key":"2103_CR57","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1186\/1471-2288-14-137","volume":"14","author":"T Van der Ploeg","year":"2014","unstructured":"Van der Ploeg T, Austin PC, Steyerberg EW. Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Med Res Methodol. 2014;14:137.","journal-title":"BMC Med Res Methodol"},{"key":"2103_CR58","doi-asserted-by":"publisher","first-page":"821","DOI":"10.1177\/0272989X21996328","volume":"41","author":"C Bonner","year":"2021","unstructured":"Bonner C, Trevena LJ, Gaissmaier W, et al. Current best practice for presenting probabilities in patient decision Aids: fundamental principles. Med Decis Making. 2021;41:821\u201333.","journal-title":"Med Decis Making"},{"key":"2103_CR59","unstructured":"Liu JZ, Padhy S, Ren J et al. A simple approach to improve single-model deep uncertainty via distance-awareness. arXiv. 2022;2205.00403. https:\/\/arxiv.org\/abs\/2205.00403."},{"key":"2103_CR60","unstructured":"Thomassen D, le Cessie S, van Houwelingen H, Steyerberg E. Effective sample size: a measure of individual uncertainty in predictions. arXiv. 2023;2309.09824. https:\/\/arxiv.org\/abs\/2309.09824."}],"container-title":["BMC Medical Research Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-023-02103-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12874-023-02103-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-023-02103-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T05:02:42Z","timestamp":1700802162000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedresmethodol.biomedcentral.com\/articles\/10.1186\/s12874-023-02103-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,24]]},"references-count":60,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2103"],"URL":"https:\/\/doi.org\/10.1186\/s12874-023-02103-3","relation":{},"ISSN":["1471-2288"],"issn-type":[{"value":"1471-2288","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,24]]},"assertion":[{"value":"9 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 November 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 November 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"LV reported receiving grants from the Swedish Research Council, Malm\u00f6 University Hospital and Sk\u00e5ne University Hospital, Allm\u00e4nna Sjukhusets i Malm\u00f6 Stiftelse f\u00f6r bek\u00e4mpande av cancer (the Malm\u00f6 General Hospital Foundation for Fighting Against Cancer), Avtal om l\u00e4karutbildning och forskning (ALF)\u2013medel, and Landstingsfinansierad Regional Forskning during the conduct of the study; and teaching fees from Samsung outside the submitted work. DT and BVC reported receiving grants from the Research Foundation\u2013Flanders (FWO) and Internal Funds KU Leuven during the conduct of the study. TB reported receiving grants from NIHR Biomedical Research Centre, speaking honoraria and departmental funding from Samsung Healthcare and grants from Roche Diagnostics, Illumina, and Abbott. No other disclosures were reported. All other authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"All mother studies received ethics approval from the Research Ethics Committee of the University Hospitals Leuven and from each local ethics committee. All participants provided informed consent. All methods were carried out in accordance with relevant guidelines and regulations. We obtained approval from the Ethics Committee in Leuven (S64709) for secondary use of the data for methodological purposes.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"276"}}