{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T04:44:28Z","timestamp":1764996268175,"version":"3.37.3"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T00:00:00Z","timestamp":1594252800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T00:00:00Z","timestamp":1594252800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Dutch Ministry of Health, Welfare and Sport | Rijksinstituut voor Volksgezondheid en Milieu","award":["318 804"],"award-info":[{"award-number":["318 804"]}]},{"name":"Erasmus MC and the Netherlands Lung Foundation","award":["4.2.14.063JO"],"award-info":[{"award-number":["4.2.14.063JO"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>There is a strong association between social deprivation and adverse perinatal health outcomes, but related risk factors receive little attention in current antenatal risk selection. To increase awareness of healthcare professionals for these risk factors, a model for antenatal risk surveillance and care was developed in The Netherlands, called the \u2018Rotterdam Reproductive Risk Reduction\u2019 (R4U) scorecard. The aim of this study was to validate the R4U-scorecard. This study was conducted using external, prospective data from thirty-two midwifery practices, and fifteen hospitals in The Netherlands. The main outcome measures were the discrimination of the prognostic models for the probability of a pregnant woman developing adverse pregnancy outcomes (babies born preterm or small for gestational age), and calibration. We performed cross-validation and updated the model using statistical re-estimation of all predictors. 1752 participants were included, of whom 282 (16%) had one of the predefined adverse outcomes. The discriminative value of the original scoring system was poor [area under the curve (AUC) of 0.58 (95% CI 0.53\u20130.64)]. The model showed moderate calibration. The updated R4U-scorecard showed good generalisability to the validation set but did not alter the predictive value [AUC 0.61 (95% CI 0.56\u20130.66)]. By using external data and by updating the prognostic model, we have provided a comprehensive evaluation of the R4U-scorecard. Further improvement in classification of high-risk pregnancies is important considering the necessity of early risk detection for healthcare professionals to take appropriate actions to prevent these risks from becoming manifest problems.<\/jats:p>","DOI":"10.1038\/s41598-020-68101-3","type":"journal-article","created":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T10:06:38Z","timestamp":1594289198000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Validation of a prognostic model for adverse perinatal health outcomes"],"prefix":"10.1038","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7178-8910","authenticated-orcid":false,"given":"Jacqueline","family":"Lagendijk","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ewout W.","family":"Steyerberg","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Leonie A.","family":"Daalderop","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jasper V.","family":"Been","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eric A. P.","family":"Steegers","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anke G.","family":"Posthumus","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,7,9]]},"reference":[{"key":"68101_CR1","doi-asserted-by":"publisher","first-page":"1661","DOI":"10.1016\/S0140-6736(08)61690-6","volume":"372","author":"M Marmot","year":"2008","unstructured":"Marmot, M. et al. Closing the gap in a generation: health equity through action on the social determinants of health. Lancet 372, 1661\u20131669 (2008).","journal-title":"Lancet"},{"key":"68101_CR2","doi-asserted-by":"publisher","first-page":"64","DOI":"10.3109\/17538157.2010.492923","volume":"35","author":"M Gissler","year":"2010","unstructured":"Gissler, M. et al. Perinatal health monitoring in Europe: results from the EURO-PERISTAT project. Inform Health Soc Care 35, 64\u201379 (2010).","journal-title":"Inform Health Soc Care"},{"key":"68101_CR3","doi-asserted-by":"publisher","first-page":"727","DOI":"10.1111\/aogs.12430","volume":"93","author":"AA Vos","year":"2014","unstructured":"Vos, A. A., Posthumus, A. G., Bonsel, G. J., Steegers, E. A. & Denktas, S. Deprived neighborhoods and adverse perinatal outcome: a systematic review and meta-analysis. Acta Obstet. Gynecol. Scand 93, 727\u2013740 (2014).","journal-title":"Acta Obstet. Gynecol. Scand"},{"key":"68101_CR4","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1111\/j.1471-0528.2007.01582.x","volume":"115","author":"R Gray","year":"2008","unstructured":"Gray, R. et al. Social inequalities in preterm birth in Scotland 1980\u20132003: findings from an area-based measure of deprivation. BJOG 115, 82\u201390 (2008).","journal-title":"BJOG"},{"key":"68101_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1136\/bmjopen-2012-000964","volume":"2","author":"AL Weightman","year":"2012","unstructured":"Weightman, A. L. et al. Social inequality and infant health in the UK: systematic review and meta-analyses. BMJ Open 2, 1 (2012).","journal-title":"BMJ Open"},{"key":"68101_CR6","unstructured":"zorgverzekeringen, C. v. VERLOSKUNDIG VADEMECUM 2003 'eindrapport van de Commissie Verloskunde van het College voor zorgverzekeringen'. (2003)."},{"key":"68101_CR7","doi-asserted-by":"publisher","first-page":"579","DOI":"10.1007\/s10654-018-0387-7","volume":"33","author":"J Lagendijk","year":"2018","unstructured":"Lagendijk, J. et al. Antenatal non-medical risk assessment and care pathways to improve pregnancy outcomes: a cluster randomised controlled trial. Eur. J. Epidemiol. 33, 579\u2013589. https:\/\/doi.org\/10.1007\/s10654-018-0387-7 (2018).","journal-title":"Eur. J. Epidemiol."},{"key":"68101_CR8","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/1745-6215-16-8","volume":"16","author":"AA Vos","year":"2015","unstructured":"Vos, A. A. et al. Effectiveness of score card-based antenatal risk selection, care pathways, and multidisciplinary consultation in the Healthy Pregnancy 4 All study (HP4ALL): study protocol for a cluster randomized controlled trial. Trials 16, 8 (2015).","journal-title":"Trials"},{"key":"68101_CR9","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1186\/1471-2393-14-253","volume":"14","author":"S Denktas","year":"2014","unstructured":"Denktas, S. et al. Design and outline of the Healthy Pregnancy 4 All study. BMC Pregnancy Childbirth 14, 253 (2014).","journal-title":"BMC Pregnancy Childbirth"},{"key":"68101_CR10","doi-asserted-by":"publisher","first-page":"e002","DOI":"10.5334\/ijic.1512","volume":"15","author":"AA Vos","year":"2015","unstructured":"Vos, A. A. et al. An instrument for broadened risk assessment in antenatal health care including non-medical issues. Int J Integr Care 15, e002 (2015).","journal-title":"Int J Integr Care"},{"key":"68101_CR11","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/j.jclinepi.2015.04.005","volume":"69","author":"EW Steyerberg","year":"2016","unstructured":"Steyerberg, E. W. & Harrell, F. E. Jr. Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 69, 245\u2013247 (2016).","journal-title":"J Clin Epidemiol"},{"key":"68101_CR12","doi-asserted-by":"publisher","first-page":"515","DOI":"10.7326\/0003-4819-130-6-199903160-00016","volume":"130","author":"AC Justice","year":"1999","unstructured":"Justice, A. C., Covinsky, K. E. & Berlin, J. A. Assessing the generalizability of prognostic information. Ann Intern Med 130, 515\u2013524 (1999).","journal-title":"Ann Intern Med"},{"key":"68101_CR13","doi-asserted-by":"publisher","first-page":"2567","DOI":"10.1002\/sim.1844","volume":"23","author":"EW Steyerberg","year":"2004","unstructured":"Steyerberg, E. W., Borsboom, G. J., van Houwelingen, H. C., Eijkemans, M. J. & Habbema, J. D. Validation and updating of predictive logistic regression models: a study on sample size and shrinkage. Stat Med 23, 2567\u20132586 (2004).","journal-title":"Stat Med"},{"key":"68101_CR14","doi-asserted-by":"publisher","first-page":"1085","DOI":"10.1016\/j.jclinepi.2008.04.008","volume":"61","author":"DB Toll","year":"2008","unstructured":"Toll, D. B., Janssen, K. J., Vergouwe, Y. & Moons, K. G. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol 61, 1085\u20131094 (2008).","journal-title":"J Clin Epidemiol"},{"key":"68101_CR15","doi-asserted-by":"publisher","first-page":"2","DOI":"10.2188\/jea.JE20110116","volume":"22","author":"N Kondo","year":"2012","unstructured":"Kondo, N. Socioeconomic disparities and health: impacts and pathways. J Epidemiol 22, 2\u20136 (2012).","journal-title":"J Epidemiol"},{"key":"68101_CR16","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1038\/pr.2014.122","volume":"76","author":"D Pillas","year":"2014","unstructured":"Pillas, D. et al. Social inequalities in early childhood health and development: a European-wide systematic review. Pediatr Res 76, 418\u2013424 (2014).","journal-title":"Pediatr Res"},{"key":"68101_CR17","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1186\/s12939-015-0263-y","volume":"14","author":"L Chauvel","year":"2015","unstructured":"Chauvel, L. & Leist, A. K. Socioeconomic hierarchy and health gradient in Europe: the role of income inequality and of social origins. Int J Equity Health 14, 132 (2015).","journal-title":"Int J Equity Health"},{"key":"68101_CR18","doi-asserted-by":"publisher","first-page":"1925","DOI":"10.1093\/eurheartj\/ehu207","volume":"35","author":"EW Steyerberg","year":"2014","unstructured":"Steyerberg, E. W. & Vergouwe, Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J 35, 1925\u20131931 (2014).","journal-title":"Eur Heart J"},{"key":"68101_CR19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1136\/bmjopen-2014-006284","volume":"5","author":"SF van Voorst","year":"2015","unstructured":"van Voorst, S. F. et al. Effectiveness of general preconception care accompanied by a recruitment approach: protocol of a community-based cohort study (the Healthy Pregnancy 4 All study). BMJ Open 5, 1 (2015).","journal-title":"BMJ Open"},{"key":"68101_CR20","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/j.midw.2014.08.002","volume":"31","author":"MJ van Veen","year":"2015","unstructured":"van Veen, M. J. et al. Feasibility and reliability of a newly developed antenatal risk score card in routine care. Midwifery 31, 147\u2013154 (2015).","journal-title":"Midwifery"},{"key":"68101_CR21","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1186\/1472-6947-10-45","volume":"10","author":"G Cevenini","year":"2010","unstructured":"Cevenini, G. & Barbini, P. A bootstrap approach for assessing the uncertainty of outcome probabilities when using a scoring system. BMC Med Inform Decis Mak 10, 45 (2010).","journal-title":"BMC Med Inform Decis Mak"},{"key":"68101_CR22","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/j.midw.2015.11.006","volume":"38","author":"AG Posthumus","year":"2016","unstructured":"Posthumus, A. G., Birnie, E., van Veen, M. J., Steegers, E. A. & Bonsel, G. J. An antenatal prediction model for adverse birth outcomes in an urban population: the contribution of medical and non-medical risks. Midwifery 38, 78\u201386 (2016).","journal-title":"Midwifery"},{"key":"68101_CR23","doi-asserted-by":"publisher","first-page":"737","DOI":"10.1016\/j.earlhumdev.2009.09.008","volume":"85","author":"GH Visser","year":"2009","unstructured":"Visser, G. H., Eilers, P. H., Elferink-Stinkens, P. M., Merkus, H. M. & Wit, J. M. New Dutch reference curves for birthweight by gestational age. Early Hum. Dev. 85, 737\u2013744 (2009).","journal-title":"Early Hum. Dev."},{"key":"68101_CR24","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1186\/1471-2288-10-81","volume":"10","author":"D Vergouw","year":"2010","unstructured":"Vergouw, D. et al. The search for stable prognostic models in multiple imputed data sets. BMC Med. Res. Methodol. 10, 81\u201381. https:\/\/doi.org\/10.1186\/1471-2288-10-81 (2010).","journal-title":"BMC Med. Res. Methodol."},{"key":"68101_CR25","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1002\/mpr.329","volume":"20","author":"MJ Azur","year":"2011","unstructured":"Azur, M. J., Stuart, E. A., Frangakis, C. & Leaf, P. J. Multiple imputation by chained equations: what is it and how does it work?. Int. J. Methods Psychiatr. Res. 20, 40\u201349. https:\/\/doi.org\/10.1002\/mpr.329 (2011).","journal-title":"Int. J. Methods Psychiatr. Res."},{"key":"68101_CR26","doi-asserted-by":"publisher","first-page":"928","DOI":"10.1161\/CIRCULATIONAHA.106.672402","volume":"115","author":"NR Cook","year":"2007","unstructured":"Cook, N. R. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 115, 928\u2013935 (2007).","journal-title":"Circulation"},{"key":"68101_CR27","doi-asserted-by":"publisher","first-page":"1043","DOI":"10.1080\/03610928008827941","volume":"9","author":"DW Hosmer","year":"1980","unstructured":"Hosmer, D. W. & Lemesbow, S. Goodness of fit tests for the multiple logistic regression model. Commun. Stat. Theory Methods 9, 1043\u20131069. https:\/\/doi.org\/10.1080\/03610928008827941 (1980).","journal-title":"Commun. Stat. Theory Methods"},{"key":"68101_CR28","doi-asserted-by":"crossref","unstructured":"Frank E, H., Kerry L, L. E. E. & Daniel B, M. (1996). Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat. Med. 15, 361\u2013387, doi:10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4"},{"key":"68101_CR29","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.1002\/(SICI)1097-0258(20000430)19:8<1059::AID-SIM412>3.0.CO;2-0","volume":"19","author":"EW Steyerberg","year":"2000","unstructured":"Steyerberg, E. W., Eijkemans, M. J., Harrell, F. E. Jr. & Habbema, J. D. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Stat Med 19, 1059\u20131079 (2000).","journal-title":"Stat Med"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-68101-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-68101-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-68101-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,6]],"date-time":"2022-12-06T13:22:14Z","timestamp":1670332934000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-020-68101-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,9]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["68101"],"URL":"https:\/\/doi.org\/10.1038\/s41598-020-68101-3","relation":{},"ISSN":["2045-2322"],"issn-type":[{"type":"electronic","value":"2045-2322"}],"subject":[],"published":{"date-parts":[[2020,7,9]]},"assertion":[{"value":"5 September 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 June 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 July 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"11243"}}