{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T07:14:09Z","timestamp":1775027649006,"version":"3.50.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,4,18]],"date-time":"2023-04-18T00:00:00Z","timestamp":1681776000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,18]],"date-time":"2023-04-18T00:00:00Z","timestamp":1681776000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100013274","name":"Institut National d'assurance Maladie-Invalidit\u00e9","doi-asserted-by":"publisher","award":["CGV n\u00b0 2012\/235"],"award-info":[{"award-number":["CGV n\u00b0 2012\/235"]}],"id":[{"id":"10.13039\/501100013274","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003130","name":"Fonds Wetenschappelijk Onderzoek","doi-asserted-by":"publisher","award":["G067509N"],"award-info":[{"award-number":["G067509N"]}],"id":[{"id":"10.13039\/501100003130","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Early distinction between mild and serious infections (SI) is challenging in children in ambulatory care. Clinical prediction models (CPMs), developed to aid physicians in clinical decision-making, require broad external validation before clinical use. We aimed to externally validate four CPMs, developed in emergency departments, in ambulatory care.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We applied the CPMs in a prospective cohort of acutely ill children presenting to general practices, outpatient paediatric practices or emergency departments in Flanders, Belgium. For two multinomial regression models, Feverkidstool and Craig model, discriminative ability and calibration were assessed, and a model update was performed by re-estimation of coefficients with correction for overfitting. For two risk scores, the SBI score and PAWS, the diagnostic test accuracy was assessed.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>A total of 8211 children were included, comprising 498 SI and 276 serious bacterial infections (SBI). Feverkidstool had a C-statistic of 0.80 (95% confidence interval 0.77\u20130.84) with good calibration for pneumonia and 0.74 (0.70\u20130.79) with poor calibration for other SBI. The Craig model had a C-statistic of 0.80 (0.77\u20130.83) for pneumonia, 0.75 (0.70\u20130.80) for complicated urinary tract infections and 0.63 (0.39\u20130.88) for bacteraemia, with poor calibration. The model update resulted in improved C-statistics for all outcomes and good overall calibration for Feverkidstool and the Craig model. SBI score and PAWS performed extremely weak with sensitivities of 0.12 (0.09\u20130.15) and 0.32 (0.28\u20130.37).<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>Feverkidstool and the Craig model show good discriminative ability for predicting SBI and a potential for early recognition of SBI, confirming good external validity in a low prevalence setting of SBI. The SBI score and PAWS showed poor diagnostic performance.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Trial registration<\/jats:title>\n                <jats:p>ClinicalTrials.gov, NCT02024282. Registered on 31 December 2013.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12916-023-02860-4","type":"journal-article","created":{"date-parts":[[2023,4,18]],"date-time":"2023-04-18T12:03:37Z","timestamp":1681819417000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Clinical prediction models for serious infections in children: external validation in ambulatory care"],"prefix":"10.1186","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1065-0934","authenticated-orcid":false,"given":"David A. G.","family":"Bos","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tine","family":"De Burghgraeve","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"An","family":"De Sutter","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Frank","family":"Buntinx","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan Y.","family":"Verbakel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,4,18]]},"reference":[{"key":"2860_CR1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1471-2296-7-23","volume":"7","author":"A van den Bruel","year":"2006","unstructured":"van den Bruel A, Bartholomeeusen S, Aertgeerts B, Truyers C, Buntinx F. Serious infections in children: an incidence study in family practice. BMC Fam Pract. 2006;7:1\u20139.","journal-title":"BMC Fam Pract"},{"key":"2860_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1136\/bmjopen-2015-008657","volume":"5","author":"JY Verbakel","year":"2015","unstructured":"Verbakel JY, Lemiengre MB, de Burghgraeve T, de Sutter A, Aertgeerts B, Bullens DMA, et al. Validating a decision tree for serious infection: diagnostic accuracy in acutely ill children in ambulatory care. BMJ Open. 2015;5:1\u20138.","journal-title":"BMJ Open"},{"key":"2860_CR3","first-page":"1015","volume":"340","author":"JC Craig","year":"2010","unstructured":"Craig JC, Williams GJ, Jones M, Codarini M, Macaskill P, Hayen A, et al. The accuracy of clinical symptoms and signs for the diagnosis of serious bacterial infection in young febrile children: prospective cohort study of 15 781 febrile illnesses. BMJ (Online). 2010;340:1015.","journal-title":"BMJ (Online)"},{"key":"2860_CR4","first-page":"1","volume":"346","author":"RG Nijman","year":"2013","unstructured":"Nijman RG, Vergouwe Y, Thompson M, van Veen M, van Meurs AHJ, van der Lei J, et al. Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study. BMJ (Online). 2013;346:1\u201316.","journal-title":"BMJ (Online)"},{"key":"2860_CR5","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1136\/adc.2010.183111","volume":"96","author":"AJ Brent","year":"2011","unstructured":"Brent AJ, Lakhanpaul M, Thompson M, Collier J, Ray S, Ninis N, et al. Risk score to stratify children with suspected serious bacterial infection: observational cohort study. Arch Dis Child. 2011;96:361\u20137.","journal-title":"Arch Dis Child"},{"key":"2860_CR6","first-page":"538","volume":"57","author":"A van den Bruel","year":"2007","unstructured":"van den Bruel A, Aertgeerts B, Bruyninckx R, Aerts M, Buntinx F. Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary care. Br J Gen Pract. 2007;57:538\u201346.","journal-title":"Br J Gen Pract"},{"key":"2860_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1471-2431-14-246","volume":"14","author":"M Lemiengre","year":"2014","unstructured":"Lemiengre M, Verbakel J, Burghgraeve T, Aertgeerts B, de Baets F, Buntinx F, et al. Optimizing antibiotic prescribing for acutely ill children in primary care (ERNIE2 study protocol, part b): a cluster randomized, Factorial controlled trial evaluating the effect of a point-of-care C-reactive protein test and a brief intervention combined. BMC Pediatr. 2014;14:1\u20139.","journal-title":"BMC Pediatr"},{"key":"2860_CR8","doi-asserted-by":"publisher","first-page":"745","DOI":"10.1136\/emj.2007.054965","volume":"25","author":"P Egdell","year":"2008","unstructured":"Egdell P, Finlay L, Pedley DK. The PAWS score: validation of an early warning scoring system for the initial assessment of children in the emergency department. Emerg Med J. 2008;25:745\u20139.","journal-title":"Emerg Med J"},{"key":"2860_CR9","doi-asserted-by":"publisher","first-page":"826","DOI":"10.1016\/S0895-4356(03)00207-5","volume":"56","author":"SE Bleeker","year":"2003","unstructured":"Bleeker SE, Moll HA, Steyerberg EW, Donders ART, Derksen-Lubsen G, Grobbee DE, et al. External validation is necessary in prediction research: a clinical example. J Clin Epidemiol. 2003;56:826\u201332.","journal-title":"J Clin Epidemiol"},{"key":"2860_CR10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12916-019-1466-7","volume":"17","author":"B van Calster","year":"2019","unstructured":"van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW, Bossuyt P, et al. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17:1\u20137.","journal-title":"BMC Med"},{"key":"2860_CR11","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1186\/1472-6947-11-62","volume":"11","author":"E Wallace","year":"2011","unstructured":"Wallace E, Smith SM, Perera-Salazar R, Vaucher P, McCowan C, Collins G, et al. Framework for the impact analysis and implementation of clinical prediction rules (CPRs). BMC Med Inform Decis Mak. 2011;11:62.","journal-title":"BMC Med Inform Decis Mak"},{"key":"2860_CR12","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1186\/1741-7015-11-10","volume":"11","author":"JY Verbakel","year":"2013","unstructured":"Verbakel JY, van den Bruel A, Thompson M, Stevens R, Aertgeerts B, Oostenbrink R, et al. How well do clinical prediction rules perform in identifying serious infections in acutely ill children across an international network of ambulatory care datasets? BMC Med. 2013;11:10.","journal-title":"BMC Med"},{"key":"2860_CR13","first-page":"1033","volume":"103","author":"E de Vos-Kerkhof","year":"2018","unstructured":"de Vos-Kerkhof E, Gomez B, Milcent K, Steyerberg EW, Nijman RG, Smit FJ, et al. Clinical prediction models for young febrile infants at the emergency department: an international validation study. Arch Dis Child. 2018;103:1033\u201341.","journal-title":"Arch Dis Child"},{"key":"2860_CR14","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1038\/pr.2017.216","volume":"83","author":"RG Nijman","year":"2018","unstructured":"Nijman RG, Vergouwe Y, Moll HA, Smit FJ, Weerkamp F, Steyerberg EW, et al. Validation of the Feverkidstool and procalcitonin for detecting serious bacterial infections in febrile children. Pediatr Res. 2018;83:466\u201376.","journal-title":"Pediatr Res"},{"issue":"1","key":"2860_CR15","doi-asserted-by":"publisher","first-page":"e000416","DOI":"10.1136\/bmjpo-2018-000416","volume":"3","author":"C van Houten","year":"2019","unstructured":"van Houten C, van de Maat JS, Naaktgeboren C, Bont L, Oostenbrink R. Update of a clinical prediction model for serious bacterial infections in preschool children by adding a host-protein-based assay: a diagnostic study. BMJ Paediatr Open. 2019;3(1):e000416.","journal-title":"BMJ Paediatr Open"},{"issue":"2","key":"2860_CR16","doi-asserted-by":"publisher","first-page":"e20162853","DOI":"10.1542\/peds.2016-2853","volume":"140","author":"AD Irwin","year":"2017","unstructured":"Irwin AD, Grant A, Williams R, Kolamunnage-Dona R, Drew RJ, Paulus S, et al. Predicting risk of serious bacterial infections in febrile children in the emergency department. Pediatrics. 2017;140(2):e20162853.","journal-title":"Pediatrics"},{"issue":"5","key":"2860_CR17","doi-asserted-by":"publisher","first-page":"e0127620","DOI":"10.1371\/journal.pone.0127620","volume":"10","author":"E de Vos-Kerkhof","year":"2015","unstructured":"de Vos-Kerkhof E, Nijman RG, Vergouwe Y, Polinder S, Steyerberg EW, van der Lei J, et al. Impact of a clinical decision model for febrile children at risk for serious bacterial infections at the emergency department: a randomized controlled trial. PLoS One. 2015;10(5):e0127620.","journal-title":"PLoS One"},{"key":"2860_CR18","doi-asserted-by":"publisher","unstructured":"Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med. 2015;13(1). https:\/\/doi.org\/10.1186\/s12916-014-0241-z.","DOI":"10.1186\/s12916-014-0241-z"},{"key":"2860_CR19","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1186\/1471-2431-14-207","volume":"14","author":"J Verbakel","year":"2014","unstructured":"Verbakel J, Lemiengre M, Burghgraeve T, de Sutter A, Bullens D, Aertgeerts B, et al. Diagnosing serious infections in acutely ill children in ambulatory care (ERNIE 2 study protocol, part A): diagnostic accuracy of a clinical decision tree and added value of a point-of-care C-reactive protein test and oxygen saturation. BMC Pediatr. 2014;14:207.","journal-title":"BMC Pediatr"},{"key":"2860_CR20","doi-asserted-by":"crossref","unstructured":"Steyerberg EW. Updating for a New Setting. In: Gail M, Samet JM, Singer B, editors. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Second Edition. Cham (Switzerland): Springer Cham; 2019. p. 399-429.","DOI":"10.1007\/978-3-030-16399-0_20"},{"key":"2860_CR21","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/j.jclinepi.2015.12.005","volume":"74","author":"B Van Calster","year":"2016","unstructured":"Van Calster B, Nieboer D, Vergouwe Y, De Cock B, Pencina MJ, Steyerberg EW. A calibration hierarchy for risk models was defined: from utopia to empirical data. J Clin Epidemiol. 2016;74:167\u201376.","journal-title":"J Clin Epidemiol"},{"key":"2860_CR22","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1016\/j.jclinepi.2013.04.014","volume":"66","author":"K Van Hoorde","year":"2013","unstructured":"Van Hoorde K, Vergouwe Y, Timmerman D, Van Huffel S, Steyerberg EW, Van Calster B. Simple dichotomous updating methods improved the validity of polytomous prediction models. J Clin Epidemiol. 2013;66:1158\u201365.","journal-title":"J Clin Epidemiol"},{"key":"2860_CR23","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-19425-7","volume-title":"Regression modeling strategies","author":"FE Harrell","year":"2015","unstructured":"Harrell FE. Regression modeling strategies. 2nd ed. Cham: Springer International Publishing; 2015.","edition":"2"},{"key":"2860_CR24","doi-asserted-by":"publisher","first-page":"2585","DOI":"10.1002\/sim.6114","volume":"33","author":"K Van Hoorde","year":"2014","unstructured":"Van Hoorde K, Vergouwe Y, Timmerman D, Van Huffel S, Steyerberg EW, Van Calster B. Assessing calibration of multinomial risk prediction models. Stat Med. 2014;33:2585\u201396.","journal-title":"Stat Med"},{"key":"2860_CR25","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1186\/s41512-016-0002-x","volume":"1","author":"B Van Calster","year":"2017","unstructured":"Van Calster B, Van Hoorde K, Vergouwe Y, Bobdiwala S, Condous G, Kirk E, et al. Validation and updating of risk models based on multinomial logistic regression. Diagn Progn Res. 2017;1:2.","journal-title":"Diagn Progn Res"},{"key":"2860_CR26","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-21706-2","volume-title":"Modern applied statistics with S-PLUS","author":"WN Venables","year":"2002","unstructured":"Venables WN, Ripley BD. Modern applied statistics with S-PLUS. New York: Springer, New York; 2002."},{"key":"2860_CR27","volume-title":"R: a language and environment for statistical computing","author":"R core team","year":"2020","unstructured":"R core team. R: a language and environment for statistical computing. 2020."},{"key":"2860_CR28","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1186\/1471-2105-12-77","volume":"12","author":"X Robin","year":"2011","unstructured":"Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, et al. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.","journal-title":"BMC Bioinformatics"},{"key":"2860_CR29","doi-asserted-by":"crossref","unstructured":"van Buuren S. Flexible imputation of missing data. second edition.\u00a0Boca Raton: CRC press; 2018.","DOI":"10.1201\/9780429492259"},{"issue":"1","key":"2860_CR30","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1186\/s12879-019-4371-y","volume":"19","author":"K Keitel","year":"2019","unstructured":"Keitel K, Kilowoko M, Kyungu E, Genton B, D\u2019Acremont V. Performance of prediction rules and guidelines in detecting serious bacterial infections among Tanzanian febrile children. BMC Infect Dis. 2019;19(1):769.","journal-title":"BMC Infect Dis"},{"issue":"1","key":"2860_CR31","doi-asserted-by":"publisher","first-page":"e1003034","DOI":"10.1371\/journal.pmed.1003034","volume":"17","author":"JS van de Maat","year":"2020","unstructured":"van de Maat JS, Peeters D, Nieboer D, van Wermeskerken AM, Smit FJ, Noordzij JG, et al. Evaluation of a clinical decision rule to guide antibiotic prescription in children with suspected lower respiratory tract infection in the Netherlands: a stepped-wedge cluster randomised trial. PLoS Med. 2020;17(1):e1003034.","journal-title":"PLoS Med"},{"issue":"11","key":"2860_CR32","doi-asserted-by":"publisher","first-page":"1026","DOI":"10.1097\/INF.0000000000002794","volume":"39","author":"J van de Maat","year":"2020","unstructured":"van de Maat J, van der Ven M, Driessen G, van Wermeskerken AM, Smit F, Noordzij J, et al. Cost study of a cluster randomized trial on a clinical decision rule guiding antibiotic treatment in children with suspected lower respiratory tract infections in the emergency department. Pediatr Infect Dis J. 2020;39(11):1026\u201331.","journal-title":"Pediatr Infect Dis J"},{"key":"2860_CR33","doi-asserted-by":"publisher","first-page":"563","DOI":"10.1007\/s00431-015-2667-5","volume":"175","author":"E de Vos-Kerkhof","year":"2016","unstructured":"de Vos-Kerkhof E, Roland D, de Bekker-Grob E, Oostenbrink R, Lakhanpaul M, Moll HA. Clinicians\u2019 overestimation of febrile child risk assessment. Eur J Pediatr. 2016;175:563\u201372.","journal-title":"Eur J Pediatr"}],"container-title":["BMC Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12916-023-02860-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12916-023-02860-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12916-023-02860-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,18]],"date-time":"2023-04-18T12:03:52Z","timestamp":1681819432000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedicine.biomedcentral.com\/articles\/10.1186\/s12916-023-02860-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,18]]},"references-count":33,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2860"],"URL":"https:\/\/doi.org\/10.1186\/s12916-023-02860-4","relation":{},"ISSN":["1741-7015"],"issn-type":[{"value":"1741-7015","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,18]]},"assertion":[{"value":"15 November 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 April 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 April 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study was approved by the Ethics Committee Research UZ\/KU Leuven under the reference S54664. Formal written consent was obtained for each child.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"151"}}