{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T19:26:09Z","timestamp":1792697169368,"version":"4.5.0"},"reference-count":53,"publisher":"SAGE Publications","issue":"11","license":[{"start":{"date-parts":[[2017,3,23]],"date-time":"2017-03-23T00:00:00Z","timestamp":1490227200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Stat Methods Med Res"],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:p>Funnel plots are graphical tools to assess and compare clinical performance of a group of care professionals or care institutions on a quality indicator against a benchmark. Incorrect construction of funnel plots may lead to erroneous assessment and incorrect decisions potentially with severe consequences. We provide workflow-based guidance for data analysts on constructing funnel plots for the evaluation of binary quality indicators, expressed as proportions, risk-adjusted rates or standardised rates. Our guidelines assume the following steps: (1) defining policy level input; (2) checking the quality of models used for case-mix correction; (3) examining whether the number of observations per hospital is sufficient; (4) testing for overdispersion of the values of the quality indicator; (5) testing whether the values of quality indicators are associated with institutional characteristics; and (6) specifying how the funnel plot should be constructed. We illustrate our guidelines using data from the Dutch National Intensive Care Evaluation registry. We expect that our guidelines will be useful to data analysts preparing funnel plots and to registries, or other organisations publishing quality indicators. This is particularly true if these people and organisations wish to use standard operating procedures when constructing funnel plots, perhaps to comply with the demands of certification.<\/jats:p>","DOI":"10.1177\/0962280217700169","type":"journal-article","created":{"date-parts":[[2017,3,23]],"date-time":"2017-03-23T03:10:19Z","timestamp":1490238619000},"page":"3350-3366","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":47,"title":["Guidelines on constructing funnel plots for quality indicators: A case study on mortality in intensive care unit patients"],"prefix":"10.1177","volume":"27","author":[{"given":"Ilona WM","family":"Verburg","sequence":"first","affiliation":[{"name":"Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands"},{"name":"Dutch National Intensive Care Registry (NICE), Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca","family":"Holman","sequence":"additional","affiliation":[{"name":"Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands"},{"name":"Dutch National Intensive Care Registry (NICE), Amsterdam, The Netherlands"},{"name":"Clinical Research Unit, Academic Medical Center, University of Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Niels","family":"Peek","sequence":"additional","affiliation":[{"name":"Health eResearch Centre, Division of Imaging, Informatics, and Data Science, University of Manchester, Manchester, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ameen","family":"Abu-Hanna","sequence":"additional","affiliation":[{"name":"Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicolette F","family":"de Keizer","sequence":"additional","affiliation":[{"name":"Academic Medical Center, University of Amsterdam, Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands"},{"name":"Dutch National Intensive Care Registry (NICE), Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2017,3,23]]},"reference":[{"key":"bibr1-0962280217700169","doi-asserted-by":"publisher","DOI":"10.2307\/2983325"},{"key":"bibr2-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejso.2012.10.007"},{"key":"bibr3-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1002\/sim.1970"},{"key":"bibr4-0962280217700169","unstructured":"Dutch National Intensive Care Evaluation (NICE) foundation. Data in beeld, http:\/\/www.stichting-nice.nl\/datainbeeld\/public (2014)."},{"key":"bibr5-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1111\/bjop.12081"},{"key":"bibr6-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1097\/SLA.0b013e31819a47b1"},{"key":"bibr7-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1093\/intqhc\/mzm060"},{"key":"bibr8-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1186\/1478-7954-9-58"},{"key":"bibr9-0962280217700169","author":"Few S","year":"2013","journal-title":"Visual Business Intelligence Newsletter"},{"key":"bibr10-0962280217700169","first-page":"37","volume":"94","author":"Grant SW","year":"2015","journal-title":"BMJ"},{"key":"bibr11-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1111\/j.1553-2712.2012.01304.x"},{"key":"bibr12-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1016\/j.bjoms.2014.08.020"},{"key":"bibr13-0962280217700169","unstructured":"Dutch National Intensive Care Evaluation (NICE) foundation, www.stichting-nice.nl (2014, accessed 13 March 2017)."},{"key":"bibr14-0962280217700169","unstructured":"Public Health Observatories (PHOs), http:\/\/www.apho.org.uk\/resource (2013, accessed 13 March 2017)."},{"key":"bibr15-0962280217700169","unstructured":"Australian and New Zealand Intensive Care Society, www.anzics.com.au\/ (2014, accessed 13 March 2017)."},{"key":"bibr16-0962280217700169","unstructured":"Canadian Institute for Health Information https:\/\/www.cihi.ca\/en (2015, accessed 13 March 2017)."},{"key":"bibr17-0962280217700169","unstructured":"Scottisch Intensive Care Society. www.scottishintensivecare.org.uk (2015, accessed 13 March 2017)."},{"key":"bibr18-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1097\/CCM.0b013e318232d7b0"},{"key":"bibr19-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1097\/00003246-198805000-00002"},{"key":"bibr20-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1097\/01.CCM.0000215112.84523.F0"},{"key":"bibr21-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1093\/ije\/dyv291"},{"key":"bibr22-0962280217700169","unstructured":"Stevens P, Pooley R. Essentials of state and activity diagrams. Using UML: software engineering with objects and components, United Kingdom: Addison-Wesley, 1999, pp. 239\u2013239."},{"key":"bibr23-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0045723"},{"key":"bibr24-0962280217700169","doi-asserted-by":"publisher","DOI":"10.2307\/2371837"},{"key":"bibr25-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1097\/CCM.0000000000001521"},{"key":"bibr26-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1097\/EDE.0b013e3181c30fb2"},{"key":"bibr27-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2015.12.005"},{"key":"bibr28-0962280217700169","doi-asserted-by":"publisher","DOI":"10.14219\/jada.archive.2012.0071"},{"key":"bibr29-0962280217700169","first-page":"239","volume":"236","author":"Mehdi T","year":"2011","journal-title":"Int J Public Health"},{"key":"bibr30-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1097\/MLR.0b013e31827ff0dc"},{"key":"bibr31-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2012-001689"},{"key":"bibr32-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-12-98"},{"key":"bibr33-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1136\/qshc.2005.013755"},{"key":"bibr34-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1992.10475225"},{"key":"bibr35-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-9473(98)00007-3"},{"key":"bibr36-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-14-25"},{"key":"bibr37-0962280217700169","first-page":"1457","volume":"63","author":"Pratt JW","year":"1968","journal-title":"J Am Stat Assoc"},{"key":"bibr38-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9876.2005.00489.x"},{"key":"bibr39-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-985X.2011.01010.x"},{"key":"bibr40-0962280217700169","unstructured":"Veroniki AS and Georgia (eds)\n                      Methods to estimate the heterogeneity variance, its uncertainty and to draw inference on the metaanalysis summary effect\n                      . Cochrane Colloquium; 2013 23\/09\/2013. Quebec: European Research Council."},{"key":"bibr41-0962280217700169","first-page":"261","volume":"30","author":"Viechtbauer W","year":"2005","journal-title":"Am Educ Res Assoc"},{"key":"bibr42-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1016\/0197-2456(86)90046-2"},{"key":"bibr43-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-9473(98)00007-3"},{"key":"bibr44-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1002\/sim.4780142408"},{"key":"bibr45-0962280217700169","unstructured":"Hartung J, Argac D and Makambi K. Homogeneity tests in meta-analysis. In:\n                      Meta-Analysis: new developments and applications in medical and social sciences\n                      , Hogrefe and Huber, USA, 2003, pp.3\u201320."},{"key":"bibr46-0962280217700169","first-page":"7","volume":"4","author":"Scrucca L","year":"2004","journal-title":"R News"},{"key":"bibr47-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1093\/intqhc\/mzv092"},{"key":"bibr48-0962280217700169","volume-title":"A language and environment for statistical computing. 2.15.1 ed","author":"Development Core Team R","year":"2005"},{"key":"bibr49-0962280217700169","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-130-12-199906150-00019"},{"key":"bibr50-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b3985"},{"key":"bibr51-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1080\/03610918.2013.781629"},{"key":"bibr52-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0109684"},{"key":"bibr53-0962280217700169","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2007.00593.x"}],"container-title":["Statistical Methods in Medical Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0962280217700169","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0962280217700169","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0962280217700169","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T23:49:57Z","timestamp":1777679397000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0962280217700169"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3,23]]},"references-count":53,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["10.1177\/0962280217700169"],"URL":"https:\/\/doi.org\/10.1177\/0962280217700169","relation":{},"ISSN":["0962-2802","1477-0334"],"issn-type":[{"value":"0962-2802","type":"print"},{"value":"1477-0334","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,3,23]]}}}