{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T16:57:48Z","timestamp":1790873868276,"version":"4.1.0"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,2,11]],"date-time":"2020-02-11T00:00:00Z","timestamp":1581379200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2020,2,11]],"date-time":"2020-02-11T00:00:00Z","timestamp":1581379200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000053","name":"National Eye Institute","doi-asserted-by":"crossref","award":["UG EY020522"],"award-info":[{"award-number":["UG EY020522"]}],"id":[{"id":"10.13039\/100000053","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Res Methodol"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review (\u201creview\u201d) does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review\u2019s pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review\u2019s outcomes and, using an algorithm, categorized one outcome as its \u201cmain\u201d outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0\u2013100%) and 28% (IQR 0\u201371%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21\/51 reviews) or the specific measurements for the outcome were inconsistent (16\/51 reviews).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12874-020-0898-2","type":"journal-article","created":{"date-parts":[[2020,2,11]],"date-time":"2020-02-11T12:05:41Z","timestamp":1581422741000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study"],"prefix":"10.1186","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2062-3382","authenticated-orcid":false,"given":"Ian J.","family":"Saldanha","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristina B.","family":"Lindsley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Money","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hannah J.","family":"Kimmel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bryant T.","family":"Smith","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kay","family":"Dickersin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,2,11]]},"reference":[{"key":"898_CR1","unstructured":"Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, DC: The National Academies Press; 2011."},{"issue":"2","key":"898_CR2","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1002\/jrsm.12","volume":"1","author":"M Borenstein","year":"2010","unstructured":"Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods. 2010;1(2):97\u2013111.","journal-title":"Res Synth Methods"},{"key":"898_CR3","volume-title":"Methodological Expectations of Cochrane Intervention Reviews. Version 1.02 ed","author":"JPT Higgins","year":"2016","unstructured":"Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews. Version 1.02 ed. London: Cochrane; 2016."},{"issue":"1","key":"898_CR4","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1016\/j.ophtha.2018.09.008","volume":"126","author":"IJ Saldanha","year":"2019","unstructured":"Saldanha IJ, Le JT, Solomon SD, Repka MX, Akpek EK, Li T. Choosing Core outcomes for use in clinical trials in ophthalmology: perspectives from three ophthalmology outcomes working groups. Ophthalmology. 2019;126(1):6\u20139.","journal-title":"Ophthalmology"},{"key":"898_CR5","doi-asserted-by":"crossref","unstructured":"Meinert CL. Clinical trials dictionary: terminology and usage recommendation. 2nd ed Hoboken, NJ: Wiley; 2012.","DOI":"10.1002\/9781118315293"},{"key":"898_CR6","volume-title":"Cochrane Handbook for Systematic Reviews of Interventions","year":"2019","unstructured":"Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions. 2nd Edition. Chichester: Wiley; 2019."},{"key":"898_CR7","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1016\/j.jclinepi.2017.07.014","volume":"91","author":"Evan Mayo-Wilson","year":"2017","unstructured":"Mayo-Wilson E, Li T, Fusco N, et al. Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy. J Clin Epidemiol. 2017;91:95\u2013110.","journal-title":"Journal of Clinical Epidemiology"},{"issue":"1","key":"898_CR8","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1002\/jrsm.1277","volume":"9","author":"Evan Mayo-Wilson","year":"2017","unstructured":"Mayo-Wilson E, Li T, Fusco N, Dickersin K. Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study). Res Synth Methods. 2018;9(1):2\u201312.","journal-title":"Research Synthesis Methods"},{"issue":"10","key":"898_CR9","doi-asserted-by":"publisher","first-page":"e109400","DOI":"10.1371\/journal.pone.0109400","volume":"9","author":"IJ Saldanha","year":"2014","unstructured":"Saldanha IJ, Dickersin K, Wang X, Li T. Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability. PLoS One. 2014;9(10):e109400.","journal-title":"PLoS One"},{"key":"898_CR10","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.jclinepi.2017.02.005","volume":"84","author":"IJ Saldanha","year":"2017","unstructured":"Saldanha IJ, Li T, Yang C, Owczarzak J, Williamson PR, Dickersin K. Clinical trials and systematic reviews addressing similar interventions for the same condition do not consider similar outcomes to be important: a case study in HIV\/AIDS. J Clin Epidemiol. 2017;84:85\u201394.","journal-title":"J Clin Epidemiol"},{"issue":"9","key":"898_CR11","doi-asserted-by":"publisher","first-page":"933","DOI":"10.1001\/jamaophthalmol.2017.2583","volume":"135","author":"IJ Saldanha","year":"2017","unstructured":"Saldanha IJ, Lindsley K, Do DV, et al. Comparison of clinical trial and systematic review outcomes for the 4 Most prevalent eye diseases. JAMA Ophthalmology. 2017;135(9):933\u201340.","journal-title":"JAMA Ophthalmology"},{"issue":"9","key":"898_CR12","doi-asserted-by":"publisher","first-page":"852","DOI":"10.1056\/NEJMsa1012065","volume":"364","author":"DA Zarin","year":"2011","unstructured":"Zarin DA, Tse T, Williams RJ, Califf RM, ide NC. The ClinicalTrials.gov results database--update and key issues. N Engl J Med. 2011;364(9):852\u201360.","journal-title":"N Engl J Med"},{"key":"898_CR13","first-page":"Cd010516","volume":"7","author":"VV Juthani","year":"2017","unstructured":"Juthani VV, Clearfield E, Chuck RS. Non-steroidal anti-inflammatory drugs versus corticosteroids for controlling inflammation after uncomplicated cataract surgery. Cochrane Database Syst Rev. 2017;7:Cd010516.","journal-title":"Cochrane Database Syst Rev"},{"key":"898_CR14","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/2046-4053-1-15","volume":"1","author":"S Ip","year":"2012","unstructured":"Ip S, Hadar N, Keefe S, et al. A web-based archive of systematic review data. Syst Rev. 2012;1:15.","journal-title":"Syst Rev"},{"issue":"4","key":"898_CR15","doi-asserted-by":"publisher","first-page":"287","DOI":"10.7326\/M14-1603","volume":"162","author":"T Li","year":"2015","unstructured":"Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162(4):287\u201394.","journal-title":"Ann Intern Med"},{"key":"898_CR16","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1186\/1745-6215-8-39","volume":"8","author":"M Clarke","year":"2007","unstructured":"Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39.","journal-title":"Trials"},{"key":"898_CR17","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1186\/1745-6215-13-132","volume":"13","author":"PR Williamson","year":"2012","unstructured":"Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.","journal-title":"Trials"},{"issue":"12","key":"898_CR18","doi-asserted-by":"publisher","first-page":"e0209869","DOI":"10.1371\/journal.pone.0209869","volume":"13","author":"E Gargon","year":"2018","unstructured":"Gargon E, Gorst SL, Harman NL, Smith V, Matvienko-Sikar K, Williamson PR. Choosing important health outcomes for comparative effectiveness research: 4th annual update to a systematic review of core outcome sets for research. PLoS One. 2018;13(12):e0209869.","journal-title":"PLoS One"},{"key":"898_CR19","unstructured":"Williamson PR, Oliveira RD, Clarke M, et al. The relevance and uptake of core outcome sets in Cochrane systematic reviews Submitted."},{"key":"898_CR20","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1186\/s13643-016-0188-6","volume":"5","author":"M Clarke","year":"2016","unstructured":"Clarke M, Williamson PR. Core outcome sets and systematic reviews. Syst Rev. 2016;5:11.","journal-title":"Syst Rev"},{"issue":"7","key":"898_CR21","doi-asserted-by":"publisher","first-page":"735","DOI":"10.1503\/cmaj.1041086","volume":"171","author":"AW Chan","year":"2004","unstructured":"Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171(7):735\u201340.","journal-title":"CMAJ"},{"issue":"6","key":"898_CR22","doi-asserted-by":"publisher","first-page":"e002709","DOI":"10.1136\/bmjopen-2013-002709","volume":"3","author":"Kerry Dwan","year":"2013","unstructured":"Dwan K, Kirkham JJ, Williamson PR, Gamble C. Selective reporting of outcomes in randomised controlled trials in systematic reviews of cystic fibrosis. BMJ Open. 2013;3(6).","journal-title":"BMJ Open"},{"key":"898_CR23","doi-asserted-by":"publisher","first-page":"k3802","DOI":"10.1136\/bmj.k3802","volume":"362","author":"JJ Kirkham","year":"2018","unstructured":"Kirkham JJ, Altman DG, Chan AW, Gamble C, Dwan KM, Williamson PR. Outcome reporting bias in trials: a methodological approach for assessment and adjustment in systematic reviews. BMJ. 2018;362:k3802.","journal-title":"BMJ"},{"key":"898_CR24","doi-asserted-by":"publisher","first-page":"c365","DOI":"10.1136\/bmj.c365","volume":"340","author":"JJ Kirkham","year":"2010","unstructured":"Kirkham JJ, Dwan KM, Altman DG, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.","journal-title":"BMJ"},{"issue":"20","key":"898_CR25","doi-asserted-by":"publisher","first-page":"2179","DOI":"10.1002\/sim.5356","volume":"31","author":"JJ Kirkham","year":"2012","unstructured":"Kirkham JJ, Riley RD, Williamson PR. A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews. Stat Med. 2012;31(20):2179\u201395.","journal-title":"Stat Med"},{"key":"898_CR26","doi-asserted-by":"crossref","unstructured":"Page MJ, McKenzie JE, Kirkham J, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev. 2014;(10):Mr000035.","DOI":"10.1002\/14651858.MR000035.pub2"},{"issue":"Suppl 3","key":"898_CR27","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1186\/s13063-017-1978-4","volume":"18","author":"PR Williamson","year":"2017","unstructured":"Williamson PR, Altman DG, Bagley H, et al. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280.","journal-title":"Trials"},{"issue":"9912","key":"898_CR28","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/S0140-6736(13)62229-1","volume":"383","author":"I Chalmers","year":"2014","unstructured":"Chalmers I, Bracken MB, Djulbegovic B, et al. How to increase value and reduce waste when research priorities are set. Lancet (London, England). 2014;383(9912):156\u201365.","journal-title":"Lancet (London, England)"},{"issue":"7","key":"898_CR29","doi-asserted-by":"publisher","first-page":"e102670","DOI":"10.1371\/journal.pone.0102670","volume":"9","author":"M Clarke","year":"2014","unstructured":"Clarke M, Brice A, Chalmers I. Accumulating research: a systematic account of how cumulative meta-analyses would have provided knowledge, improved health, reduced harm and saved resources. PLoS One. 2014;9(7):e102670.","journal-title":"PLoS One"},{"issue":"9912","key":"898_CR30","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/S0140-6736(13)62329-6","volume":"383","author":"MR Macleod","year":"2014","unstructured":"Macleod MR, Michie S, Roberts I, et al. Biomedical research: increasing value, reducing waste. Lancet (London, England). 2014;383(9912):101\u20134.","journal-title":"Lancet (London, England)"},{"issue":"10","key":"898_CR31","doi-asserted-by":"publisher","first-page":"1315","DOI":"10.1136\/bjophthalmol-2016-309695","volume":"101","author":"A Law","year":"2017","unstructured":"Law A, Lindsley K, Rouse B, Wormald R, Dickersin K, Li T. Missed opportunity from randomised controlled trials of medical interventions for open-angle glaucoma. Br J Ophthalmol. 2017;101(10):1315\u20137.","journal-title":"Br J Ophthalmol"},{"key":"898_CR32","doi-asserted-by":"publisher","first-page":"j3919","DOI":"10.1136\/bmj.j3919","volume":"358","author":"KM Dwan","year":"2017","unstructured":"Dwan KM, Williamson PR, Kirkham JJ. Do systematic reviews still exclude studies with \u201cno relevant outcome data\u201d? BMJ. 2017;358:j3919.","journal-title":"BMJ"},{"key":"898_CR33","doi-asserted-by":"publisher","first-page":"g6501","DOI":"10.1136\/bmj.g6501","volume":"349","author":"P Saini","year":"2014","unstructured":"Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014;349:g6501.","journal-title":"BMJ"}],"container-title":["BMC Medical Research Methodology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-020-0898-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12874-020-0898-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12874-020-0898-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,2,9]],"date-time":"2021-02-09T19:46:17Z","timestamp":1612899977000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedresmethodol.biomedcentral.com\/articles\/10.1186\/s12874-020-0898-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,11]]},"references-count":33,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["898"],"URL":"https:\/\/doi.org\/10.1186\/s12874-020-0898-2","relation":{},"ISSN":["1471-2288"],"issn-type":[{"value":"1471-2288","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,11]]},"assertion":[{"value":"22 August 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 January 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Drs. Saldanha and Dickersin and Ms. Lindsley and Money declare affiliation with Cochrane Eyes and Vision during conduct of the work related to this manuscript. Ms. Kimmel and Mr. Smith were research assistants on this study. All of the reviews examined in this manuscript were produced by Cochrane Eyes and Vision. No other disclosures are reported.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"30"}}