{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T17:49:37Z","timestamp":1788198577748,"version":"build-2803163510"},"reference-count":46,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2017,9,3]],"date-time":"2017-09-03T00:00:00Z","timestamp":1504396800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/501100000024","name":"Canadian Institutes of Health Research","doi-asserted-by":"publisher","award":["MOP89377"],"award-info":[{"award-number":["MOP89377"]}],"id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Research Synthesis Methods"],"published-print":{"date-parts":[[2018,3]]},"abstract":"<jats:p>While the systematic review process is intended to maximize objectivity and limit researchers' biases, examples remain of discordant recommendations from meta\u2010analyses. Current guidelines to explore discrepancies assume the variation is produced by methodological differences and thus focus only on the study process. Because heterogeneity of interpretation also occurs when experts examine the same data, our purpose was to examine if there are <jats:italic>reasoning<\/jats:italic> differences, ie, in how information is processed and valued. We created simulated meta\u2010analyses based on <jats:italic>idealized<\/jats:italic> randomized studies (ie, perfect studies with no bias) to ensure differences in interpretations could only be due to reasoning. We recruited published meta\u2010analysts using purposeful variables. We conducted 3 audio\u2010recorded interviews per participant using structured and semi\u2010structured interviews, with paraphrasing and reflective listening to enhance and verify responses. Recruitment and analysis of transcripts and field notes followed the principles of grounded theory (eg, theoretical saturation, constant comparative analysis). Results show the complexity of meta\u2010analytic reasoning. At each step of the process, participants attempted to reconcile disparate forms of knowledge to determine a <jats:italic>right<\/jats:italic> answer (moral concern) and <jats:italic>accurately<\/jats:italic> draw a treatment effect (epistemological concern). The reasoning processes often shifted between considering the meta\u2010analysis as if the data were <jats:italic>whole<\/jats:italic>, and as if the data were discrete components (individual studies). These findings highlight paradigmatic tensions regarding the epistemological premises of meta\u2010analysis, resembling previous historical investigations of the functioning of scientific communities. In understanding why different meta\u2010analysts interpret data differently, it may be unrealistic to expect <jats:italic>objective<\/jats:italic> homogenous recommendations based on meta\u2010analyses.<\/jats:p>","DOI":"10.1002\/jrsm.1258","type":"journal-article","created":{"date-parts":[[2017,7,25]],"date-time":"2017-07-25T10:55:14Z","timestamp":1500980114000},"page":"25-40","source":"Crossref","is-referenced-by-count":3,"title":["Reconciling disparate data to determine the <i>right<\/i> answer: A grounded theory of meta analysts' reasoning in meta\u2010analysis"],"prefix":"10.1002","volume":"9","author":[{"given":"Lisa","family":"Chan","sequence":"first","affiliation":[{"name":"Ingram School of Nursing McGill University  Montreal Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mary Ellen","family":"Macdonald","sequence":"additional","affiliation":[{"name":"Division of Oral Health and Society Faculty of Dentistry McGill University  Montreal Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Franco A.","family":"Carnevale","sequence":"additional","affiliation":[{"name":"Ingram School of Nursing McGill University  Montreal Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Russell J.","family":"Steele","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics McGill University  Montreal Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9914-3498","authenticated-orcid":false,"given":"Ian","family":"Shrier","sequence":"additional","affiliation":[{"name":"Centre for Clinical Epidemiology, Lady Davis Institute, Jewish General Hospital McGill University  Montreal Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2017,9,3]]},"reference":[{"key":"e_1_2_6_2_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.323.7317.821"},{"key":"e_1_2_6_3_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.g3725"},{"key":"e_1_2_6_4_1","doi-asserted-by":"publisher","DOI":"10.1093\/ndt\/16.suppl_6.4"},{"key":"e_1_2_6_5_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.310.6987.1122"},{"key":"e_1_2_6_6_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.312.7023.71"},{"key":"e_1_2_6_7_1","doi-asserted-by":"crossref","DOI":"10.1002\/9780470693926","volume-title":"Rationale, Potentials, and Promise of Systematic Reviews. In: Systematic Reviews in Health Care: Meta\u2010analysis in Context","author":"Egger M","year":"2001"},{"key":"e_1_2_6_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(99)04149-5"},{"key":"e_1_2_6_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.4780060704"},{"key":"e_1_2_6_10_1","first-page":"1411","article-title":"A guide to interpreting discordant systematic reviews","volume":"156","author":"Jadad A","year":"1997","journal-title":"Can Med Assoc J"},{"key":"e_1_2_6_11_1","doi-asserted-by":"publisher","DOI":"10.1258\/jrsm.96.1.17"},{"key":"e_1_2_6_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0889-857X(03)00063-2"},{"key":"e_1_2_6_13_1","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6947-8-19"},{"key":"e_1_2_6_14_1","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6963-5-25"},{"key":"e_1_2_6_15_1","volume-title":"Action Science: Concepts, Methods and Skills for Research and Intervention","author":"Argyris C","year":"1985"},{"key":"e_1_2_6_16_1","volume-title":"Clinical Wisdom and Interventions in Critical Care: A Thinking\u2010Inaction Approach","author":"Benner P","year":"1999"},{"key":"e_1_2_6_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-2725-4_11"},{"key":"e_1_2_6_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1466-769X.2004.00175.x"},{"key":"e_1_2_6_19_1","volume-title":"The Reflective Practitioner: How Professionals Think in Action","author":"Schon D","year":"1983"},{"key":"e_1_2_6_20_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12910-015-0063-3"},{"key":"e_1_2_6_21_1","volume-title":"The Discovery of Grounded Theory: Strategies for Qualitative Research","author":"Glaser B","year":"2012"},{"key":"e_1_2_6_22_1","volume-title":"Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory","author":"Corbin J","year":"2014"},{"key":"e_1_2_6_23_1","doi-asserted-by":"publisher","DOI":"10.1093\/fampra\/13.6.522"},{"key":"e_1_2_6_24_1","unstructured":"RCore Team. (2015).A language and environment for statistical computing. Retrieved from Vienna Austria:http:\/\/www.r\u2010project.org\/"},{"key":"e_1_2_6_25_1","first-page":"33","volume-title":"Interpretive Social Science: A Second Look","author":"Taylor C","year":"1987"},{"key":"e_1_2_6_26_1","volume-title":"The Structure of Scientific Revolutions","author":"Kuhn TS","year":"1962"},{"key":"e_1_2_6_27_1","volume-title":"Genesis and Development of a Scientific Fact, first published by Benno Schwabe, Basel, 1935; English translation by Bardley F. and Trenn T.J., University of Chicago Press","author":"Fleck L","year":"1979"},{"key":"e_1_2_6_28_1","unstructured":"HackingI.Styles of scientific reasoning. In:RajchmanJ WestC eds. .New York:Postanalytic Philosophy;1985:145\u2010164."},{"key":"e_1_2_6_29_1","volume-title":"Francis Bacon: His Career and His Thought","author":"Anderson F","year":"1962"},{"key":"e_1_2_6_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/0197-2456(95)00134-4"},{"key":"e_1_2_6_31_1","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/2.4.463"},{"key":"e_1_2_6_32_1","doi-asserted-by":"publisher","DOI":"10.1093\/ije\/dyq248"},{"key":"e_1_2_6_33_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-985X.2008.00547.x"},{"key":"e_1_2_6_34_1","doi-asserted-by":"publisher","DOI":"10.1214\/088342304000000350"},{"key":"e_1_2_6_35_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.d5928"},{"key":"e_1_2_6_36_1","doi-asserted-by":"publisher","DOI":"10.1097\/00001648-199901000-00008"},{"key":"e_1_2_6_37_1","doi-asserted-by":"publisher","DOI":"10.1097\/01.ede.0000135174.63482.43"},{"key":"e_1_2_6_38_1","unstructured":"Savovi\u0107 J. Higgins J. Sterne J. Boutron I. &Hr\u00f3bjartsson A.(2015).Introducing a revised risk of bias tool for randomized trials (workshop). Paper presented at the 23rd Cochrane Colloquium."},{"key":"e_1_2_6_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-4696-1"},{"key":"e_1_2_6_40_1","volume-title":"The Reflective Practitioner: How Professionals Think in Action","author":"Sch\u00f6n DA","year":"1983"},{"key":"e_1_2_6_41_1","doi-asserted-by":"crossref","DOI":"10.4135\/9781446221792","volume-title":"Rethinking Psychology","author":"Smith JA","year":"1995"},{"key":"e_1_2_6_42_1","doi-asserted-by":"publisher","DOI":"10.1198\/016214505000000105"},{"key":"e_1_2_6_43_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9884.00113"},{"key":"e_1_2_6_44_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X03259817"},{"key":"e_1_2_6_45_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0266462303000138"},{"key":"e_1_2_6_46_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.2420"},{"key":"e_1_2_6_47_1","doi-asserted-by":"publisher","DOI":"10.1186\/2046-4053-3-37"}],"container-title":["Research Synthesis Methods"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fjrsm.1258","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/jrsm.1258","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,2]],"date-time":"2023-09-02T16:44:16Z","timestamp":1693673056000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jrsm.1258"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,9,3]]},"references-count":46,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2018,3]]}},"alternative-id":["10.1002\/jrsm.1258"],"URL":"https:\/\/doi.org\/10.1002\/jrsm.1258","archive":["Portico"],"relation":{},"ISSN":["1759-2879","1759-2887"],"issn-type":[{"value":"1759-2879","type":"print"},{"value":"1759-2887","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,9,3]]}}}