{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T16:03:53Z","timestamp":1781366633632,"version":"3.54.1"},"publisher-location":"New York, NY, USA","reference-count":50,"publisher":"ACM","license":[{"start":{"date-parts":[[2008,10,9]],"date-time":"2008-10-09T00:00:00Z","timestamp":1223510400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2008,10,9]]},"DOI":"10.1145\/1414004.1414034","type":"proceedings-article","created":{"date-parts":[[2008,10,14]],"date-time":"2008-10-14T17:49:52Z","timestamp":1224006592000},"page":"178-187","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":137,"title":["Strength of evidence in systematic reviews in software engineering"],"prefix":"10.1145","author":[{"given":"Tore","family":"Dyb\u00e5","sequence":"first","affiliation":[{"name":"SINTEF, Trondheim, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Torgeir","family":"Dings\u00f8yr","sequence":"additional","affiliation":[{"name":"SINTEF, Trondheim, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2008,10,9]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.17"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/REBSE.2007.4"},{"key":"e_1_3_2_1_3_1","first-page":"477","article-title":"Association between Industry Funding and Statistically Significant Pro-Industry Findings in Medical and Surgical Randomized Trials","volume":"170","author":"Bhandari M.","year":"2004","unstructured":"Bhandari , M. , Busse , J. W. , Jackowski , D. , Montori , V. M. , Sch\u00fcnemann , H. , Sprague , S. , Mears , D. , Schemitsch , E. H. , Heels-Ansdell , D. , and Devereaux , P. J. ( 2004 ) Association between Industry Funding and Statistically Significant Pro-Industry Findings in Medical and Surgical Randomized Trials , CMAJ , 170 : 477 -- 480 Bhandari, M., Busse, J. W., Jackowski, D., Montori, V. M., Sch\u00fcnemann, H., Sprague, S., Mears, D., Schemitsch, E. H., Heels-Ansdell, D., and Devereaux, P. J. (2004) Association between Industry Funding and Statistically Significant Pro-Industry Findings in Medical and Surgical Randomized Trials, CMAJ, 170: 477--480","journal-title":"CMAJ"},{"key":"e_1_3_2_1_4_1","volume-title":"Systematic Review in Software Engineering","author":"Biolchini J.","year":"2005","unstructured":"Biolchini , J. , Mian , P. G. , Natali , A. C. C. , and Travassos , G. H . ( 2005 ) Systematic Review in Software Engineering , Univ. Rio de Janeiro , TR , ES 679\/05. Biolchini, J., Mian, P. G., Natali, A. C. C., and Travassos, G. H. (2005) Systematic Review in Software Engineering, Univ. Rio de Janeiro, TR, ES 679\/05."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/0001-6918(80)90034-7"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2006.07.009"},{"key":"e_1_3_2_1_7_1","volume-title":"Experimental and Quasi-Experimental Designs for Research","author":"Campbell D. T.","year":"1963","unstructured":"Campbell , D. T. and Stanley , J. C . ( 1963 ) Experimental and Quasi-Experimental Designs for Research , Boston : Houghton Mifflin Company Campbell, D. T. and Stanley, J. C. (1963) Experimental and Quasi-Experimental Designs for Research, Boston: Houghton Mifflin Company"},{"key":"e_1_3_2_1_8_1","volume-title":"Quasi-Experimentation: Design &amp","author":"Cook T. D.","year":"1979","unstructured":"Cook , T. D. and Campbell , D. T . ( 1979 ) Quasi-Experimentation: Design &amp ; Analysis Issues for Field Settings, Boston : Houghton Mifflin Company Cook, T. D. and Campbell, D. T. (1979) Quasi-Experimentation: Design & Analysis Issues for Field Settings, Boston: Houghton Mifflin Company"},{"key":"e_1_3_2_1_9_1","volume-title":"Synthesizing Research","author":"Cooper H.","year":"1998","unstructured":"Cooper , H. ( 1998 ) Synthesizing Research ( 3 rd Ed.), Thousand Oaks, CA : Sage . Cooper, H. (1998) Synthesizing Research (3rd Ed.), Thousand Oaks, CA: Sage.","edition":"3"},{"key":"e_1_3_2_1_10_1","volume-title":"Handbook of Research Synthesis","author":"Cooper H.","year":"1994","unstructured":"Cooper , H. and Hedges , L. V. (Eds.) ( 1994 ) Handbook of Research Synthesis , New York : Russell Sage Foundation . Cooper, H. and Hedges, L. V. (Eds.) (1994) Handbook of Research Synthesis, New York: Russell Sage Foundation."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2006.17"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2007.39"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2007.02.009"},{"key":"e_1_3_2_1_14_1","volume-title":"J. of Health Services Research &amp","author":"Dixon-Woods M.","year":"2005","unstructured":"Dixon-Woods , M. , Agarwal , S. , Jones , D. , Young , B. , and Sutton , A . ( 2005 ) Synthesising Qualitative and Quantitative Evidence: A Review of Possible Methods , J. of Health Services Research &amp ; Policy , 10(1): 45--53. Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., and Sutton, A. (2005) Synthesising Qualitative and Quantitative Evidence: A Review of Possible Methods, J. of Health Services Research & Policy, 10(1): 45--53."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.01.006"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2007.21"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2005.08.009"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2005.6"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"crossref","DOI":"10.1002\/9780470693926","volume-title":"Systematic Reviews in Health Care: Meta-analysis in Context","author":"Egger M.","year":"2001","unstructured":"Egger , M. , Smith , G. D. , and Altman , D. G . ( 2001 ) Systematic Reviews in Health Care: Meta-analysis in Context ( 2 nd Ed.), London : BMJ Publishing Group . Egger, M., Smith, G. D., and Altman, D. G. (2001) Systematic Reviews in Health Care: Meta-analysis in Context (2nd Ed.), London: BMJ Publishing Group.","edition":"2"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.328.7454.1490"},{"key":"e_1_3_2_1_21_1","volume-title":"How to Read a Paper: The Basics of Evidence-Based Medicine","author":"Greenhalgh T.","year":"2006","unstructured":"Greenhalgh , T. ( 2006 ) How to Read a Paper: The Basics of Evidence-Based Medicine ( 3 rd Ed.), London : BMJ Publishing Group . Greenhalgh, T. (2006) How to Read a Paper: The Basics of Evidence-Based Medicine (3rd Ed.), London: BMJ Publishing Group.","edition":"3"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.12"},{"issue":"0","key":"e_1_3_2_1_23_1","first-page":"0","article-title":"Cochrane Handbook for Systematic Reviews of Interventions","volume":"5","author":"Higgins J. P. T.","year":"2008","unstructured":"Higgins J. P. T. and Green , S. (Eds.) ( 2008 ), Cochrane Handbook for Systematic Reviews of Interventions , Version 5 . 0 . 0 (updated February 2008), The Cochrane Collaboration, available from www.cochrane-handbook.org. Higgins J. P. T. and Green, S. (Eds.) (2008), Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0.0 (updated February 2008), The Cochrane Collaboration, available from www.cochrane-handbook.org.","journal-title":"Version"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2007.29"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/0197-2456(95)00134-4"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISESE.2005.1541818"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.3"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2007.02.015"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.04.006"},{"key":"e_1_3_2_1_30_1","unstructured":"Khan K. S. ter Riet G. Glanville J. Sowden A. J. and Kleijnen J. (Eds.) (2001) Undertaking Systematic Review of Research on Effectiveness CRD's Guidance for those Carrying Out or Commissioning Reviews CRD Report Number 4 (2nd Ed.) NHS Centre for Reviews and Dissemination University of York.  Khan K. S. ter Riet G. Glanville J. Sowden A. J. and Kleijnen J. (Eds.) (2001) Undertaking Systematic Review of Research on Effectiveness CRD's Guidance for those Carrying Out or Commissioning Reviews CRD Report Number 4 (2nd Ed.) NHS Centre for Reviews and Dissemination University of York."},{"key":"e_1_3_2_1_31_1","first-page":"25","volume-title":"Proceedings from Int. Workshop, Dagstuhl Castle","author":"Kitchenham B. A.","year":"2007","unstructured":"Kitchenham , B. A. ( 2007 ) Empirical Paradigm - The Role of Experiments, in V. R. Basili et al. (Eds.), Empirical Software Engineering Issues: Critical Assessment and Future Directions , Proceedings from Int. Workshop, Dagstuhl Castle , June 26-30, 2006, Lecture Notes in Compute Science 4336, Springer , pp. 25 -- 32 . Kitchenham, B. A. (2007) Empirical Paradigm - The Role of Experiments, in V. R. Basili et al. (Eds.), Empirical Software Engineering Issues: Critical Assessment and Future Directions, Proceedings from Int. Workshop, Dagstuhl Castle, June 26-30, 2006, Lecture Notes in Compute Science 4336, Springer, pp. 25--32."},{"key":"e_1_3_2_1_32_1","volume-title":"Version 2.3","author":"Kitchenham B. A.","year":"2007","unstructured":"Kitchenham , B. A. and Charters , S . ( 2007 ) Guidelines for performing Systematic Literature Reviews in Software Engineering , Version 2.3 , Keele University , EBSE Technical Report, EBSE-2007-01. Kitchenham, B. A. and Charters, S. (2007) Guidelines for performing Systematic Literature Reviews in Software Engineering, Version 2.3, Keele University, EBSE Technical Report, EBSE-2007-01."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-007-9053-5"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.5555\/998675.999432"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1027796"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.2307\/249410"},{"key":"e_1_3_2_1_37_1","series-title":"Lecture Notes on Empirical Software Engineering","first-page":"167","volume-title":"(Quasi-) Experimental Studies in Industrial Settings","author":"Laitenberger O.","year":"2003","unstructured":"Laitenberger , O. and Rombach , D . ( 2003 ) (Quasi-) Experimental Studies in Industrial Settings , in N. Juristo and A. M. Moreno (Eds.), Lecture Notes on Empirical Software Engineering , Singapore : World Scientific (Series on Software Engineering and Knowledge Engineering 12), pp. 167 -- 227 . Laitenberger, O. and Rombach, D. (2003) (Quasi-) Experimental Studies in Industrial Settings, in N. Juristo and A. M. Moreno (Eds.), Lecture Notes on Empirical Software Engineering, Singapore: World Scientific (Series on Software Engineering and Knowledge Engineering 12), pp. 167--227."},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.2307\/248698"},{"key":"e_1_3_2_1_39_1","volume-title":"Questioning Extreme Programming","author":"Mcbreen P.","year":"2003","unstructured":"Mcbreen , P. ( 2003 ) Questioning Extreme Programming , Boston, MA, USA : Pearson Education . Mcbreen, P. (2003) Questioning Extreme Programming, Boston, MA, USA: Pearson Education."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISESE.2005.1541857"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(00)04337-3"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(99)04149-5"},{"key":"e_1_3_2_1_43_1","volume-title":"Systematic Reviews: Synthesis of Best Evidence for Health Care Decisions","author":"Mulrow C.","year":"1998","unstructured":"Mulrow , C. and Cook , D. (Eds.) ( 1998 ) Systematic Reviews: Synthesis of Best Evidence for Health Care Decisions , Philadelphia : Am. College of Physicians . Mulrow, C. and Cook, D. (Eds.) (1998) Systematic Reviews: Synthesis of Best Evidence for Health Care Decisions, Philadelphia: Am. College of Physicians."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"crossref","DOI":"10.4135\/9781412985000","volume-title":"Meta-Ethnography: Synthesizing Qualitative Studies","author":"Noblit G. W.","year":"1988","unstructured":"Noblit , G. W. and Hare , R. D . ( 1988 ) Meta-Ethnography: Synthesizing Qualitative Studies , Thousand Oaks : Sage . Noblit, G. W. and Hare, R. D. (1988) Meta-Ethnography: Synthesizing Qualitative Studies, Thousand Oaks: Sage."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"crossref","DOI":"10.1002\/9780470754887","volume-title":"Systematic Reviews in the Social Sciences: A Practical Guide","author":"Petticrew M.","year":"2006","unstructured":"Petticrew , M. and Roberts , H . ( 2006 ) Systematic Reviews in the Social Sciences: A Practical Guide , Oxford, UK : Blackwell . Petticrew, M. and Roberts, H. (2006) Systematic Reviews in the Social Sciences: A Practical Guide, Oxford, UK: Blackwell."},{"key":"e_1_3_2_1_46_1","volume-title":"Experimental and Quasi-Experimental Designs for Generalized Causal Inference","author":"Shadish W. R.","year":"2002","unstructured":"Shadish , W. R. , Cook , T. D. and Campbell , D. T . ( 2002 ), Experimental and Quasi-Experimental Designs for Generalized Causal Inference , Boston : Houghton Mifflin . Shadish, W. R., Cook, T. D. and Campbell, D. T. (2002), Experimental and Quasi-Experimental Designs for Generalized Causal Inference, Boston: Houghton Mifflin."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.30"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.97"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2006.09.046"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.283.15.2008"}],"event":{"name":"ESEM '08: 2008 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement","location":"Kaiserslautern Germany","acronym":"ESEM '08","sponsor":["ACM Association for Computing Machinery","SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1414004.1414034","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1414004.1414034","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T14:47:12Z","timestamp":1750258032000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1414004.1414034"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,10,9]]},"references-count":50,"alternative-id":["10.1145\/1414004.1414034","10.1145\/1414004"],"URL":"https:\/\/doi.org\/10.1145\/1414004.1414034","relation":{},"subject":[],"published":{"date-parts":[[2008,10,9]]},"assertion":[{"value":"2008-10-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}