{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:58:19Z","timestamp":1760029099705,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,9,17]],"date-time":"2018-09-17T00:00:00Z","timestamp":1537142400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100006162","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Ci\u00eancia e Tecnologia do Estado de Pernambuco","doi-asserted-by":"publisher","award":["APQ-0399-1.03\/17, APQ\/0388-1.03\/14"],"award-info":[{"award-number":["APQ-0399-1.03\/17, APQ\/0388-1.03\/14"]}],"id":[{"id":"10.13039\/501100006162","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["465614\/2014-0, 304499\/2016-1, 141705\/2015-9"],"award-info":[{"award-number":["465614\/2014-0, 304499\/2016-1, 141705\/2015-9"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,9,17]]},"DOI":"10.1145\/3266237.3266264","type":"proceedings-article","created":{"date-parts":[[2018,9,13]],"date-time":"2018-09-13T12:54:52Z","timestamp":1536843292000},"page":"52-61","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Threats to validity in controlled experiments in software engineering"],"prefix":"10.1145","author":[{"given":"Eudis","family":"Teixeira","sequence":"first","affiliation":[{"name":"Universidade Federal de Pernambuco - Recife, Pernambuco, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liliane","family":"Fonseca","sequence":"additional","affiliation":[{"name":"Universidade Federal de Pernambuco - Recife, Pernambuco, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sergio","family":"Soares","sequence":"additional","affiliation":[{"name":"Universidade Federal de Pernambuco - Recife, Pernambuco, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,9,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Basics of qualitative research: techniques and procedures for developing grounded theory","author":"Corbin Juliet","unstructured":"Juliet Corbin , and Strauss Anselm . 2008. Basics of qualitative research: techniques and procedures for developing grounded theory ( 2 nd edition). Sage , Thousand Oaks . Juliet Corbin, and Strauss Anselm. 2008. Basics of qualitative research: techniques and procedures for developing grounded theory (2nd edition). Sage, Thousand Oaks.","edition":"2"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/942801.943640"},{"key":"e_1_3_2_1_3_1","first-page":"225","volume-title":"An\u00e1lise de conte\u00fado. Lisboa: edi\u00e7\u00f5es, 70","author":"Bardln Lawrence","unstructured":"Lawrence Bardln .1977. An\u00e1lise de conte\u00fado. Lisboa: edi\u00e7\u00f5es, 70 , p. 225 . Lawrence Bardln.1977. An\u00e1lise de conte\u00fado. Lisboa: edi\u00e7\u00f5es, 70, p.225."},{"key":"e_1_3_2_1_4_1","first-page":"49","volume-title":"Proceedings of the II Workshop de Engenharia de Software Baseada em Buscas-WESB (Vol. 12","author":"Barros M\u00e1rcio","year":"2011","unstructured":"M\u00e1rcio Barros , Arilo Dias-Neto . 2011 . Desenvolvendo uma abordagem sistem\u00e1tica para avalia\u00e7\u00e3o dos estudos experiemtais em search based software engineering . In Proceedings of the II Workshop de Engenharia de Software Baseada em Buscas-WESB (Vol. 12 , p. 49 -- 56 ). M\u00e1rcio Barros, Arilo Dias-Neto. 2011. Desenvolvendo uma abordagem sistem\u00e1tica para avalia\u00e7\u00e3o dos estudos experiemtais em search based software engineering. In Proceedings of the II Workshop de Engenharia de Software Baseada em Buscas-WESB (Vol. 12, p. 49--56)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/1018434.1021583"},{"volume-title":"Quasi-experimentation - Design and Analysis Issues for Field Settings","author":"Cook Thomas","key":"e_1_3_2_1_6_1","unstructured":"Thomas Cook and Donald Campbell . 1979. Quasi-experimentation - Design and Analysis Issues for Field Settings . Houghton Mifflin Company , Boston . Thomas Cook and Donald Campbell. 1979. Quasi-experimentation - Design and Analysis Issues for Field Settings. Houghton Mifflin Company, Boston."},{"key":"e_1_3_2_1_7_1","volume-title":"Donald Campbell and William Shadish","author":"Cook Thomas","year":"2002","unstructured":"Thomas Cook , Donald Campbell and William Shadish . 2002 . Experimental and quasi-experimental designs for generalized causal inference. Boston : Houghton Mifflin, New York, NY. Thomas Cook, Donald Campbell and William Shadish. 2002. Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin, New York, NY."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2011.37"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1414004.1414034"},{"key":"e_1_3_2_1_10_1","first-page":"285","volume-title":"Selecting empirical methods for software engineering research. Guide to advanced empirical software engineering","author":"Easterbrook Steve","unstructured":"Steve Easterbrook , Janice Singer , Margaret-Anne Storey , and Daniela Damian . 2008. Selecting empirical methods for software engineering research. Guide to advanced empirical software engineering , pp. 285 -- 311 . Springer , London . Steve Easterbrook, Janice Singer, Margaret-Anne Storey, and Daniela Damian. 2008. Selecting empirical methods for software engineering research. Guide to advanced empirical software engineering, pp.285--311. Springer, London."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2015.7321185"},{"key":"e_1_3_2_1_12_1","unstructured":"Robert Feldt and Ana Magazinius. 2010. Validity Threats in Empirical Software Engineering Research-An Initial Survey. In SEKE (pp. 374--379).  Robert Feldt and Ana Magazinius. 2010. Validity Threats in Empirical Software Engineering Research-An Initial Survey. In SEKE (pp. 374--379)."},{"key":"e_1_3_2_1_13_1","volume-title":"Research Methods in the social science","author":"Frankfort-Nachmias Chava","year":"1996","unstructured":"Chava Frankfort-Nachmias and David Nachmias . 1996. Research Methods in the social science ( 5 nd edition) New Yourk, St Martin's Press , 1996 . 600p. Chava Frankfort-Nachmias and David Nachmias. 1996. Research Methods in the social science (5nd edition) New Yourk, St Martin's Press, 1996. 600p.","edition":"5"},{"key":"e_1_3_2_1_14_1","unstructured":"Sally Green and Julian Higgins. 2008. Cochrane handbook for systematic reviews of interventions.  Sally Green and Julian Higgins. 2008. Cochrane handbook for systematic reviews of interventions."},{"key":"e_1_3_2_1_15_1","volume-title":"Version 20. IBM.","author":"IBM Corp. Released 20.0.0. 2011.","year":"2011","unstructured":"IBM Corp. Released 20.0.0. 2011. SPSS Statistics for Windows , Version 20. IBM. 2011 . Software. Retrieved from http:\/\/www.01.ibm.com\/software\/analytics\/spss\/products\/statistics\/ IBM Corp. Released 20.0.0. 2011. SPSS Statistics for Windows, Version 20. IBM. 2011. Software. Retrieved from http:\/\/www.01.ibm.com\/software\/analytics\/spss\/products\/statistics\/"},{"key":"e_1_3_2_1_17_1","volume-title":"Basics of software engineering experimentation","author":"Juristo Natalia","unstructured":"Natalia Juristo and Ana Moreno . 2013. Basics of software engineering experimentation ( 1 nd edition).. Springer Science & Business Media . Natalia Juristo and Ana Moreno. 2013. Basics of software engineering experimentation (1nd edition).. Springer Science & Business Media.","edition":"1"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1852786.1852789"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.08.007"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-006-5967-6"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2460999.2461011"},{"volume-title":"Questionnaire design, interviewing and attitude measurement","author":"Oppenheim Abraham","key":"e_1_3_2_1_22_1","unstructured":"Abraham Oppenheim . 2000. Questionnaire design, interviewing and attitude measurement . Bloomsbury Publishing . Abraham Oppenheim. 2000. Questionnaire design, interviewing and attitude measurement. Bloomsbury Publishing."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/505532.505535"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02249052"},{"key":"e_1_3_2_1_25_1","volume-title":"Real world research","author":"Robson Colin","unstructured":"Colin Robson and Kieran McCartan . 2002. Real world research ( 4 nd edition). Blackwell Publications . USA. Colin Robson and Kieran McCartan. 2002. Real world research (4nd edition). Blackwell Publications. USA.","edition":"4"},{"key":"e_1_3_2_1_26_1","volume-title":"Janice Singer and Dag Sj\u00f8berg","author":"Shull Forrest","year":"2008","unstructured":"Forrest Shull , Janice Singer and Dag Sj\u00f8berg . 2008 . Guide to Empirical Software Engineering, Springer , ISBN: 978-1-84800-043-8. Forrest Shull, Janice Singer and Dag Sj\u00f8berg. 2008. Guide to Empirical Software Engineering, Springer, ISBN: 978-1-84800-043-8."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2818754.2818759"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.97"},{"key":"e_1_3_2_1_29_1","volume-title":"Dmytro Gurov and Augusto Amaral","author":"Travassos Guilherme","year":"2002","unstructured":"Guilherme Travassos , Dmytro Gurov and Augusto Amaral . 2002 . Introdu\u00e7\u00e3o \u00e0 engenharia de software experimental (p. 53). UFRJ. Guilherme Travassos, Dmytro Gurov and Augusto Amaral. 2002. Introdu\u00e7\u00e3o \u00e0 engenharia de software experimental (p. 53). UFRJ."},{"volume-title":"Proceeding of the WSESE03","author":"Travassos Guilherme","key":"e_1_3_2_1_30_1","unstructured":"Guilherme Travassos and M\u00e1rcio Barros . 2003. Contributions of In Virtuo and In Silico Experiments for the Future of Empirical Studies in Software Engineering . In Proceeding of the WSESE03 , Fraunhofer IRB Verlag , Roma . Guilherme Travassos and M\u00e1rcio Barros. 2003. Contributions of In Virtuo and In Silico Experiments for the Future of Empirical Studies in Software Engineering. In Proceeding of the WSESE03, Fraunhofer IRB Verlag, Roma."},{"volume-title":"Goal Question Metric (GQM) approach. Encyclopedia of software engineering","author":"Solingen Rini","key":"e_1_3_2_1_31_1","unstructured":"Rini Solingen , Victor Basili , Gianluigi Caldiera and H. Dieter Rombach . 2002. Goal Question Metric (GQM) approach. Encyclopedia of software engineering . Rini Solingen, Victor Basili, Gianluigi Caldiera and H. Dieter Rombach. 2002. Goal Question Metric (GQM) approach. Encyclopedia of software engineering."},{"key":"e_1_3_2_1_32_1","volume-title":"Bj\u00f6rn Regnell and Anders Wessl\u00e9n","author":"Wohlin Claes","year":"2012","unstructured":"Claes Wohlin , Per Runeson , Martin H\u00f6st , Magnus Ohlsson , Bj\u00f6rn Regnell and Anders Wessl\u00e9n . 2012 . Experimentation in software engineering. Springer Science & Business Media . Claes Wohlin, Per Runeson, Martin H\u00f6st, Magnus Ohlsson, Bj\u00f6rn Regnell and Anders Wessl\u00e9n. 2012. Experimentation in software engineering. Springer Science & Business Media."},{"key":"e_1_3_2_1_33_1","volume-title":"Metodologia de pesquisa para ci\u00eancia da computa\u00e7\u00e3o","author":"Wazlawick Raul","unstructured":"Raul Wazlawick . 2014. Metodologia de pesquisa para ci\u00eancia da computa\u00e7\u00e3o ( 2 nd edition), (Vol. 2). Elsevier Brasil Raul Wazlawick. 2014. Metodologia de pesquisa para ci\u00eancia da computa\u00e7\u00e3o (2nd edition), (Vol. 2). Elsevier Brasil","edition":"2"}],"event":{"name":"SBES '18: XXXII BRAZILIAN SYMPOSIUM ON SOFTWARE ENGINEERING","acronym":"SBES '18","location":"Sao Carlos Brazil"},"container-title":["Proceedings of the XXXII Brazilian Symposium on Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3266237.3266264","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3266237.3266264","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T20:26:51Z","timestamp":1750278411000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3266237.3266264"}},"subtitle":["what the experts say and why this is relevant"],"short-title":[],"issued":{"date-parts":[[2018,9,17]]},"references-count":32,"alternative-id":["10.1145\/3266237.3266264","10.1145\/3266237"],"URL":"https:\/\/doi.org\/10.1145\/3266237.3266264","relation":{},"subject":[],"published":{"date-parts":[[2018,9,17]]},"assertion":[{"value":"2018-09-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}