{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T05:45:54Z","timestamp":1776318354422,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":54,"publisher":"ACM","license":[{"start":{"date-parts":[[2017,8,21]],"date-time":"2017-08-21T00:00:00Z","timestamp":1503273600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-1421503, CCF-1438982, CCF-1439957, CCF-1553741"],"award-info":[{"award-number":["CCF-1421503, CCF-1438982, CCF-1439957, CCF-1553741"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2017,8,21]]},"DOI":"10.1145\/3106237.3106270","type":"proceedings-article","created":{"date-parts":[[2017,8,2]],"date-time":"2017-08-02T19:36:18Z","timestamp":1501702578000},"page":"197-207","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":162,"title":["Trade-offs in continuous integration: assurance, security, and flexibility"],"prefix":"10.1145","author":[{"given":"Michael","family":"Hilton","sequence":"first","affiliation":[{"name":"Oregon State University, USA"}]},{"given":"Nicholas","family":"Nelson","sequence":"additional","affiliation":[{"name":"Oregon State University, USA"}]},{"given":"Timothy","family":"Tunnell","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, USA"}]},{"given":"Darko","family":"Marinov","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, USA"}]},{"given":"Danny","family":"Dig","sequence":"additional","affiliation":[{"name":"Oregon State University, USA"}]}],"member":"320","published-online":{"date-parts":[[2017,8,21]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985871"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Alberto Bacchelli and Christian Bird. 2013. Expectations Outcomes and Challenges of Modern Code Review. In ICSE.   Alberto Bacchelli and Christian Bird. 2013. Expectations Outcomes and Challenges of Modern Code Review. In ICSE.","DOI":"10.1109\/ICSE.2013.6606617"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.796139"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/367008.367015"},{"key":"e_1_3_2_1_5_1","volume-title":"Human Research and Data Collection via the Internet. Annual Review of Psychology","author":"Birnbaum Michael H","year":"2004","unstructured":"Michael H Birnbaum . 2004. Human Research and Data Collection via the Internet. Annual Review of Psychology ( 2004 ). Michael H Birnbaum. 2004. Human Research and Data Collection via the Internet. Annual Review of Psychology (2004)."},{"key":"e_1_3_2_1_6_1","volume-title":"Object-Oriented Design with Applications","author":"Booch Grady","unstructured":"Grady Booch . 1990. Object-Oriented Design with Applications . Benjamin-Cummings Publishing Co., Inc. Grady Booch. 1990. Object-Oriented Design with Applications. Benjamin-Cummings Publishing Co., Inc."},{"key":"e_1_3_2_1_7_1","volume-title":"Pedersen","author":"Campbell John L.","year":"2013","unstructured":"John L. Campbell , Charles Quincy , Jordan Osserman , and Ove K . Pedersen . 2013 . Coding In-depth Semistructured Interviews: Problems of Unitization and Intercoder Reliability and Agreement. Sociological Methods & Research ( 2013). John L. Campbell, Charles Quincy, Jordan Osserman, and Ove K. Pedersen. 2013. Coding In-depth Semistructured Interviews: Problems of Unitization and Intercoder Reliability and Agreement. Sociological Methods & Research (2013)."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2015.7332446"},{"key":"e_1_3_2_1_9_1","unstructured":"Michael de Jong and Arie van Deursen. 2015. Continuous Deployment and Schema Evolution in SQL Databases. In RELENG.   Michael de Jong and Arie van Deursen. 2015. Continuous Deployment and Schema Evolution in SQL Databases. In RELENG."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Adam Debbiche Mikael Dien\u00e9r and Richard Berntsson Svensson. 2014. Challenges When Adopting Continuous Integration: A Case Study. In PROFES.  Adam Debbiche Mikael Dien\u00e9r and Richard Berntsson Svensson. 2014. Challenges When Adopting Continuous Integration: A Case Study. In PROFES.","DOI":"10.1007\/978-3-319-13835-0_2"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884812"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"crossref","unstructured":"Nima Dini Allison Sullivan Milos Gligoric and Gregg Rothermel. 2016. The Effect of Test Suite Type on Regression Test Selection. In ISSRE.  Nima Dini Allison Sullivan Milos Gligoric and Gregg Rothermel. 2016. The Effect of Test Suite Type on Regression Test Selection. In ISSRE.","DOI":"10.1109\/ISSRE.2016.28"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635910"},{"key":"e_1_3_2_1_14_1","unstructured":"Martin Fowler. 2006. Continuous Integration. http:\/\/martinfowler.com\/articles\/ continuousIntegration.html. (2006).  Martin Fowler. 2006. Continuous Integration. http:\/\/martinfowler.com\/articles\/ continuousIntegration.html. (2006)."},{"key":"e_1_3_2_1_15_1","volume-title":"The SAGE Encyclopedia of Qualitative Research Methods","author":"Given Lisa M","unstructured":"Lisa M Given . 2008. The SAGE Encyclopedia of Qualitative Research Methods . Lisa M Given. 2008. The SAGE Encyclopedia of Qualitative Research Methods."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2771783.2771784"},{"key":"e_1_3_2_1_17_1","unstructured":"Markus G\u00e4lli Michele Lanza Oscar Nierstrasz and Roel Wuyts. 2004. Ordering Broken Unit Tests for Focused Debugging. In ICSM.  Markus G\u00e4lli Michele Lanza Oscar Nierstrasz and Roel Wuyts. 2004. Ordering Broken Unit Tests for Focused Debugging. In ICSM."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884791"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970358"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1639950.1640023"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2008.11"},{"key":"e_1_3_2_1_22_1","volume-title":"Botched Releases: Do We Need to Roll Back? Empirical Study on a Commercial Web App. In SANER.","author":"Kerzazi Noureddine","year":"2016","unstructured":"Noureddine Kerzazi and Bram Adams . 2016 . Botched Releases: Do We Need to Roll Back? Empirical Study on a Commercial Web App. In SANER. Noureddine Kerzazi and Bram Adams. 2016. Botched Releases: Do We Need to Roll Back? Empirical Study on a Commercial Web App. In SANER."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.26"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134355"},{"key":"e_1_3_2_1_25_1","volume-title":"Debugging Revisited: Toward Understanding the Debugging Needs of Contemporary Software Developers. In ESEM.","author":"Layman Lucas","year":"2013","unstructured":"Lucas Layman , Madeline Diep , Meiyappan Nagappan , Janice Singer , Robert Deline , and Gina Venolia . 2013 . Debugging Revisited: Toward Understanding the Debugging Needs of Contemporary Software Developers. In ESEM. Lucas Layman, Madeline Diep, Meiyappan Nagappan, Janice Singer, Robert Deline, and Gina Venolia. 2013. Debugging Revisited: Toward Understanding the Debugging Needs of Contemporary Software Developers. In ESEM."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2015.50"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635920"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"crossref","unstructured":"K\u0131van\u00e7 Mu\u015flu Christian Bird Nachiappan Nagappan and Jacek Czerwonka. 2014. Transition from Centralized to Decentralized Version Control Systems: A Case Study on Reasons Barriers and Outcomes. In ICSE.  K\u0131van\u00e7 Mu\u015flu Christian Bird Nachiappan Nagappan and Jacek Czerwonka. 2014. Transition from Centralized to Decentralized Version Control Systems: A Case Study on Reasons Barriers and Outcomes. In ICSE.","DOI":"10.1145\/2568225.2568284"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"crossref","unstructured":"K\u0131van\u00e7 Mu\u015flu Yuriy Brun and Alexandra Meliou. 2015. Preventing Data Errors with Continuous Testing. In ISSTA.  K\u0131van\u00e7 Mu\u015flu Yuriy Brun and Alexandra Meliou. 2015. Preventing Data Errors with Continuous Testing. In ISSTA.","DOI":"10.1145\/2771783.2771792"},{"key":"e_1_3_2_1_30_1","unstructured":"Helena Holmstr\u00f6m Olsson Hiva Alahyari and Jan Bosch. 2012. Climbing the \"Stairway to Heaven\" \u2013 A Multiple-Case Study Exploring Barriers in the Transition from Agile Development towards Continuous Deployment of Software. In Euromicro SEAA.  Helena Holmstr\u00f6m Olsson Hiva Alahyari and Jan Bosch. 2012. Climbing the \"Stairway to Heaven\" \u2013 A Multiple-Case Study Exploring Barriers in the Transition from Agile Development towards Continuous Deployment of Software. In Euromicro SEAA."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568274"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2006.10.004"},{"key":"e_1_3_2_1_33_1","unstructured":"Puppet and DevOps Research and Assessments (DORA). 2016. 2016 State of DevOps Report. (2016).  Puppet and DevOps Research and Assessments (DORA). 2016. 2016 State of DevOps Report. (2016)."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Gregg Rothermel Roland H. Untch Chengyun Chu and Mary Jean Harrold. 1999. Test Case Prioritization: An Empirical Study. In ICSM.   Gregg Rothermel Roland H. Untch Chengyun Chu and Mary Jean Harrold. 1999. Test Case Prioritization: An Empirical Study. In ICSM.","DOI":"10.1109\/ICSM.1999.792604"},{"key":"e_1_3_2_1_35_1","volume-title":"The Coding Manual for Qualitative Researchers (3 ed.)","author":"Salda\u00f1a Johnny","unstructured":"Johnny Salda\u00f1a . 2015. The Coding Manual for Qualitative Researchers (3 ed.) . SAGE Publications . Johnny Salda\u00f1a. 2015. The Coding Manual for Qualitative Researchers (3 ed.). SAGE Publications."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2903493"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889223"},{"key":"e_1_3_2_1_38_1","volume-title":"Gall","author":"Schermann Gerald","year":"2016","unstructured":"Gerald Schermann , J\u00fcrgen Cito , Philipp Leitner , and Harald C . Gall . 2016 . Towards Quality Gates in Continuous Delivery and Deployment. In ICPC. Gerald Schermann, J\u00fcrgen Cito, Philipp Leitner, and Harald C. Gall. 2016. Towards Quality Gates in Continuous Delivery and Deployment. In ICPC."},{"key":"e_1_3_2_1_39_1","volume-title":"Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences","author":"Seidman Irving","unstructured":"Irving Seidman . 2006. Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences . Teachers College Press . Irving Seidman. 2006. Interviewing as Qualitative Research: A Guide for Researchers in Education and the Social Sciences. Teachers College Press."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635921"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"crossref","unstructured":"Forrest Shull Janice Singer and Dag I. K. Sj\u00f8berg (Eds.). 2008. Guide to Advanced Empirical Software Engineering.   Forrest Shull Janice Singer and Dag I. K. Sj\u00f8berg (Eds.). 2008. Guide to Advanced Empirical Software Engineering.","DOI":"10.1007\/978-1-84800-044-5"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Edward Smith Robert Loftin Emerson Murphy-Hill Christian Bird and Thomas Zimmermann. 2013. Improving Developer Participation Rates in Surveys. In CHASE.  Edward Smith Robert Loftin Emerson Murphy-Hill Christian Bird and Thomas Zimmermann. 2013. Improving Developer Participation Rates in Surveys. In CHASE.","DOI":"10.1109\/CHASE.2013.6614738"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2016.06.017"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Daniel St\u00e5hl and Jan Bosch. 2014. Automated Software Integration Flows in Industry: A Multiple-Case Study. In ICSE Companion.  Daniel St\u00e5hl and Jan Bosch. 2014. Automated Software Integration Flows in Industry: A Multiple-Case Study. In ICSE Companion.","DOI":"10.1145\/2591062.2591186"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889207"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/AGILE.2009.16"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/AGILE.2007.29"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2393596.2393656"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Akond Ashfaque Ur Rahman and Laurie Williams. 2016. Security Practices in DevOps. In HotSos.  Akond Ashfaque Ur Rahman and Laurie Williams. 2016. Security Practices in DevOps. In HotSos.","DOI":"10.1145\/2898375.2898383"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786850"},{"key":"e_1_3_2_1_51_1","volume-title":"10th Annual State of Agile Report. (2016)","unstructured":"VersionOne. 2016. 10th Annual State of Agile Report. (2016) . VersionOne. 2016. 10th Annual State of Agile Report. (2016)."},{"key":"e_1_3_2_1_52_1","volume-title":"FITTEST: A New Continuous and Automated Testing Process for Future Internet Applications. In CSMR-WCRE.","author":"Vos Tanja","year":"2014","unstructured":"Tanja Vos , Paolo Tonella , Wishnu Prasetya , Peter M. Kruse , Alessandra Bagnato , Mark Harman , and Onn Shehory . 2014 . FITTEST: A New Continuous and Automated Testing Process for Future Internet Applications. In CSMR-WCRE. Tanja Vos, Paolo Tonella, Wishnu Prasetya, Peter M. Kruse, Alessandra Bagnato, Mark Harman, and Onn Shehory. 2014. FITTEST: A New Continuous and Automated Testing Process for Future Internet Applications. In CSMR-WCRE."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786852"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1002\/stv.430"}],"event":{"name":"ESEC\/FSE'17: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering","location":"Paderborn Germany","acronym":"ESEC\/FSE'17","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3106237.3106270","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3106237.3106270","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3106237.3106270","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:30:37Z","timestamp":1750217437000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3106237.3106270"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,21]]},"references-count":54,"alternative-id":["10.1145\/3106237.3106270","10.1145\/3106237"],"URL":"https:\/\/doi.org\/10.1145\/3106237.3106270","relation":{},"subject":[],"published":{"date-parts":[[2017,8,21]]},"assertion":[{"value":"2017-08-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}