{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T14:55:01Z","timestamp":1786114501082,"version":"3.56.0"},"reference-count":122,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2019,11,14]],"date-time":"2019-11-14T00:00:00Z","timestamp":1573689600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior","doi-asserted-by":"crossref","award":["Finance Code 001"],"award-info":[{"award-number":["Finance Code 001"]}],"id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"crossref","award":["14\/50937-1 and 15\/24485-9"],"award-info":[{"award-number":["14\/50937-1 and 15\/24485-9"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"crossref","award":["465446\/2014-0"],"award-info":[{"award-number":["465446\/2014-0"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2020,11,30]]},"abstract":"<jats:p>DevOpsis a collaborative and multidisciplinary organizational effort to automate continuous delivery of new software updates while guaranteeing their correctness and reliability. The present survey investigates and discusses DevOps challenges from the perspective of engineers, managers, and researchers. We review the literature and develop a DevOps conceptual map, correlating the DevOps automation tools with these concepts. We then discuss their practical implications for engineers, managers, and researchers. Finally, we critically explore some of the most relevant DevOps challenges reported by the literature.<\/jats:p>","DOI":"10.1145\/3359981","type":"journal-article","created":{"date-parts":[[2019,11,14]],"date-time":"2019-11-14T22:07:36Z","timestamp":1573769256000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":360,"title":["A Survey of DevOps Concepts and Challenges"],"prefix":"10.1145","volume":"52","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5762-5413","authenticated-orcid":false,"given":"Leonardo","family":"Leite","sequence":"first","affiliation":[{"name":"University of S\u00e3o, S\u00e3o Paulo, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carla","family":"Rocha","sequence":"additional","affiliation":[{"name":"University of Bras\u00edlia, Brasilia, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio","family":"Kon","sequence":"additional","affiliation":[{"name":"University of S\u00e3o, S\u00e3o Paulo, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dejan","family":"Milojicic","sequence":"additional","affiliation":[{"name":"Hewlett Packard Labs, Palo Alto, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paulo","family":"Meirelles","sequence":"additional","affiliation":[{"name":"Federal University of S\u00e3o, S\u00e3o Paulo, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,11,14]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 40th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET\u201918)","author":"Bai X.","unstructured":"X. Bai , M. Li , D. Pei , S. Li , and D. Ye . 2018. Continuous delivery of personalized assessment and feedback in agile software engineering projects . In Proceedings of the 40th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET\u201918) . 58--67. Code : I818. X. Bai, M. Li, D. Pei, S. Li, and D. Ye. 2018. Continuous delivery of personalized assessment and feedback in agile software engineering projects. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET\u201918). 58--67. Code: I818."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.64"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.60"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.4541051"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/3158161.3158170"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.66"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2015.27"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2899415.2899426"},{"key":"e_1_2_1_9_1","first-page":"110","article-title":"Identifying knowledge, skills, and abilities (KSA) for DevOps-aware server side web application with the grounded theory","volume":"32","author":"Chung Sam","year":"2016","unstructured":"Sam Chung and Soon Bang . 2016 . Identifying knowledge, skills, and abilities (KSA) for DevOps-aware server side web application with the grounded theory . J. Comput. Sci. Coll. 32 , 1 (2016), 110 -- 116 . Code: A16. Sam Chung and Soon Bang. 2016. Identifying knowledge, skills, and abilities (KSA) for DevOps-aware server side web application with the grounded theory. J. Comput. Sci. Coll. 32, 1 (2016), 110--116. Code: A16.","journal-title":"J. Comput. Sci. Coll."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.07.009"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508075.2508432"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/INM.2015.7140503"},{"key":"e_1_2_1_13_1","volume-title":"Enterprise, Business-Process and Information Systems Modeling","author":"de Feijter Rico","unstructured":"Rico de Feijter , Sietse Overbeek , Rob van Vliet , Erik Jagroep , and Sjaak Brinkkemper . 2018. DevOps competences and maturity for software producing organizations . In Enterprise, Business-Process and Information Systems Modeling . Springer , 244--259. Code : S 805. Rico de Feijter, Sietse Overbeek, Rob van Vliet, Erik Jagroep, and Sjaak Brinkkemper. 2018. DevOps competences and maturity for software producing organizations. In Enterprise, Business-Process and Information Systems Modeling. Springer, 244--259. Code: S805."},{"key":"e_1_2_1_14_1","first-page":"3","article-title":"DevOps: A software revolution in the making","volume":"24","author":"Debois Patrick","year":"2011","unstructured":"Patrick Debois . 2011 . DevOps: A software revolution in the making . Cutter IT J. 24 , 8 (2011), 3 -- 5 . Code: B4. Patrick Debois. 2011. DevOps: A software revolution in the making. Cutter IT J. 24, 8 (2011), 3--5. Code: B4.","journal-title":"Cutter IT J."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 11th IEEE International Conference on Global Software Engineering (ICGSE\u201916)","author":"Diel Elisa","unstructured":"Elisa Diel , Sabrina Marczak , and Daniela S. Cruzes . 2016. Communication challenges and strategies in distributed DevOps . In Proceedings of the 11th IEEE International Conference on Global Software Engineering (ICGSE\u201916) . 24--28. Code : I19. Elisa Diel, Sabrina Marczak, and Daniela S. Cruzes. 2016. Communication challenges and strategies in distributed DevOps. In Proceedings of the 11th IEEE International Conference on Global Software Engineering (ICGSE\u201916). 24--28. Code: I19."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/RELENG.2015.10"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.68"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2013.25"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159169"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1142055.1142065"},{"key":"e_1_2_1_21_1","volume-title":"Continuous delivery sounds great, but will it work here?Queue 15, 6","author":"Humble Jez","year":"2017","unstructured":"Jez Humble . 2017. Continuous delivery sounds great, but will it work here?Queue 15, 6 ( 2017 ), 57--76. Code : B22. Jez Humble. 2017. Continuous delivery sounds great, but will it work here?Queue 15, 6 (2017), 57--76. Code: B22."},{"key":"e_1_2_1_22_1","first-page":"6","article-title":"Why enterprises must adopt DevOps to enable continuous delivery","volume":"24","author":"Humble Jez","year":"2011","unstructured":"Jez Humble and Joanne Molesky . 2011 . Why enterprises must adopt DevOps to enable continuous delivery . Cutter IT J. 24 , 8 (2011), 6 . Code: B5. Jez Humble and Joanne Molesky. 2011. Why enterprises must adopt DevOps to enable continuous delivery. Cutter IT J. 24, 8 (2011), 6. Code: B5.","journal-title":"Cutter IT J."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICGSE.2017.16"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3230833.3233275"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098954.3103172"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/IC2E.2016.26"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2018.1661330"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-NIER.2017.20"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2015.50"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCC.2018.022171668"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2016.671"},{"key":"e_1_2_1_32_1","article-title":"Design and implementation of a social networking platform for cloud deployment specialists","volume":"6","author":"Magoutis Kostas","year":"2015","unstructured":"Kostas Magoutis , Christos Papoulas , Antonis Papaioannou , Flora Karniavoura , Dimitrios-Georgios Akestoridis , Nikos Parotsidis , Maria Korozi , Asterios Leonidis , Stavroula Ntoa , and Constantine Stephanidis . 2015 . Design and implementation of a social networking platform for cloud deployment specialists . J. Int. Serv. Appl. 6 , 1 (2015). Code: S3. Kostas Magoutis, Christos Papoulas, Antonis Papaioannou, Flora Karniavoura, Dimitrios-Georgios Akestoridis, Nikos Parotsidis, Maria Korozi, Asterios Leonidis, Stavroula Ntoa, and Constantine Stephanidis. 2015. Design and implementation of a social networking platform for cloud deployment specialists. J. Int. Serv. Appl. 6, 1 (2015). Code: S3.","journal-title":"J. Int. Serv. Appl."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/AGILE.2013.17"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-33515-5_11"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2012.54"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2016.45"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/RTEICT.2016.7807905"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183440.3183452"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the International Conference on Advances in Computing, Communication, Automation (ICACCA\u201916)","author":"Rajkumar M.","unstructured":"M. Rajkumar , A. K. Pole , V. S. Adige , and P. Mahanta . 2016. DevOps culture and its impact on cloud delivery and software development . In Proceedings of the International Conference on Advances in Computing, Communication, Automation (ICACCA\u201916) . 1--6. Code : I48. M. Rajkumar, A. K. Pole, V. S. Adige, and P. Mahanta. 2016. DevOps culture and its impact on cloud delivery and software development. In Proceedings of the International Conference on Advances in Computing, Communication, Automation (ICACCA\u201916). 1--6. Code: I48."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2524713.2524721"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCOM.2018.1600104"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2961111.2962587"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCC.2014.50"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2018.111095426"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2017.4541032"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/IC2E.2015.23"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-26148-5_23"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.86"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.4018\/IJSSE.2016100103"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2015.23"},{"key":"e_1_2_1_51_1","volume-title":"Survey Report","year":"2017","unstructured":"xMatters. 2017. xMatters Atlassian DevOps Maturity Survey Report 2017 . Retrieved from: https:\/\/www.xmatters.com\/press-release\/xmatters-atlassian-2017-devops-maturity-survey-report\/. xMatters. 2017. xMatters Atlassian DevOps Maturity Survey Report 2017. Retrieved from: https:\/\/www.xmatters.com\/press-release\/xmatters-atlassian-2017-devops-maturity-survey-report\/."},{"key":"e_1_2_1_52_1","unstructured":"YouTube. 2018. How Netflix Thinks of DevOps. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;UTKIT6STSVM.  YouTube. 2018. How Netflix Thinks of DevOps. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;UTKIT6STSVM."},{"key":"e_1_2_1_53_1","unstructured":"Nicole Forsgren Velasquez Alanna Brown Gene Kim Nigel Kersten and Jez Humble. 2016. 2016 State of DevOps Report. Retrieved from: https:\/\/puppet.com\/resources\/whitepaper\/2016-state-of-devops-report.  Nicole Forsgren Velasquez Alanna Brown Gene Kim Nigel Kersten and Jez Humble. 2016. 2016 State of DevOps Report. Retrieved from: https:\/\/puppet.com\/resources\/whitepaper\/2016-state-of-devops-report."},{"key":"e_1_2_1_54_1","unstructured":"Hrishikesh Barua. 2015. The Role of Configuration Management in a Containerized World. Retrieved from: https:\/\/www.infoq.com\/news\/2015\/12\/containers-vs-config-mgmt.  Hrishikesh Barua. 2015. The Role of Configuration Management in a Containerized World. Retrieved from: https:\/\/www.infoq.com\/news\/2015\/12\/containers-vs-config-mgmt."},{"key":"e_1_2_1_55_1","volume-title":"DevOps: A Software Architect\u2019s Perspective","author":"Bass Len","unstructured":"Len Bass , Ingo Weber , and Liming Zhu . 2015. DevOps: A Software Architect\u2019s Perspective . Addison-Wesley Professional . Len Bass, Ingo Weber, and Liming Zhu. 2015. DevOps: A Software Architect\u2019s Perspective. Addison-Wesley Professional."},{"key":"e_1_2_1_56_1","unstructured":"Helen Beal. 2015. Where are you on the DevOps Maturity Scale Webcast. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;a50ArHzVRqk.  Helen Beal. 2015. Where are you on the DevOps Maturity Scale Webcast. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;a50ArHzVRqk."},{"key":"e_1_2_1_57_1","volume-title":"Extreme Programming Explained: Embrace Change","author":"Beck Kent","unstructured":"Kent Beck and Cynthia Andres . 2004. Extreme Programming Explained: Embrace Change . Addison-Wesley Professional . Kent Beck and Cynthia Andres. 2004. Extreme Programming Explained: Embrace Change. Addison-Wesley Professional."},{"key":"e_1_2_1_58_1","unstructured":"Betsy Beyer Chris Jones Jennifer Petoff and Niall Richard Murphy. 2016. Site Reliability Engineering: How Google Runs Production Systems. O\u2019Reilly Media.  Betsy Beyer Chris Jones Jennifer Petoff and Niall Richard Murphy. 2016. Site Reliability Engineering: How Google Runs Production Systems. O\u2019Reilly Media."},{"key":"e_1_2_1_59_1","unstructured":"Jonas Bon\u00e9r Dave Farley Roland Kuhn and Martin Thompson. 2014. The Reactive Manifesto. Retrieved from: https:\/\/www.reactivemanifesto.org\/.  Jonas Bon\u00e9r Dave Farley Roland Kuhn and Martin Thompson. 2014. The Reactive Manifesto. Retrieved from: https:\/\/www.reactivemanifesto.org\/."},{"key":"e_1_2_1_60_1","unstructured":"Rob Brigham. 2015. DevOps at Amazon: A Look at Our Tools and Processes. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;esEFaY0FDKc.  Rob Brigham. 2015. DevOps at Amazon: A Look at Our Tools and Processes. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;esEFaY0FDKc."},{"key":"e_1_2_1_61_1","unstructured":"Donovan Brown. 2018. Our DevOps Journey\u2014Microsoft\u2019s Internal Transformation Story. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;cbFzojQOjyA.  Donovan Brown. 2018. Our DevOps Journey\u2014Microsoft\u2019s Internal Transformation Story. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;cbFzojQOjyA."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134500"},{"key":"e_1_2_1_63_1","unstructured":"Necco Ceresani. 2016. The Periodic Table of DevOps Tools v.2 Is Here. Retrieved from: https:\/\/blog.xebialabs.com\/2016\/06\/14\/periodic-table-devops-tools-v-2\/.  Necco Ceresani. 2016. The Periodic Table of DevOps Tools v.2 Is Here. Retrieved from: https:\/\/blog.xebialabs.com\/2016\/06\/14\/periodic-table-devops-tools-v-2\/."},{"key":"e_1_2_1_64_1","unstructured":"Kathy Charmaz. 2008. Chapter 7: Grounded theory as an emergent method. In Handbook of Emergent Methods. The Guilford Press.  Kathy Charmaz. 2008. Chapter 7: Grounded theory as an emergent method. In Handbook of Emergent Methods. The Guilford Press."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2007.07.027"},{"key":"e_1_2_1_66_1","volume-title":"Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory","author":"Corbin Juliet","unstructured":"Juliet Corbin and Anselm Strauss . 2014. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory ( 4 th ed.). SAGE Publications, Inc. Juliet Corbin and Anselm Strauss. 2014. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory (4th ed.). SAGE Publications, Inc.","edition":"4"},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the 30th Brazilian Symposium on Software Engineering (SBES\u201916)","author":"Breno","unstructured":"Breno B. Nicolau de Fran\u00e7a, Helvio Jeronimo, Jr., and Guilherme Horta Travassos. 2016. Characterizing DevOps by hearing multiple voices . In Proceedings of the 30th Brazilian Symposium on Software Engineering (SBES\u201916) . ACM, 53--62. Breno B. Nicolau de Fran\u00e7a, Helvio Jeronimo, Jr., and Guilherme Horta Travassos. 2016. Characterizing DevOps by hearing multiple voices. In Proceedings of the 30th Brazilian Symposium on Software Engineering (SBES\u201916). ACM, 53--62."},{"key":"e_1_2_1_68_1","volume-title":"Agile Infrastructure Operations. At Agile 2008 Toronto. Retrieved on","author":"Debois Patrick","year":"2019","unstructured":"Patrick Debois . 2008. Agile Infrastructure Operations. At Agile 2008 Toronto. Retrieved on October 2019 from http:\/\/www.jedi.be\/presentations\/agile-infrastructure-agile-2008.pdf. Patrick Debois. 2008. Agile Infrastructure Operations. At Agile 2008 Toronto. Retrieved on October 2019 from http:\/\/www.jedi.be\/presentations\/agile-infrastructure-agile-2008.pdf."},{"key":"e_1_2_1_69_1","unstructured":"Phil Dougherty. 2015. Containers vs. Config Management. Retrieved from: https:\/\/blog.containership.io\/containers-vs-config-management-e64cbb744a94.  Phil Dougherty. 2015. Containers vs. Config Management. Retrieved from: https:\/\/blog.containership.io\/containers-vs-config-management-e64cbb744a94."},{"key":"e_1_2_1_70_1","volume-title":"A mapping study on cooperation between information system development and operations","author":"Erich Floris","unstructured":"Floris Erich , Chintan Amrit , and Maya Daneva . 2014. A mapping study on cooperation between information system development and operations . In Product-Focused Software Process Improvement, Andreas Jedlitschka, Pasi Kuvaja, Marco Kuhrmann, Tomi M\u00e4nnist\u00f6, J\u00fcrgen M\u00fcnch, and Mikko Raatikainen (Eds.). Springer International Publishing , Cham , 277--280. Floris Erich, Chintan Amrit, and Maya Daneva. 2014. A mapping study on cooperation between information system development and operations. In Product-Focused Software Process Improvement, Andreas Jedlitschka, Pasi Kuvaja, Marco Kuhrmann, Tomi M\u00e4nnist\u00f6, J\u00fcrgen M\u00fcnch, and Mikko Raatikainen (Eds.). Springer International Publishing, Cham, 277--280."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1885"},{"key":"e_1_2_1_72_1","volume-title":"Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations","author":"Forsgren Nicole","year":"2018","unstructured":"Nicole Forsgren , Jez Humble , and Gene Kim . 2018 . Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations . IT Revolution Press . Nicole Forsgren, Jez Humble, and Gene Kim. 2018. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations. IT Revolution Press."},{"key":"e_1_2_1_73_1","unstructured":"Martin Fowler. 2004. Strangler Application. Retrieved from: https:\/\/www.martinfowler.com\/bliki\/Strangler-Application.html.  Martin Fowler. 2004. Strangler Application. Retrieved from: https:\/\/www.martinfowler.com\/bliki\/Strangler-Application.html."},{"key":"e_1_2_1_74_1","unstructured":"Martin Fowler. 2010. Blue Green Deployment. Retrieved from: https:\/\/martinfowler.com\/bliki\/BlueGreen-Deployment.html.  Martin Fowler. 2010. Blue Green Deployment. Retrieved from: https:\/\/martinfowler.com\/bliki\/BlueGreen-Deployment.html."},{"key":"e_1_2_1_75_1","volume-title":"Proceedings of the 21st Pacific Asia Conference on Information Systems (PACIS\u201917)","author":"Ghantous Georges Bou","year":"2017","unstructured":"Georges Bou Ghantous and Asif Gill . 2017 . DevOps: Concepts, practices, tools, benefits and challenges . In Proceedings of the 21st Pacific Asia Conference on Information Systems (PACIS\u201917) . 96:1--96:12. Georges Bou Ghantous and Asif Gill. 2017. DevOps: Concepts, practices, tools, benefits and challenges. In Proceedings of the 21st Pacific Asia Conference on Information Systems (PACIS\u201917). 96:1--96:12."},{"key":"e_1_2_1_76_1","volume-title":"Howard Jeffrey Scheiber, and Nancy C. Corbin","author":"Hager Peter J.","year":"1997","unstructured":"Peter J. Hager , Howard Jeffrey Scheiber, and Nancy C. Corbin . 1997 . Designing 8 Delivering : Scientific, Technical, and Managerial Presentations. John Wiley 8 Sons. Peter J. Hager, Howard Jeffrey Scheiber, and Nancy C. Corbin. 1997. Designing 8 Delivering: Scientific, Technical, and Managerial Presentations. John Wiley 8 Sons."},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.5555\/1349426.1349444"},{"key":"e_1_2_1_78_1","unstructured":"Pete Hodgson. 2017. Feature Toggles (aka Feature Flags). Retrieved from: https:\/\/martinfowler.com\/articles\/feature-toggles.html.  Pete Hodgson. 2017. Feature Toggles (aka Feature Flags). Retrieved from: https:\/\/martinfowler.com\/articles\/feature-toggles.html."},{"key":"e_1_2_1_79_1","unstructured":"Jonah Horowitz. 2017. Configuration Management Is an Antipattern. Retrieved from: https:\/\/hackernoon.com\/configuration-management-is-an-antipattern-e677e34be64c.  Jonah Horowitz. 2017. Configuration Management Is an Antipattern. Retrieved from: https:\/\/hackernoon.com\/configuration-management-is-an-antipattern-e677e34be64c."},{"key":"e_1_2_1_80_1","unstructured":"Jez Humble. 2010. Continuous Delivery vs Continuous Deployment. Retrieved from: https:\/\/continuousdelivery.com\/2010\/08\/continuous-delivery-vs-continuous-deployment\/.  Jez Humble. 2010. Continuous Delivery vs Continuous Deployment. Retrieved from: https:\/\/continuousdelivery.com\/2010\/08\/continuous-delivery-vs-continuous-deployment\/."},{"key":"e_1_2_1_81_1","unstructured":"Jez Humble. 2012. There\u2019s No Such Thing as a \u201cDevOps Team.\u201d Retrieved from: https:\/\/continuousdelivery.com\/2012\/10\/theres-no-such-thing-as-a-devops-team\/.  Jez Humble. 2012. There\u2019s No Such Thing as a \u201cDevOps Team.\u201d Retrieved from: https:\/\/continuousdelivery.com\/2012\/10\/theres-no-such-thing-as-a-devops-team\/."},{"key":"e_1_2_1_82_1","volume-title":"Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation","author":"Humble Jez","year":"2010","unstructured":"Jez Humble and David Farley . 2010 . Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation . Addison-Wesley Professional . Jez Humble and David Farley. 2010. Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. Addison-Wesley Professional."},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/2962695.2962707"},{"key":"e_1_2_1_84_1","unstructured":"Adam Jacob. 2015. Chef Style DevOps Kungfu. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;_DEToXsgrPc.  Adam Jacob. 2015. Chef Style DevOps Kungfu. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;_DEToXsgrPc."},{"key":"e_1_2_1_85_1","unstructured":"Dan Kelly. 2016. Configuration Management and Containers: Which Is Better? Retrieved from: https:\/\/blog.containership.io\/configuration-management-and-containers-which-is-better.  Dan Kelly. 2016. Configuration Management and Containers: Which Is Better? Retrieved from: https:\/\/blog.containership.io\/configuration-management-and-containers-which-is-better."},{"key":"e_1_2_1_86_1","volume-title":"Proceedings of the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201916)","volume":"1","author":"Kerzazi N.","unstructured":"N. Kerzazi and B. Adams . 2016. Botched releases: Do we need to roll back? Empirical study on a commercial web app . In Proceedings of the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201916) , Vol. 1 . 574--583. N. Kerzazi and B. Adams. 2016. Botched releases: Do we need to roll back? Empirical study on a commercial web app. In Proceedings of the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201916), Vol. 1. 574--583."},{"key":"e_1_2_1_87_1","volume-title":"Proceedings of the IEEE\/ACM International Workshop on Continuous Software Evolution and Delivery (CSED\u201916)","author":"Kerzazi N.","unstructured":"N. Kerzazi and B. Adams . 2016. Who needs release and DevOps engineers, and why? In Proceedings of the IEEE\/ACM International Workshop on Continuous Software Evolution and Delivery (CSED\u201916) . 77--83. N. Kerzazi and B. Adams. 2016. Who needs release and DevOps engineers, and why? In Proceedings of the IEEE\/ACM International Workshop on Continuous Software Evolution and Delivery (CSED\u201916). 77--83."},{"key":"e_1_2_1_88_1","unstructured":"Gene Kim. 2012. The Three Ways: The Principles Underpinning DevOps. Retrieved from: http:\/\/itrevolution.com\/the-three-ways-principles-underpinning-devops\/.  Gene Kim. 2012. The Three Ways: The Principles Underpinning DevOps. Retrieved from: http:\/\/itrevolution.com\/the-three-ways-principles-underpinning-devops\/."},{"key":"e_1_2_1_89_1","unstructured":"Gene Kim Kevin Behr and Kim Spafford. 2014. The Phoenix Project: A Novel about IT DevOps and Helping Your Business Win. IT Revolution.  Gene Kim Kevin Behr and Kim Spafford. 2014. The Phoenix Project: A Novel about IT DevOps and Helping Your Business Win. IT Revolution."},{"key":"e_1_2_1_90_1","volume-title":"The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations","author":"Kim Gene","unstructured":"Gene Kim , Jez Humble , Patrick Debois , and John Willis . 2016. The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations . IT Revolution Press . Gene Kim, Jez Humble, Patrick Debois, and John Willis. 2016. The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. IT Revolution Press."},{"key":"e_1_2_1_91_1","unstructured":"Henrik Kniberg. 2014. Spotify engineering culture (part 1). Retrieved from: https:\/\/labs.spotify.com\/2014\/03\/27\/spotify-engineering-culture-part-1.  Henrik Kniberg. 2014. Spotify engineering culture (part 1). Retrieved from: https:\/\/labs.spotify.com\/2014\/03\/27\/spotify-engineering-culture-part-1."},{"key":"e_1_2_1_92_1","volume-title":"The Rational Unified Process Made Easy: A Practitioner\u2019s Guide to the RUP","author":"Kroll Per","unstructured":"Per Kroll and Philippe Kruchten . 2003. The Rational Unified Process Made Easy: A Practitioner\u2019s Guide to the RUP . Addison-Wesley Professional . Per Kroll and Philippe Kruchten. 2003. The Rational Unified Process Made Easy: A Practitioner\u2019s Guide to the RUP. Addison-Wesley Professional."},{"key":"e_1_2_1_93_1","unstructured":"Patrick Kua. 2013. An Appropriate Use of Metrics. Retrieved from: https:\/\/martinfowler.com\/articles\/useOfMetrics.html.  Patrick Kua. 2013. An Appropriate Use of Metrics. Retrieved from: https:\/\/martinfowler.com\/articles\/useOfMetrics.html."},{"key":"e_1_2_1_94_1","unstructured":"James Lewis and Martin Fowler. 2014. Microservices. Retrieved from: https:\/\/www.martinfowler.com\/articles\/microservices.html.  James Lewis and Martin Fowler. 2014. Microservices. Retrieved from: https:\/\/www.martinfowler.com\/articles\/microservices.html."},{"key":"e_1_2_1_95_1","volume-title":"Agile Processes in Software Engineering and Extreme Programming","author":"Lwakatare Lucy Ellen","unstructured":"Lucy Ellen Lwakatare , Pasi Kuvaja , and Markku Oivo . 2015. Dimensions of DevOps . In Agile Processes in Software Engineering and Extreme Programming . Springer International Publishing , 212--217. Lucy Ellen Lwakatare, Pasi Kuvaja, and Markku Oivo. 2015. Dimensions of DevOps. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, 212--217."},{"key":"e_1_2_1_96_1","unstructured":"Robert C. Martin. 2008. Chapter 12: Emergence. In Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall.  Robert C. Martin. 2008. Chapter 12: Emergence. In Clean Code: A Handbook of Agile Software Craftsmanship. Prentice Hall."},{"key":"e_1_2_1_97_1","volume-title":"Proceedings of the NATO Conference on Software Engineering, Software Engineering Concepts and Techniques. 88--98","author":"McIlroy M. Douglas","year":"1968","unstructured":"M. Douglas McIlroy , J. M. Buxton , Peter Naur , and Brian Randell . 1968 . Mass-produced software components . In Proceedings of the NATO Conference on Software Engineering, Software Engineering Concepts and Techniques. 88--98 . M. Douglas McIlroy, J. M. Buxton, Peter Naur, and Brian Randell. 1968. Mass-produced software components. In Proceedings of the NATO Conference on Software Engineering, Software Engineering Concepts and Techniques. 88--98."},{"key":"e_1_2_1_98_1","unstructured":"Peter Mell and Timothy Grance. 2011. The NIST definition of cloud computing. Retrieved from: http:\/\/csrc.nist.gov\/publications\/nistpubs\/800-145\/SP800-145.pdf.  Peter Mell and Timothy Grance. 2011. The NIST definition of cloud computing. Retrieved from: http:\/\/csrc.nist.gov\/publications\/nistpubs\/800-145\/SP800-145.pdf."},{"key":"e_1_2_1_99_1","unstructured":"Matthew B. Miles and A. Michael Huberman. 1994. Chapter 2: Focusing and bounding the collection of data\u2014The substantive start. In Qualitative Data Analysis: An Expanded Sourcebook (6th ed.). SAGE Publications.  Matthew B. Miles and A. Michael Huberman. 1994. Chapter 2: Focusing and bounding the collection of data\u2014The substantive start. In Qualitative Data Analysis: An Expanded Sourcebook (6th ed.). SAGE Publications."},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2011.2"},{"key":"e_1_2_1_101_1","unstructured":"Kief Morris. 2016. Infrastructure as Code: Managing Servers in the Cloud. O\u2019Reilly Media.  Kief Morris. 2016. Infrastructure as Code: Managing Servers in the Cloud. O\u2019Reilly Media."},{"key":"e_1_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICWT.2018.8527810"},{"key":"e_1_2_1_103_1","unstructured":"Michael T. Nygard. 2009. Release It! Design and Deploy Production-Ready Software. Pragmatic Bookshelf.  Michael T. Nygard. 2009. Release It! Design and Deploy Production-Ready Software. Pragmatic Bookshelf."},{"key":"e_1_2_1_104_1","volume-title":"Implementing Lean Software Development: From Concept to Cash","author":"Poppendieck Mary","unstructured":"Mary Poppendieck and Tom Poppendieck . 2006. Implementing Lean Software Development: From Concept to Cash . Addison-Wesley Professional . Mary Poppendieck and Tom Poppendieck. 2006. Implementing Lean Software Development: From Concept to Cash. Addison-Wesley Professional."},{"key":"e_1_2_1_105_1","volume-title":"Software Engineering: A Practitioner\u2019s Approach","author":"Pressman Roger S.","year":"2005","unstructured":"Roger S. Pressman . 2005 . Software Engineering: A Practitioner\u2019s Approach ( 6 th ed.). Palgrave Macmillan . Roger S. Pressman. 2005. Software Engineering: A Practitioner\u2019s Approach (6th ed.). Palgrave Macmillan.","edition":"6"},{"key":"e_1_2_1_106_1","unstructured":"Mike Roberts. 2018. Serverless Architectures. Retrieved from: https:\/\/martinfowler.com\/articles\/serverless.html.  Mike Roberts. 2018. Serverless Architectures. Retrieved from: https:\/\/martinfowler.com\/articles\/serverless.html."},{"key":"e_1_2_1_107_1","unstructured":"Kevin Roebuck. 2011. DevOps: High-impact Strategies\u2014What You Need to Know: Definitions Adoptions Impact Benefits Maturity Vendors. Tebbo.  Kevin Roebuck. 2011. DevOps: High-impact Strategies\u2014What You Need to Know: Definitions Adoptions Impact Benefits Maturity Vendors. Tebbo."},{"key":"e_1_2_1_108_1","unstructured":"Margaret Rouse. 2015. What Is NoOps?\u2014Definition from WhatIs.com. Retrieved from: https:\/\/searchcloudapplications.techtarget.com\/definition\/noops.  Margaret Rouse. 2015. What Is NoOps?\u2014Definition from WhatIs.com. Retrieved from: https:\/\/searchcloudapplications.techtarget.com\/definition\/noops."},{"key":"e_1_2_1_109_1","unstructured":"Danilo Sato. 2014. CanaryRelease. Retrieved from: https:\/\/martinfowler.com\/bliki\/CanaryRelease.html.  Danilo Sato. 2014. CanaryRelease. Retrieved from: https:\/\/martinfowler.com\/bliki\/CanaryRelease.html."},{"key":"e_1_2_1_110_1","volume-title":"Practice: Reliable and Automated Software Delivery. Casa do C\u00f3digo.","author":"Sato Danilo","unstructured":"Danilo Sato . 2014. Chapter 12: Infrastructure as code. In DevOps in Practice: Reliable and Automated Software Delivery. Casa do C\u00f3digo. Danilo Sato. 2014. Chapter 12: Infrastructure as code. In DevOps in Practice: Reliable and Automated Software Delivery. Casa do C\u00f3digo."},{"key":"e_1_2_1_111_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2288-11-128"},{"key":"e_1_2_1_112_1","unstructured":"Julia Silge. 2017. How Much Do Developers Earn? Find Out with the Stack Overflow Salary Calculator. Retrieved from: https:\/\/stackoverflow.blog\/2017\/09\/19\/much-developers-earn-find-stack-overflow-salary-calculator\/.  Julia Silge. 2017. How Much Do Developers Earn? Find Out with the Stack Overflow Salary Calculator. Retrieved from: https:\/\/stackoverflow.blog\/2017\/09\/19\/much-developers-earn-find-stack-overflow-salary-calculator\/."},{"key":"e_1_2_1_113_1","unstructured":"Matthew Skelton and Manuel Pais. 2013. DevOps Topologies. Retrieved from: https:\/\/web.devopstopologies.com\/.  Matthew Skelton and Manuel Pais. 2013. DevOps Topologies. Retrieved from: https:\/\/web.devopstopologies.com\/."},{"key":"e_1_2_1_114_1","volume-title":"Agile Processes in Software Engineering and Extreme Programming","author":"Smeds Jens","unstructured":"Jens Smeds , Kristian Nybom , and Ivan Porres . 2015. DevOps: A definition and perceived adoption impediments . In Agile Processes in Software Engineering and Extreme Programming . Springer International Publishing , 166--177. Jens Smeds, Kristian Nybom, and Ivan Porres. 2015. DevOps: A definition and perceived adoption impediments. In Agile Processes in Software Engineering and Extreme Programming. Springer International Publishing, 166--177."},{"key":"e_1_2_1_115_1","volume-title":"Software Engineering","author":"Sommerville Ian","unstructured":"Ian Sommerville . 2011. Software Engineering ( 9 th ed.). Addison-Wesley . Ian Sommerville. 2011. Software Engineering (9th ed.). Addison-Wesley.","edition":"9"},{"key":"e_1_2_1_116_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2017.8114695"},{"key":"e_1_2_1_117_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884833"},{"key":"e_1_2_1_118_1","volume-title":"Proceedings of the Brazilian Conference on Software: Theory and Practice (Tools Session) (CBSoft\u201910)","volume":"29","author":"Terceiro Antonio","year":"2010","unstructured":"Antonio Terceiro , Joenio Costa , Jo\u00e3o Miranda , Paulo Meirelles , Luiz Rom\u00e1rio Rios , Lucianna Almeida , Christina Chavez , and Fabio Kon . 2010 . Analizo: An extensible multi-language source code analysis and visualization toolkit . In Proceedings of the Brazilian Conference on Software: Theory and Practice (Tools Session) (CBSoft\u201910) , Vol. 29 . Antonio Terceiro, Joenio Costa, Jo\u00e3o Miranda, Paulo Meirelles, Luiz Rom\u00e1rio Rios, Lucianna Almeida, Christina Chavez, and Fabio Kon. 2010. Analizo: An extensible multi-language source code analysis and visualization toolkit. In Proceedings of the Brazilian Conference on Software: Theory and Practice (Tools Session) (CBSoft\u201910), Vol. 29."},{"key":"e_1_2_1_119_1","unstructured":"J. C. van Winkel. 2017. Life of an SRE at Google. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;7Oe8mYPBZmw.  J. C. van Winkel. 2017. Life of an SRE at Google. Retrieved from: https:\/\/www.youtube.com\/watch?v&equals;7Oe8mYPBZmw."},{"key":"e_1_2_1_120_1","unstructured":"Nicole Forsgren Velasquez Gene Kim Nigel Kersten and Jez Humble. 2014. 2014 State of DevOps Report. Retrieved from: https:\/\/puppet.com\/resources\/whitepaper\/2014-state-devops-report.  Nicole Forsgren Velasquez Gene Kim Nigel Kersten and Jez Humble. 2014. 2014 State of DevOps Report. Retrieved from: https:\/\/puppet.com\/resources\/whitepaper\/2014-state-devops-report."},{"key":"e_1_2_1_121_1","unstructured":"Adam Wiggins. 2011. The Twelve-Factor App. Retrieved from: https:\/\/12factor.net\/.  Adam Wiggins. 2011. The Twelve-Factor App. Retrieved from: https:\/\/12factor.net\/."},{"key":"e_1_2_1_122_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3359981","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3359981","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:13:27Z","timestamp":1750202007000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3359981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,14]]},"references-count":122,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2020,11,30]]}},"alternative-id":["10.1145\/3359981"],"URL":"https:\/\/doi.org\/10.1145\/3359981","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,14]]},"assertion":[{"value":"2019-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-11-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}