{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T14:59:46Z","timestamp":1784905186634,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":66,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,6,18]],"date-time":"2024-06-18T00:00:00Z","timestamp":1718668800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,6,18]]},"DOI":"10.1145\/3661167.3661236","type":"proceedings-article","created":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T12:24:25Z","timestamp":1718367865000},"page":"520-527","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Multivocal Literature Review on DevOps Critical Success Factors"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2428-2984","authenticated-orcid":false,"given":"Nasreen","family":"Azad","sequence":"first","affiliation":[{"name":"Software Engineering, LUT University, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5073-3750","authenticated-orcid":false,"given":"Sami","family":"Hyrynsalmi","sequence":"additional","affiliation":[{"name":"Department Software Engineering, LUT University, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,6,18]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Agile software development methods: Review and analysis. arXiv preprint arXiv:1709.08439","author":"Abrahamsson Pekka","year":"2017","unstructured":"Pekka Abrahamsson, Outi Salo, Jussi Ronkainen, and Juhani Warsta. 2017. Agile software development methods: Review and analysis. arXiv preprint arXiv:1709.08439 (2017)."},{"key":"e_1_3_2_1_2_1","volume-title":"Abeer Abdul-Aziz Alsanad, and Abdu Gumaei","author":"Akbar Muhammad\u00a0Azeem","year":"2020","unstructured":"Muhammad\u00a0Azeem Akbar, Sajjad Mahmood, Muhammad Shafiq, Ahmed Alsanad, Abeer Abdul-Aziz Alsanad, and Abdu Gumaei. 2020. Identification and prioritization of DevOps success factors using fuzzy-AHP approach. Soft Computing (2020), 1\u201325."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.11591\/ijeecs.v21.i1.pp348-353"},{"key":"e_1_3_2_1_4_1","volume-title":"Capabilities and practices in DevOps: a multivocal literature review","author":"Manuel\u00a0Duarte Amaro Ricardo","year":"2022","unstructured":"Ricardo Manuel\u00a0Duarte Amaro, Ruben Pereira, and Miguel\u00a0Mira da Silva. 2022. Capabilities and practices in DevOps: a multivocal literature review. IEEE Transactions on Software Engineering (2022)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3524614.3528627"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-91983-2_17"},{"key":"e_1_3_2_1_7_1","volume-title":"DevOps Challenges in Organizations: Through Professional Lens. In Software Business: 13th International Conference, ICSOB 2022, Bolzano, Italy, November 8\u201311, 2022, Proceedings. Springer, 260\u2013277","author":"Azad Nasreen","year":"2022","unstructured":"Nasreen Azad and Sami Hyrynsalmi. 2022. DevOps Challenges in Organizations: Through Professional Lens. In Software Business: 13th International Conference, ICSOB 2022, Bolzano, Italy, November 8\u201311, 2022, Proceedings. Springer, 260\u2013277."},{"key":"e_1_3_2_1_8_1","volume-title":"DevOps critical succes factors\u2014A systematic literature review. Information and Software Technology","author":"Azad Nasreen","year":"2023","unstructured":"Nasreen Azad and Sami Hyrynsalmi. 2023. DevOps critical succes factors\u2014A systematic literature review. Information and Software Technology (2023), 107150."},{"key":"e_1_3_2_1_9_1","volume-title":"DevOps critical succes factors\u2014A systematic literature review. Information and Software Technology","author":"Azad Nasreen","year":"2023","unstructured":"Nasreen Azad and Sami Hyrynsalmi. 2023. DevOps critical succes factors\u2014A systematic literature review. Information and Software Technology (2023), 107150."},{"key":"e_1_3_2_1_10_1","volume-title":"SDLC perspective. arXiv preprint arXiv:0911.0494","author":"Banerjee C","year":"2009","unstructured":"C Banerjee and SK Pandey. 2009. Software security rules, SDLC perspective. arXiv preprint arXiv:0911.0494 (2009)."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.2329"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3297663.3309672"},{"key":"e_1_3_2_1_13_1","volume-title":"Product Ops: Understanding and Defining an Emerging Discipline. In Int.Conf. on Software Product Management. GI, 11.","author":"Bogdan Moroz","year":"2023","unstructured":"Moroz Bogdan, Andrey Saltan, and Sami Hyrynsalmi. 2023. Product Ops: Understanding and Defining an Emerging Discipline. In Int.Conf. on Software Product Management. GI, 11."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2017.02.013"},{"key":"e_1_3_2_1_15_1","volume-title":"On the journey to continuous deployment: Technical and social challenges along the way. Information and Software technology 57","author":"Claps Gerry\u00a0Gerard","year":"2015","unstructured":"Gerry\u00a0Gerard Claps, Richard\u00a0Berntsson Svensson, and Ayb\u00fcke Aurum. 2015. On the journey to continuous deployment: Technical and social challenges along the way. Information and Software technology 57 (2015), 21\u201331."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/Agile.2008.42"},{"key":"e_1_3_2_1_17_1","volume-title":"Why are many businesses instilling a DevOps culture into their organization?Empirical Software Engineering 26, 2","author":"D\u00edaz Jessica","year":"2021","unstructured":"Jessica D\u00edaz, Daniel L\u00f3pez-Fern\u00e1ndez, Jorge P\u00e9rez, and \u00c1ngel Gonz\u00e1lez-Prieto. 2021. Why are many businesses instilling a DevOps culture into their organization?Empirical Software Engineering 26, 2 (2021), 1\u201350."},{"key":"e_1_3_2_1_18_1","volume-title":"International Conference on Product-Focused Software Process Improvement. Springer, 323\u2013330","author":"D\u00edaz Jessica","year":"2019","unstructured":"Jessica D\u00edaz, Jorge\u00a0E Perez, Agustin Yague, Andrea Villegas, and Antonio de Antona. 2019. Devops in practice\u2013a preliminary analysis of two multinational companies. In International Conference on Product-Focused Software Process Improvement. Springer, 323\u2013330."},{"key":"e_1_3_2_1_19_1","volume-title":"What do we know about agile software development?IEEE software 26, 5","author":"Dyba Tore","year":"2009","unstructured":"Tore Dyba and Torgeir Dingsoyr. 2009. What do we know about agile software development?IEEE software 26, 5 (2009), 6\u20139."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/RELENG.2015.10"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1885"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2013.25"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.06.063"},{"key":"e_1_3_2_1_24_1","volume-title":"N., J. Humble","author":"Forsgren Nicole","year":"2016","unstructured":"Nicole Forsgren and Jez Humble. 2016. The role of continuous delivery in IT and organizational performance. Forsgren, N., J. Humble (2016).\" The Role of Continuous Delivery in IT and Organizational Performance.\" In the Proceedings of the Western Decision Sciences Institute (WDSI) (2016)."},{"key":"e_1_3_2_1_25_1","unstructured":"Martin Fowler and Matthew Foemmel. 2006. Continuous integration."},{"key":"e_1_3_2_1_26_1","volume-title":"Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and software technology 106","author":"Garousi Vahid","year":"2019","unstructured":"Vahid Garousi, Michael Felderer, and Mika\u00a0V M\u00e4ntyl\u00e4. 2019. Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information and software technology 106 (2019), 101\u2013121."},{"key":"e_1_3_2_1_27_1","volume-title":"Requirements management in DevOps environments: a multivocal mapping study. Requirements Engineering","author":"Hern\u00e1ndez Rogelio","year":"2023","unstructured":"Rogelio Hern\u00e1ndez, Bego\u00f1a Moros, and Joaqu\u00edn Nicol\u00e1s. 2023. Requirements management in DevOps environments: a multivocal mapping study. Requirements Engineering (2023), 1\u201330."},{"key":"e_1_3_2_1_28_1","unstructured":"Jez Humble and David Farley. 2010. Continuous delivery: reliable software releases through build test and deployment automation. Pearson Education."},{"key":"e_1_3_2_1_29_1","volume-title":"Accelerate: The science of lean software and devops: Building and scaling high performing technology organizations. IT Revolution.","author":"Humble Jez","year":"2018","unstructured":"Jez Humble and Gene Kim. 2018. Accelerate: The science of lean software and devops: Building and scaling high performing technology organizations. IT Revolution."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2010.12.002"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2372233.2372235"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10726-010-9192-8"},{"key":"e_1_3_2_1_33_1","volume-title":"Human and organizational factors in computer and information security: Pathways to vulnerabilities. Computers & security 28, 7","author":"Kraemer Sara","year":"2009","unstructured":"Sara Kraemer, Pascale Carayon, and John Clem. 2009. Human and organizational factors in computer and information security: Pathways to vulnerabilities. Computers & security 28, 7 (2009), 509\u2013520."},{"key":"e_1_3_2_1_34_1","unstructured":"JANE\u00a0LAUDON LAUDON. 2017. PEARSON MYLAB MIS WITH PEARSON ETEXT-INSTANTACCESS-MANAGEMENT INFORMATION SYSTEMS:... Managing the Digital Firm Global Edition. PEARSON EDUCATION LIMITED."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2016.10.001"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.01.015"},{"key":"e_1_3_2_1_37_1","volume-title":"Identifying and using critical success factors. Long range planning 17, 1","author":"Leidecker K","year":"1984","unstructured":"Joel\u00a0K Leidecker and Albert\u00a0V Bruno. 1984. Identifying and using critical success factors. Long range planning 17, 1 (1984), 23\u201332."},{"key":"e_1_3_2_1_38_1","volume-title":"Proceedings, Part VI 20","author":"Lohrasbinasab Iraj","year":"2020","unstructured":"Iraj Lohrasbinasab, Prameet\u00a0Bhakta Acharya, and Ricardo Colomo-Palacios. 2020. BizDevOps: a multivocal literature review. In Computational Science and Its Applications\u2013ICCSA 2020: 20th International Conference, Cagliari, Italy, July 1\u20134, 2020, Proceedings, Part VI 20. Springer, 698\u2013713."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3239235.3240299"},{"key":"e_1_3_2_1_40_1","volume-title":"Towards DevOps in the embedded systems domain: Why is it so hard?. In 2016 49th hawaii international conference on system sciences (hicss)","author":"Lwakatare Lucy\u00a0Ellen","unstructured":"Lucy\u00a0Ellen Lwakatare, Teemu Karvonen, Tanja Sauvola, Pasi Kuvaja, Helena\u00a0Holmstr\u00f6m Olsson, Jan Bosch, and Markku Oivo. 2016. Towards DevOps in the embedded systems domain: Why is it so hard?. In 2016 49th hawaii international conference on system sciences (hicss). IEEE, 5437\u20135446."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA51224.2020.00046"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2016.09.001"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/QRS51102.2020.00064"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSECP.2004.1281254"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/1395083.1395657"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ARES.2016.92"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67383-7_2"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-33515-5_11"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2804371.2804373"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2009.03.036"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-008-9065-9"},{"key":"e_1_3_2_1_52_1","volume-title":"Education: 12th SIGSAND\/PLAIS EuroSymposium 2019, Gdansk, Poland, September 19, 2019, Proceedings 12","author":"Prates Lu\u00eds","year":"2019","unstructured":"Lu\u00eds Prates, Jo\u00e3o Faustino, Miguel Silva, and R\u00faben Pereira. 2019. Devsecops metrics. In Information Systems: Research, Development, Applications, Education: 12th SIGSAND\/PLAIS EuroSymposium 2019, Gdansk, Poland, September 19, 2019, Proceedings 12. Springer, 77\u201390."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49094-6_44"},{"key":"e_1_3_2_1_54_1","volume-title":"Chief executives define their own data needs.Harvard business review 57, 2","author":"Rockart F","year":"1979","unstructured":"John\u00a0F Rockart. 1979. Chief executives define their own data needs.Harvard business review 57, 2 (1979), 81\u201393."},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.12.015"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3284179.3284328"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2811681.2824996"},{"key":"e_1_3_2_1_58_1","volume-title":"Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices","author":"Shahin Mojtaba","year":"2017","unstructured":"Mojtaba Shahin, Muhammad\u00a0Ali Babar, and Liming Zhu. 2017. Continuous integration, delivery and deployment: a systematic review on approaches, tools, challenges and practices. IEEE access 5 (2017), 3909\u20133943."},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-18612-2_14"},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106612"},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2591062.2591186"},{"key":"e_1_3_2_1_62_1","volume-title":"28th International Conference of Information Systems Development (ISD","author":"Van\u00a0Belzen Michiel","year":"2019","unstructured":"Michiel Van\u00a0Belzen, Jos Trienekens, and Rob Kusters. 2019. Critical success factors of continuous practices in a DevOps context. In 28th International Conference of Information Systems Development (ISD 2019)."},{"key":"e_1_3_2_1_63_1","volume-title":"European Conference on Service-Oriented and Cloud Computing. Springer, 108\u2013122","author":"Wettinger Johannes","year":"2014","unstructured":"Johannes Wettinger, Uwe Breitenb\u00fccher, and Frank Leymann. 2014. Devopslang\u2013bridging the gap between development and operations. In European Conference on Service-Oriented and Cloud Computing. Springer, 108\u2013122."},{"key":"e_1_3_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1080\/0960085X.2020.1782277"},{"key":"e_1_3_2_1_65_1","volume-title":"Typology and success factors of collaboration for sustainable growth in the IT service industry. Sustainability 9, 11","author":"Yoon Changbyung","year":"2017","unstructured":"Changbyung Yoon, Keeeun Lee, Byungun Yoon, and Omar Toulan. 2017. Typology and success factors of collaboration for sustainable growth in the IT service industry. Sustainability 9, 11 (2017)."},{"key":"e_1_3_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.81"}],"event":{"name":"EASE 2024: 28th International Conference on Evaluation and Assessment in Software Engineering","location":"Salerno Italy","acronym":"EASE 2024"},"container-title":["Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3661167.3661236","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3661167.3661236","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T11:13:10Z","timestamp":1755861190000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3661167.3661236"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,18]]},"references-count":66,"alternative-id":["10.1145\/3661167.3661236","10.1145\/3661167"],"URL":"https:\/\/doi.org\/10.1145\/3661167.3661236","relation":{},"subject":[],"published":{"date-parts":[[2024,6,18]]},"assertion":[{"value":"2024-06-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}